Skip to content

Could thin pool graceful cleanup thread be instantiated **not** on module import? #426

Closed
@stephen-hansen

Description

@stephen-hansen

Hi,

I'm looking at the following lines in python-oracledb:

# keep track of which pools need to be closed and ensure that they are closed
# gracefully when the main thread finishes its work
pools_to_close = set()
def close_pools_gracefully():
cdef ThinPoolImpl pool_impl
threading.main_thread().join() # wait for main thread to finish
for pool_impl in list(pools_to_close):
pool_impl._shutdown()
threading.Thread(target=close_pools_gracefully).start()

When pool.pyx gets imported (via thin_impl.pyx, and then in general __init__.py, I think), those lines start a background thread that blocks until the main thread is finished, and then cleans up everything in pools_to_close. Following the import chain above I believe this thread is created and starts running on import of python-oracledb itself, even if pools_to_close is never populated.

This is not ideal to me, I don't expect module imports to necessarily spawn threads, or cause other side effects for that matter, at import time. Just as an example, I was working on a process earlier where I want to block SIGUSR1 in every thread in my process. Even though I was setting my main thread to block SIGUSR1, I was doing this in my main() method, and the thread spawned by python-oracledb was created at import prior to my main method even running, so it was not blocking the signals that I had intended it to. For now I am getting around this by setting blocked signals prior to import.

I'm not sure if a separate thread is the best way to clean up the pools, but at least one immediate idea I had here to improve the code here was to only create the cleanup thread the first time either something is added to the pools_to_close, or you do it at either __init__ or __new__ time via e.g. a class singleton, so that the first time a ThinPoolImpl is constructed the cleanup thread is started as a property of the class. The key point here being that I would greatly prefer if the cleanup thread was created closer to usage, when needed. I'm open to other ideas though.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions