Issue in sync.py's SyncToAsync class as new ThreadPoolExecutor executors with daemon threads getting created for requests.

[sunilnitdgp]

Issue in sync.py's SyncToAsync class as new ThreadPoolExecutor executors with daemon threads getting created for requests.

[andrewgodwin]

Actually, it's normal for multiple executors to be created. The specific behaviour should be:

• If the call is marked as thread-sensitive, there's a single shared global executor that is used
• If the call is not marked as that, then the default behaviour of loop.run_in_executor is used, which I believe is to spawn a new one each time.
  • You can optionally override this by passing your own executor= argument into SyncToAsync, but this is rarely done

[sunilnitdgp]

@andrewgodwin Thanks for your comment. I'm using Django as a framework, and each call is explicitly marked as thread_sensitive=True using the default ASGI handler class. For instance:

await sync_to_async(signals.request_started.send, thread_sensitive=True)(
    sender=self.__class__, scope=scope
)

Considering this, ideally, a single shared global executor should be used. However, despite this setup, the code seems to be entering the else block, where a new thread executor is created in the current context instead of reusing the existing one:

if thread_sensitive_context in self.context_to_thread_executor:
    # Re-use thread executor in current context
    executor = self.context_to_thread_executor[thread_sensitive_context]
else:
    # Create new thread executor in current context
    executor = ThreadPoolExecutor(max_workers=1)
    self.context_to_thread_executor[thread_sensitive_context] = executor

[andrewgodwin]

Since all sorts of things that you can do in your app can affect that (different middleware, server configuration, loading in something that uses gevent, etc.), we can only really debug it if you can produce a reliable way to reproduce it in a tiny sample app, unfortunately.

[sunilnitdgp]

I understand. I'm actually working with a simple app containing just a view. Here are the steps I've followed:

1. Created a virtual environment and installed necessary packages (django, djangorestframework, adrf, aiohttp, daphne, asyncio).
2. Added daphne and adrf to INSTALLED_APPS in settings.py.
3. Created an async view in views.py and added a corresponding path in urls.py.
4. Run the server with python manage.py runserver (can see the log Starting ASGI/Daphne version 4.1.0 development server at http://127.0.0.1:8000/)
5. Utilized a benchmarking tool (in this case, ab) to test the API endpoint (ab -c 10 -n 600 -s 800007 -T application/json "http://127.0.0.1:8000/test").
6. Monitored the total number of threads created during benchmarking using top -o threads -pid <pid>.

I'm working on Mac M1 Air