Django is synchronous by nature. It was not built for an asynchronous system. However, the team is making an effort to bring true asynchronous support to Django. In the meantime, how do we make Django work async?
Note how I mentioned “true” asynchronous support. Currently, Django does support some form of asynchronous workflows, but the core part of the ORM (connecting to the DB) is still synchronous.
Why do we need async workflows in the first place?
If you are creating a RAG chatbot, calling an LLM api in a synchronous workflow will block the worker for an extremely long time. Another option would be to use celery for this, but that just adds unnecessary complications.
So how do we make it asynchronous?
A basic checklist would comprise of:
- Switch to an ASGI server
- Start using the async ORM from Django
- Stop using any sync middleware
- Stop using DRF
Sounds easy right? This will work, but the moment you get any load, it will start showing its cracks.
The big problem
In Django, connections are normally reused, but only per thread. So If you have multiple workers running, multiple celery workers and long running LLM calls, then you end up with a ton of open connections to your DB.
The thing is, DB connections are expensive. In terms of memory. In a postgres server, 1 connection can use 10 Mb of RAM. Which is extremely resource intensive. Normally, a Postgres server limits the number of connections to 20, maybe 50. So under load, you are extremely likely to hit those limits, and your APIs will start to fail.
How do we fix this?
The answer is a connection pooler like PgBouncer. PgBouncer acts as an interface to the DB. Django can make 1000s of connections to PgBouncer, but PgBouncer routes it through the 20 odd connections you have to Postgres.
This works because, a connection to PgBouncer is cheap. Each connection is only 2Kb of overhead. This means you can easily have 1000s of connections to PgBouncer.
With this change your endpoint can now handle a much larger concurrent load.
While Django might add truly async support, people might still face the exact same issue, as the problem mainly arises when we use run multiple workers, or add celery.