Per-interpreter GIL in mod_wsgi 6.0.0 - Graham Dumpleton

mod_wsgi 6.0.0 is available as a release candidate with code cleanup, new features, and operator-facing improvements. The release adds new concurrency configuration options that align with CPython’s new concurrency modes. CPython introduces per-interpreter GIL in Python 3.12 and free-threading in Python 3.13, and mod_wsgi exposes both as opt-in directives. It also provides finer-grained control over how the GIL switches between threads. The deployment problem comes from the Global Interpreter Lock, which serializes Python bytecode execution within a process regardless of OS thread count. Threads help with I/O concurrency when extensions release the GIL, but CPU-bound Python work requires multiple processes. Sub-interpreters have historically shared a single process-wide GIL until PEP 684 in Python 3.12.

"mod_wsgi 6.0.0 is currently available as a release candidate. You can install it from PyPI, or grab the source from the GitHub releases page. There is a significant amount of code cleanup behind this release, alongside a range of new features and operator-facing improvements that have been overdue for some time."

"CPython's Global Interpreter Lock serialises Python bytecode execution within a single process. It does not matter how many OS threads you create inside that process. Only one of them runs Python at a time. For WSGI deployments, this has shaped the way servers like mod_wsgi scale. Threads within a single process are useful for handling I/O concurrently, since any reasonable C extension or built-in I/O call releases the GIL while it waits on the kernel, but they do not give you parallelism for CPU-bound Python work."

"To get that, you have always needed more processes. mod_wsgi's daemon mode is built around this assumption. You configure N daemon processes, each with its own Python interpreter and its own GIL, and you get N-way Python parallelism that way. Sub-interpreters complicate the picture slightly. They have existed in CPython for a long time, and mod_wsgi has used them since the beginning, but until PEP 684 landed in Python 3.12 they all shared one process-wide GIL."

"CPython has gained two genuinely new concurrency modes over the last few releases (per-interpreter GIL in 3.12 and free-threading in 3.13), and mod_wsgi 6.0.0 exposes both as opt-in directives, along with finer-grained control over how the GIL switches between threads. This first post covers the per-interpreter GIL story and the new WSGIPerInterpreterGIL directive."

Read at Grahamdumpleton