freenode
Languages & Toolchains

Python pushed to document concurrency and memory models

Mark Shannon argues CPython still lacks written guarantees on atomicity and threading for GIL and free-threaded builds as the global lock is phased out.

CPython core developer Mark Shannon has called for Python to write down explicit concurrency and memory models for both traditional GIL builds and free-threaded ones, saying years of informal assumptions leave programmers without clear rules for concurrent code.

Despite earlier efforts such as PEP 583 and repeated questions on atomicity and thread safety, the language docs still omit concurrent access entirely. Python has an object model, Shannon noted on the discuss.python.org forum, but nothing on what happens when threads touch the same objects. The gap matters more as free-threading lands: code that quietly depended on the GIL serializing execution can fail in subtle ways once threads truly run side by side.

Shannon proposed starting with a practical outline for the GIL build: what the lock is, which common operations are atomic, how non-atomic behavior should be described, comparisons to models such as sequential consistency, and how much leeway other implementations may take. Details can be filled in later; the first need is a reasonably complete, unambiguous core.

Others pushed back on scope. Some argued that GIL semantics are CPython implementation details, not language requirements, and that energy is better spent helping code stop relying on the lock. Atomicity, they noted, is often an accident of a particular builtin method rather than a language guarantee; even setdefault on dict can race once subclasses or application state enter the picture. Java was cited as proof a high-level language can still make strong, implementation-neutral promises, while others insisted Python should not frame GIL-specific rules as language law.

A contributor building a Python thread sanitizer shared draft models reverse-engineered from CPython: sequential consistency inside an interpreter under the GIL, release-acquire ordering for free-threaded builds, preemption only at defined safe points, and limited atomicity for simple compound updates on exact builtin numerics. Shannon replied that free-threading’s working goal is sequential consistency, not merely release-acquire, and that a wider set of operations (float and string arithmetic, plain attribute loads and stores, certain list and dict indexing, literal construction) are atomic under the GIL. He also warned the models are already complex enough that ordinary developers will get them wrong, and favored reducing legal context-switch points to simplify reasoning.

Existing free-threading documentation covers some thread-safety promises but skips classic memory-model topics such as happens-before and sequential consistency. Pure Python still has no portable equivalent of Java volatile or C++ memory orders for multi-access ordering. Whether the result should be a language-level model or separate CPython documents for the two builds remains open; the shared premise is that the current silence is no longer tenable.