Fix Python < 3.11 crashes during Py_FinalizeEx (uWSGI worker recycling)

Three root causes of SIGSEGV during interpreter shutdown on Python < 3.11
were identified through uWSGI worker recycling stress tests and fixed:

1. ThreadState allocated via PyObject_Malloc — pymalloc pools can be
   disrupted during early Py_FinalizeEx, corrupting the C++ object.
   Switched to std::malloc/std::free.

2. deleteme vector used PythonAllocator (PyMem_Malloc) — same pool
   corruption issue.  Switched to std::allocator (system malloc).

3. _Py_IsFinalizing() is only set AFTER call_py_exitfuncs and
   _PyGC_CollectIfEnabled complete, so atexit handlers and __del__
   methods could call greenlet.getcurrent() when type objects were
   already invalidated, crashing in PyType_IsSubtype.  An atexit
   handler registered at module init (LIFO = runs first) now sets
   a shutdown flag checked by getcurrent(), PyGreenlet_GetCurrent(),
   and clear_deleteme_list().

All new code is guarded by #if !GREENLET_PY311 — zero impact on
Python 3.11+.  Verified: 0 segfaults in 200 uWSGI worker recycles
on Python 3.9.7 (previously 3-4 crashes in 30 recycles).

Made-with: Cursor

Author: nbouvrette

CHANGES.rst

@@ -5,17 +5,38 @@
 3.3.3 (unreleased)
 ==================
 
-- Fix a second crash path during interpreter shutdown (observed on
-  Python < 3.11): ``clear_deleteme_list()`` could crash when its
-  ``std::vector`` copy allocated through ``PythonAllocator``
-  (``PyMem_Malloc``) during early ``Py_FinalizeEx`` cleanup (before
-  ``_Py_IsFinalizing()`` is set).  The vector contents are now moved
-  with ``std::swap`` (zero-allocation, constant-time) instead of
-  copied.  Additionally, ``clear_deleteme_list()`` now preserves any
-  pending Python exception around its cleanup loop, fixing a latent
-  bug where an unrelated exception (e.g. one set by ``throw()``) could
-  be swallowed by ``PyErr_WriteUnraisable`` / ``PyErr_Clear`` inside
-  the loop.  This is distinct from the dealloc crash fixed in 3.3.2
+- Fix multiple crash paths during interpreter shutdown on Python < 3.11
+  (observed with uWSGI worker recycling).  Three root causes were
+  identified and fixed:
+
+  1. ``clear_deleteme_list()`` used a ``PythonAllocator``-backed vector
+     copy (``PyMem_Malloc``), which could SIGSEGV during early
+     ``Py_FinalizeEx`` when Python's allocator pools are partially torn
+     down.  Replaced with ``std::swap`` (zero-allocation,
+     constant-time) and switched the ``deleteme`` vector to
+     ``std::allocator`` (system ``malloc``).
+
+  2. ``ThreadState`` objects were allocated via ``PyObject_Malloc``,
+     placing them in ``pymalloc`` pools that can be disrupted during
+     finalization.  Switched to ``std::malloc`` / ``std::free`` so
+     ``ThreadState`` memory remains valid throughout ``Py_FinalizeEx``.
+
+  3. ``_Py_IsFinalizing()`` is only set *after* ``call_py_exitfuncs``
+     and ``_PyGC_CollectIfEnabled`` complete inside ``Py_FinalizeEx``,
+     so code in atexit handlers or ``__del__`` methods could still call
+     ``greenlet.getcurrent()`` when type objects had already been
+     invalidated, crashing in ``PyType_IsSubtype``.  An atexit handler
+     is now registered at module init (LIFO = runs first) that sets a
+     shutdown flag checked by ``getcurrent()``,
+     ``PyGreenlet_GetCurrent()``, and ``clear_deleteme_list()``.
+
+  Additionally, ``clear_deleteme_list()`` now preserves any pending
+  Python exception around its cleanup loop, fixing a latent bug where
+  an unrelated exception (e.g. one set by ``throw()``) could be
+  swallowed by ``PyErr_WriteUnraisable`` / ``PyErr_Clear`` inside the
+  loop.
+
+  This is distinct from the dealloc crash fixed in 3.3.2
   (`PR #495
   <https://github.com/python-greenlet/greenlet/pull/495>`_). See `PR #499
   <https://github.com/python-greenlet/greenlet/pull/499>`_ by Nicolas

src/greenlet/CObjects.cpp

@@ -29,6 +29,11 @@ extern "C" {
 static PyGreenlet*
 PyGreenlet_GetCurrent(void)
 {
+#if !GREENLET_PY311
+    if (g_greenlet_shutting_down || _Py_IsFinalizing()) {
+        return nullptr;
+    }
+#endif
     return GET_THREAD_STATE().state().get_current().relinquish_ownership();
 }
 

src/greenlet/PyModule.cpp

@@ -17,6 +17,30 @@ using greenlet::ThreadState;
 #    pragma clang diagnostic ignored "-Wunused-variable"
 #endif
 
+// On Python < 3.11, _Py_IsFinalizing() is only set AFTER
+// call_py_exitfuncs and _PyGC_CollectIfEnabled finish inside
+// Py_FinalizeEx.  Code running in atexit handlers or __del__
+// methods can still call greenlet.getcurrent(), but by that
+// time type objects may have been invalidated, causing
+// SIGSEGV in PyType_IsSubtype.  This flag is set by an atexit
+// handler registered at module init (LIFO = runs first).
+#if !GREENLET_PY311
+int g_greenlet_shutting_down = 0;
+
+static PyObject*
+_greenlet_atexit_callback(PyObject* UNUSED(self), PyObject* UNUSED(args))
+{
+    g_greenlet_shutting_down = 1;
+    Py_RETURN_NONE;
+}
+
+static PyMethodDef _greenlet_atexit_method = {
+    "_greenlet_cleanup", _greenlet_atexit_callback,
+    METH_NOARGS, NULL
+};
+#endif
+
+
 PyDoc_STRVAR(mod_getcurrent_doc,
              "getcurrent() -> greenlet\n"
              "\n"
@@ -26,6 +50,11 @@ PyDoc_STRVAR(mod_getcurrent_doc,
 static PyObject*
 mod_getcurrent(PyObject* UNUSED(module))
 {
+#if !GREENLET_PY311
+    if (g_greenlet_shutting_down || _Py_IsFinalizing()) {
+        Py_RETURN_NONE;
+    }
+#endif
     return GET_THREAD_STATE().state().get_current().relinquish_ownership_o();
 }
 

src/greenlet/TThreadState.hpp

@@ -1,6 +1,7 @@
 #ifndef GREENLET_THREAD_STATE_HPP
 #define GREENLET_THREAD_STATE_HPP
 
+#include <cstdlib>
 #include <ctime>
 #include <stdexcept>
 #include <atomic>
@@ -23,6 +24,13 @@ using greenlet::refs::CreatedModule;
 using greenlet::refs::PyErrPieces;
 using greenlet::refs::NewReference;
 
+// Defined in PyModule.cpp; set by an atexit handler to signal
+// that the interpreter is shutting down.  Only needed on
+// Python < 3.11 where _Py_IsFinalizing() is set too late.
+#if !GREENLET_PY311
+extern int g_greenlet_shutting_down;
+#endif
+
 namespace greenlet {
 /**
  * Thread-local state of greenlets.
@@ -100,7 +108,13 @@ class ThreadState {
     /* Strong reference to the trace function, if any. */
     OwnedObject tracefunc;
 
-    typedef std::vector<PyGreenlet*, PythonAllocator<PyGreenlet*> > deleteme_t;
+    // Use std::allocator (malloc/free) instead of PythonAllocator
+    // (PyMem_Malloc) for the deleteme list.  During Py_FinalizeEx on
+    // Python < 3.11, the PyObject_Malloc pool that holds ThreadState
+    // can be disrupted, corrupting any PythonAllocator-backed
+    // containers.  Using std::allocator makes this vector independent
+    // of Python's allocator lifecycle.
+    typedef std::vector<PyGreenlet*> deleteme_t;
     /* A vector of raw PyGreenlet pointers representing things that need
        deleted when this thread is running. The vector owns the
        references, but you need to manually INCREF/DECREF as you use
@@ -120,7 +134,6 @@ class ThreadState {
     static std::clock_t _clocks_used_doing_gc;
 #endif
     static ImmortalString get_referrers_name;
-    static PythonAllocator<ThreadState> allocator;
 
     G_NO_COPIES_OF_CLS(ThreadState);
 
@@ -146,15 +159,21 @@ class ThreadState {
 
 
 public:
-    static void* operator new(size_t UNUSED(count))
+    // Allocate ThreadState with malloc/free rather than Python's object
+    // allocator.  ThreadState outlives many Python objects and must
+    // remain valid throughout Py_FinalizeEx.  On Python < 3.11,
+    // PyObject_Malloc pools can be disrupted during early finalization,
+    // corrupting any C++ objects stored in them.
+    static void* operator new(size_t count)
     {
-        return ThreadState::allocator.allocate(1);
+        void* p = std::malloc(count);
+        if (!p) throw std::bad_alloc();
+        return p;
     }
 
     static void operator delete(void* ptr)
     {
-        return ThreadState::allocator.deallocate(static_cast<ThreadState*>(ptr),
-                                                 1);
+        std::free(ptr);
     }
 
     static void init()
@@ -292,6 +311,26 @@ class ThreadState {
             deleteme_t copy;
             std::swap(copy, this->deleteme);
 
+            // During Py_FinalizeEx cleanup, the GC or atexit handlers
+            // may have already collected objects in this list, leaving
+            // dangling pointers.  Attempting Py_DECREF on freed memory
+            // causes a SIGSEGV.  On Python < 3.11,
+            // g_greenlet_shutting_down covers the early stages
+            // (before _Py_IsFinalizing() is set).
+#if !GREENLET_PY311
+            if (g_greenlet_shutting_down || _Py_IsFinalizing()) {
+                return;
+            }
+#elif GREENLET_PY313
+            if (Py_IsFinalizing()) {
+                return;
+            }
+#else
+            if (_Py_IsFinalizing()) {
+                return;
+            }
+#endif
+
             // Preserve any pending exception so that cleanup-triggered
             // errors don't accidentally swallow an unrelated exception
             // (e.g. one set by throw() before a switch).
@@ -538,7 +577,6 @@ class ThreadState {
 };
 
 ImmortalString ThreadState::get_referrers_name(nullptr);
-PythonAllocator<ThreadState> ThreadState::allocator;
 #ifdef Py_GIL_DISABLED
 std::atomic<std::clock_t> ThreadState::_clocks_used_doing_gc(0);
 #else

src/greenlet/greenlet.cpp

@@ -294,6 +294,41 @@ greenlet_internal_mod_init() noexcept
 #ifdef Py_GIL_DISABLED
         PyUnstable_Module_SetGIL(m.borrow(), Py_MOD_GIL_NOT_USED);
 #endif
+
+#if !GREENLET_PY311
+        // Register an atexit handler that sets g_greenlet_shutting_down.
+        // Python's atexit is LIFO: registered last = called first.  By
+        // registering here (at import time, after most other libraries),
+        // our handler runs before their cleanup code, which may try to
+        // call greenlet.getcurrent() on objects whose type has been
+        // invalidated.  _Py_IsFinalizing() alone is insufficient
+        // because it is only set AFTER call_py_exitfuncs completes.
+        {
+            PyObject* atexit_mod = PyImport_ImportModule("atexit");
+            if (atexit_mod) {
+                PyObject* register_fn = PyObject_GetAttrString(atexit_mod, "register");
+                if (register_fn) {
+                    extern PyMethodDef _greenlet_atexit_method;
+                    PyObject* callback = PyCFunction_New(&_greenlet_atexit_method, NULL);
+                    if (callback) {
+                        PyObject* args = PyTuple_Pack(1, callback);
+                        if (args) {
+                            PyObject* result = PyObject_Call(register_fn, args, NULL);
+                            Py_XDECREF(result);
+                            Py_DECREF(args);
+                        }
+                        Py_DECREF(callback);
+                    }
+                    Py_DECREF(register_fn);
+                }
+                Py_DECREF(atexit_mod);
+            }
+            // Non-fatal: if atexit registration fails, we still have
+            // the _Py_IsFinalizing() fallback.
+            PyErr_Clear();
+        }
+#endif
+
         return m.borrow(); // But really it's the main reference.
     }
     catch (const LockInitError& e) {