Region Bugs

[xFrednet]

On regions-main

1. Immutability Bug: Moved: Immutability Implementation Adjustments for PLDI and the PEP #64 (comment)

[xFrednet]

Resolved

from immutable import freeze
from regions import Region


class A:
    pass


def build_region():
    r = Region()
    r.a = A()
    r.a.b = A()
    r.a = None
    return r


freeze(A)
r1 = build_region()
r2 = build_region()

-> Python/region.c:2306: _PyRegion_RemoveRef: Assertion `!HAS_DATA(src_region) || regiondata_is_dirty(src_region)' failed.

Credits to David for finding this

[xFrednet]

Segfault in GC discovered by CI:

Segfault from https://github.com/fxpl/cpython/actions/runs/23865394489/job/69583048904

Fatal Python error: Segmentation fault

Current thread 0x00007f91db829740 [python] (most recent call first):
  Garbage-collecting
  File "/home/runner/work/cpython/cpython/Lib/test/support/__init__.py", line 847 in gc_collect
  File "/home/runner/work/cpython/cpython/Lib/test/libregrtest/single.py", line 171 in _load_run_test
  File "/home/runner/work/cpython/cpython/Lib/test/libregrtest/single.py", line 210 in _runtest_env_changed_exc
  File "/home/runner/work/cpython/cpython/Lib/test/libregrtest/single.py", line 319 in _runtest
  File "/home/runner/work/cpython/cpython/Lib/test/libregrtest/single.py", line 348 in run_single_test
  File "/home/runner/work/cpython/cpython/Lib/test/libregrtest/worker.py", line 92 in worker_process
  File "/home/runner/work/cpython/cpython/Lib/test/libregrtest/worker.py", line 127 in main
  File "/home/runner/work/cpython/cpython/Lib/test/libregrtest/worker.py", line 131 in <module>
  File "/home/runner/work/cpython/cpython/Lib/runpy.py", line 88 in _run_code
  File "/home/runner/work/cpython/cpython/Lib/runpy.py", line 198 in _run_module_as_main

Current thread's C stack trace (most recent call first):
  Binary file "/home/runner/work/cpython/cpython/python", at _Py_DumpStack+0x46 [0x557ecf6f2fb0]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x357b88 [0x557ecf711b88]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x35832f [0x557ecf71232f]
  Binary file "/lib/x86_64-linux-gnu/libc.so.6", at +0x45330 [0x7f91db645330]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x347d66 [0x557ecf701d66]
  Binary file "/home/runner/work/cpython/cpython/python", at _PyRegion_RemoveRef+0x38 [0x557ecf702a25]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x1853c9 [0x557ecf53f3c9]
  Binary file "/home/runner/work/cpython/cpython/python", at PyDict_Clear+0x33 [0x557ecf53f4b1]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x18550e [0x557ecf53f50e]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x2d3eeb [0x557ecf68deeb]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x2d422e [0x557ecf68e22e]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x2d4d17 [0x557ecf68ed17]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x2d5707 [0x557ecf68f707]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x34d6d2 [0x557ecf7076d2]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x34d7a6 [0x557ecf7077a6]
  Binary file "/home/runner/work/cpython/cpython/python", at _PyEval_EvalFrameDefault+0x6b55 [0x557ecf63d461]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x29fa61 [0x557ecf659a61]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x29fc3e [0x557ecf659c3e]
  Binary file "/home/runner/work/cpython/cpython/python", at PyEval_EvalCode+0xcc [0x557ecf659d44]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x273488 [0x557ecf62d488]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x2735af [0x557ecf62d5af]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x1909b3 [0x557ecf54a9b3]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x1348cf [0x557ecf4ee8cf]
  Binary file "/home/runner/work/cpython/cpython/python", at PyObject_Vectorcall+0x27 [0x557ecf4ee9ab]
  Binary file "/home/runner/work/cpython/cpython/python", at _PyEval_EvalFrameDefault+0x497f [0x557ecf63b28b]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x29fa61 [0x557ecf659a61]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x29fc3e [0x557ecf659c3e]
  Binary file "/home/runner/work/cpython/cpython/python", at _PyFunction_Vectorcall+0x58 [0x557ecf4ee560]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x1361df [0x557ecf4f01df]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x1364f3 [0x557ecf4f04f3]
  Binary file "/home/runner/work/cpython/cpython/python", at PyObject_Call+0x24 [0x557ecf4f052f]
  Binary file "/home/runner/work/cpython/cpython/python", at +0x34b9c4 [0x557ecf7059c4]
  <truncated rest of calls>
1 test failed again:
    test_asyncio.test_selector_events

== Tests result: FAILURE then FAILURE ==

Guess from Co-Pilot I'm a bit sceptical about

Findings from the last commit in PR 93, prioritized by likely impact on your GC-time crash path:

1. High: mixed pending/non-pending region lookup inside remove-ref path can corrupt LRC bookkeeping

• In region.c, _PyRegion_RemoveRef first computes src and target with follow-pending semantics, but the local-source branch later decrements LRC using a fresh non-follow-pending lookup of target.
• Relevant points:
  • region.c and region.c: follow-pending regions are computed.
  • region.c: local branch uses a non-follow-pending target lookup for regiondata_dec_lrc.
• Why this is risky:
  • If target is in a merge-pending state, you can decrement the wrong region or hit invalid owner/tag assumptions during GC teardown.
  • This aligns with your stack signature where crash happens in remove-ref during dict clear.
• Minimal fix direction:
  • Use one canonical target region value throughout _PyRegion_RemoveRef, and in local branch decrement that resolved region directly instead of re-querying target with different semantics.

2. High: dict clear loop is re-entrancy fragile while decrefing values

• In dictobject.c, clear_lock_held iterates old split-table values and for each entry does remove-ref then Py_CLEAR.
• Key lines:
  • dictobject.c: remove-ref on old value.
  • dictobject.c: Py_CLEAR can run arbitrary finalizers.
• Why this is risky:
  • During Py_CLEAR, Python-level cleanup can re-enter and mutate the dict while clear_lock_held is still iterating old storage.
  • That can produce stale/double-cleared value slots and feed bad pointers into remove-ref on later iterations.
• This is a plausible trigger for intermittent segfaults in GC-heavy tests like asyncio selector-events.

3. Medium: dict dealloc also decrefs live slots without first detaching/nulling container state

• In dictobject.c, dict_dealloc walks values and decrefs each item directly.
• Relevant lines:
  • dictobject.c: remove-ref per value.
  • dictobject.c: Py_XDECREF per value.
• Why this is risky:
  • Finalizers during dealloc can observe partially torn-down dict internals.
  • Combined with region accounting calls, this increases chance of use-after-state-corruption rather than a clean assertion failure.

Assumptions and scope:

• I reviewed this against your local checkout content corresponding to the PR commit, not a line-by-line GitHub rendered diff for every large file section.
• The analysis is focused on the exact crash corridor you reported: GC -> dict clear/dealloc -> _PyRegion_RemoveRef.

If you want, I can prepare a minimal patch that only:

1. makes _PyRegion_RemoveRef use a single resolved target region consistently, and
2. hardens clear_lock_held by detaching dict state before value decrefs to prevent re-entrancy corruption.