`itertools._grouper.__next__` has the same re-entrant use-after-free that was fixed in `groupby.__next__`

[devdanzin]

Crash report

What happened?

AI Disclosure: this issue, including the reproducer code, has been drafted by an LLM.

Bug description

gh-143543 / commit a91b5c3 fixed a re-entrant use-after-free in groupby_next by snapshotting gbo->tgtkey and gbo->currkey with Py_INCREF before calling PyObject_RichCompareBool. However, the sibling function _grouper_next (the inner group iterator) has the exact same unprotected comparison at Modules/itertoolsmodule.c:681:

rcmp = PyObject_RichCompareBool(igo->tgtkey, gbo->currkey, Py_EQ);

Neither igo->tgtkey nor gbo->currkey is held with a strong reference during the comparison. A user-defined __eq__ that re-enters the _grouper iterator can trigger groupby_step, which calls Py_XSETREF(gbo->currkey, newkey) — freeing the old currkey while PyObject_RichCompareBool still holds a dangling pointer to it as local variable w. When __eq__ returns NotImplemented, do_richcompare tries the reverse comparison w.__eq__(v) on the freed object.

Reproducer

Crashes with a segfault:

import itertools

grouper_iter = None

class Key:
    __hash__ = None

    def __init__(self, do_advance):
        self.do_advance = do_advance
        self.payload = bytearray(256)

    def __eq__(self, other):
        if self.do_advance:
            self.do_advance = False
            if grouper_iter is not None:
                try:
                    next(grouper_iter)
                except StopIteration:
                    pass
            for _ in range(50):
                bytearray(256)
            return NotImplemented
        return True

def keyfunc(element):
    if element == 0:
        return Key(do_advance=True)
    return Key(do_advance=False)

g = itertools.groupby(range(4), keyfunc)
key, grouper_iter = next(g)
items = list(grouper_iter)  # segfault

Backtrace

Program received signal SIGSEGV, Segmentation fault.
do_richcompare (tstate=0x555555d6cfd8 <_PyRuntime+405272>, v=0x20002542ba0, w=0x200025428a0, op=2) at Objects/object.c:1065
warning: Source file is more recent than executable.
1065            Py_DECREF(res);

#0  do_richcompare (tstate=0x555555d6cfd8 <_PyRuntime+405272>, v=0x20002542ba0, w=0x200025428a0, op=2) at Objects/object.c:1065
#1  PyObject_RichCompare (v=v@entry=0x20002542ba0, w=w@entry=0x200025428a0, op=op@entry=2) at Objects/object.c:1109
#2  0x000055555573deca in PyObject_RichCompareBool (v=0x20002542ba0, w=0x200025428a0, op=op@entry=2) at Objects/object.c:1131
#3  0x0000555555a32c03 in _grouper_next (op=0x20002151c80) at ./Modules/itertoolsmodule.c:681
#4  0x00005555556f2805 in list_extend_iter_lock_held (self=self@entry=0x200029bd4b0, iterable=iterable@entry=0x20002151c80) at Objects/listobject.c:1318
#5  0x00005555556ed858 in _list_extend (self=self@entry=0x200029bd4b0, iterable=iterable@entry=0x20002151c80) at Objects/listobject.c:1507
#6  0x00005555556f7ba8 in list___init___impl (self=self@entry=0x200029bd4b0, iterable=0x20002151c80) at Objects/listobject.c:3541
#7  0x00005555556f0c46 in list_vectorcall (type=type@entry=0x555555cd8ef8 <PyList_Type>, args=args@entry=0x7fffffffa6c8, nargsf=nargsf@entry=9223372036854775809, kwnames=kwnames@entry=0x0)
    at Objects/listobject.c:3565
#8  0x000055555569f1de in _PyObject_VectorcallTstate (tstate=0x555555d6cfd8 <_PyRuntime+405272>, callable=0x555555cd8ef8 <PyList_Type>, args=0x7fffffffa6c8, nargsf=9223372036854775809,
    kwnames=0x0) at ./Include/internal/pycore_call.h:136
#9  0x0000555555869fc1 in _Py_VectorCallInstrumentation_StackRefSteal (callable=..., arguments=0x555555da57d0, total_args=total_args@entry=1, kwnames=kwnames@entry=...,
    call_instrumentation=false, frame=0x555555da5770, this_instr=0x200023e8f3a, tstate=0x555555d6cfd8 <_PyRuntime+405272>) at Python/ceval.c:770
#10 0x0000555555874b23 in _PyEval_EvalFrameDefault (tstate=0x555555d6cfd8 <_PyRuntime+405272>, frame=<optimized out>, throwflag=0) at Python/generated_cases.c.h:1842
#11 0x0000555555869779 in _PyEval_EvalFrame (tstate=0x555555d6cfd8 <_PyRuntime+405272>, frame=0x555555da5770, throwflag=0) at ./Include/internal/pycore_ceval.h:118
#12 _PyEval_Vector (tstate=tstate@entry=0x555555d6cfd8 <_PyRuntime+405272>, func=func@entry=0x200024c3e30, locals=locals@entry=0x2000270cbb0, args=args@entry=0x0,
    argcount=argcount@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:2133
#13 0x0000555555869292 in PyEval_EvalCode (co=co@entry=0x200023e8dd0, globals=globals@entry=0x2000270cbb0, locals=locals@entry=0x2000270cbb0) at Python/ceval.c:681
#14 0x0000555555977e7a in run_eval_code_obj (tstate=tstate@entry=0x555555d6cfd8 <_PyRuntime+405272>, co=co@entry=0x200023e8dd0, globals=globals@entry=0x2000270cbb0,
    locals=locals@entry=0x2000270cbb0) at Python/pythonrun.c:1366
#15 0x0000555555977b39 in run_mod (mod=mod@entry=0x200027e6508, filename=filename@entry=0x20002235290, globals=globals@entry=0x2000270cbb0, locals=locals@entry=0x2000270cbb0,
    flags=0x7fffffffb9f0, arena=arena@entry=0x200020508b0, interactive_src=0x0, generate_new_source=0) at Python/pythonrun.c:1469

Root cause

_grouper_next at line 681 passes igo->tgtkey and gbo->currkey to PyObject_RichCompareBool without incrementing their refcounts. PyObject_RichCompareBool → PyObject_RichCompare → do_richcompare does not INCREF the arguments. When __eq__ re-enters the _grouper iterator:

1. Inner _grouper_next calls groupby_step
2. groupby_step calls Py_XSETREF(gbo->currkey, newkey) — this decrefs the old currkey to refcount 0, freeing it
3. __eq__ returns NotImplemented
4. do_richcompare tries the reverse: w.__eq__(v) where w is the freed currkey — use-after-free

Suggested fix

Apply the same fix from groupby_next (commit a91b5c3) to _grouper_next:

// Before (vulnerable):
rcmp = PyObject_RichCompareBool(igo->tgtkey, gbo->currkey, Py_EQ);

// After (safe):
PyObject *tgtkey = igo->tgtkey;
PyObject *currkey = gbo->currkey;
Py_INCREF(tgtkey);
Py_INCREF(currkey);
int rcmp = PyObject_RichCompareBool(tgtkey, currkey, Py_EQ);
Py_DECREF(tgtkey);
Py_DECREF(currkey);

Versions affected

Same as gh-143543 — all versions with itertools.groupby, regardless of GIL or JIT being enabled. The fix for groupby_next was applied to 3.13 and 3.14; this sibling function needs the same treatment, besides also hitting 3.15.

Found using cpython-review-toolkit.

CPython versions tested on:

CPython main branch, 3.15, 3.14

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.15.0a6+ free-threading build (heads/main:592e8f0865c, Mar 2 2026, 16:35:36) [Clang 21.1.2 (2ubuntu6)]

[picnixz]

This is a dupe of #145678 I believe and we had issues with the reproducer... I'm going to close this one for now, but please check that the reproducer reproduces.

[devdanzin]

Thank you, sorry for not checking first :(

I can confirm the reproducer here works across Python versions and build options on my machine and the one in #145678 doesn't.

[picnixz]

Cool. So I'll keep yours open and close the other actually. That way, we can have a cleaner history.