Repository navigation
fix(reassembly): bypass copyreg's MRO walk for Deferred's protocol<2 pickling - #2435
Conversation
780ff6b to
1d529f0
Compare
|
Review: NEEDS CHANGES at
Also: the PR body's corekit count (866/7649) is actually the combined reassembly+corekit run, not corekit alone (693/6767) — fix the label. And since GraalPy was actually run against this fix (not just reasoned about), replace "GraalPy isn't available here" with the real result before this closes #2433. Confirmed correct and unaffected: protocol ≥2 is byte-identical to |
|
Coverage: 94.44% (unit tier, Python 3.15,
Per-file detail: the |
1d529f0 to
641331f
Compare
|
Review: NEEDS CHANGES at Only the new test file's docstrings need fixing — numbers borrowed from the wrong module, one self-contradiction, and a count that shouldn't be there per the owner's no-counted-numbers-in-docs ruling:
Fix: drop the counts (owner ruling, no counted numbers in docs/prose) and the "cannot be run" sentence; state plainly that it's confirmed on GraalPy 25.4.4 now that it has been. Non-blocking, noted for awareness only: |
…pickling copyreg._reduce_ex, which object.__reduce_ex__ hands protocol 0/1 to, walks cls.__mro__ for a base to compute state from, before __getstate__ is ever consulted. On CPython that walk passes through Generic (a heap type whose __new__ is just the inherited, differently-bound object.__new__) and stops at object (state=None); on GraalPy 25.4 Generic is itself a non-heap (built-in) type, so the walk's first check breaks there directly and calls Generic(self), raising "TypeError: typing.Generic() takes no arguments" -- independent of #2424/#2425's __getstate__/__setstate__ fix, since those only supply the dict half of a reduction the walk fails to even build. - Deferred.__reduce_ex__ now hard-codes base=object, state=None for protocol 0/1 -- the only outcome the walk ever reached for this class -- and builds the reduction tuple directly via copyreg._reconstructor, with __getstate__()'s own (dict_or_None, slots) pair restored through __setstate__ exactly as before. Protocol >= 2 defers to object.__reduce_ex__ unchanged. - It also checks for, and defers to, a subclass's own __reduce__ override before building that hard-coded tuple, matching what object.__reduce_ex__ itself already does for every protocol. Without the check, a Deferred subclass overriding __reduce__ had it silently dropped at protocol 0/1 while protocol 2+ (via object.__reduce_ex__ unchanged) kept honouring it -- a regression caught in cross-review, now covered by a new test alongside the GraalPy-failure regression test. - Dropping Generic[_KT] instead was considered and rejected: test_deferred_proto_type_2417_unit.py reads Deferred.__parameters__ at runtime, which only exists because Generic is a real base, not a TYPE_CHECKING-only annotation. No longer invokes copyreg._reduce_ex's MRO walk for protocol<2 at all, so Generic is no longer part of the route pickling takes. Confirmed on real GraalPy 25.4.4 by cross-review: test_deferred_pickle_2423_unit.py goes from 8 failed on main to 16 passed/18 subtests on this fix, and the full tests/foundation/reassembly/ suite passes 173/854 subtests, 0 failed. Build/test (CPython): mypy 98 errors/32 files (baseline, unchanged), pylint 10.00/10, isort clean; tests/foundation/reassembly (176 tests/890 subtests, including the two new regression tests) and tests/corekit (693 tests/6767 subtests, unaffected by this change) pass; 100% coverage on the changed file. Closes #2433
641331f to
b8ac765
Compare
|
Review: GOOD TO GO at Together with rounds 1–2's findings (both fixed and verified, including a real run on GraalPy 25.4.4), this PR is ready to merge. |
Please follow the guide below
make pylint,make mypy,make isort)make testpasses, and a test case covers the changedocs/source/changelog/and regeneratedCHANGELOG.mdWhat is the purpose of your pull request?
fix— corrects a defectDescription of your pull request and other information
Closes #2433.
Deferredstill failed to pickle under protocol 0/1 on GraalPy 25.4 even after #2424's__getstate__/__setstate__fix, withTypeError: typing.Generic() takes no argumentsfromcopyreg._reduce_ex.Root cause: for protocol < 2,
copyreg._reduce_exwalkscls.__mro__for a base to compute a base-object state from, before__getstate__is ever consulted. On CPython,Genericis a heap type whose__new__is just the inherited, differently-boundobject.__new__, so the walk passes it and stops atobject(state=None). On GraalPy,Genericis itself a non-heap (built-in) type, so the walk's first check (not base.__flags__ & _HEAPTYPE) breaks onGenericdirectly and callsGeneric(self), which raises.__getstate__/__setstate__never run in that failure, since the crash happens before the walk's result is combined with them.Fix:
Deferred.__reduce_ex__hard-codes thebase=object, state=Noneoutcome the walk always reached on CPython for protocol 0/1, bypassingcopyreg._reduce_ex's MRO walk (andGenericwith it) entirely. It also checks for, and defers to, a subclass's own__reduce__override before building that hard-coded tuple — a regression caught in cross-review: without the check, aDeferredsubclass overriding__reduce__had it silently dropped at protocol 0/1 while protocol 2+ (viaobject.__reduce_ex__, unchanged) kept honouring it. Protocol >= 2 is otherwise untouched. DroppingGeneric[_KT]instead was considered and rejected:test_deferred_proto_type_2417_unit.pyreadsDeferred.__parameters__at runtime, which only exists becauseGenericis a real base, notTYPE_CHECKING-only.Confirmed on real GraalPy 25.4.4 by cross-review:
test_deferred_pickle_2423_unit.pygoes from 8 failed onmainto 16 passed/18 subtests on this fix, and the fulltests/foundation/reassembly/suite passes 173/854 subtests, 0 failed, on GraalPy. A new regression test (test_deferred_pickle_graalpy_2433_unit.py) reproduces the failure mode locally by monkeypatchingcopyreg._reduce_exto raise the sameTypeError, and a second new test pins the subclass-__reduce__fix.CPython: mypy 98/32 (baseline, unchanged), pylint 10.00/10, isort clean,
tests/foundation/reassembly(176/890 subtests, including the two new regression tests) andtests/corekit(693/6767 subtests, unaffected by this change) pass, 100% coverage on the changed file.