Fix regression in dataclass narrowing for Python >= 3.13#21675
Open
ygale wants to merge 2 commits into
Open
Conversation
…ases Fixes python#21635. On Python 3.13+, @DataClass synthesizes a __replace__(self, ...) -> Self method to support copy.replace(). When mypy builds an ad-hoc intersection type to narrow an expression via issubclass()/isinstance() against two unrelated dataclasses, check_multiple_inheritance sees each dataclass's synthesized __replace__ returning its own concrete type (e.g. A vs M) and flags them as incompatible, causing intersect_instances to fail and the narrowed type to collapse to Never -- even when a real subclass of both (itself decorated with @DataClass, and so getting its own compatible synthesized __replace__) already exists in the code. This exempts __replace__ from the cross-base compatibility check the same way __init__/__new__/__init_subclass__ already are, but only when the method was generated by a plugin (plugin_generated=True) on both sides -- hand-written __replace__ overrides with genuine incompatibilities are still caught.
This comment has been minimized.
This comment has been minimized.
Follow-up to the previous commit on this branch. Same underlying bug (python#21635): the false impossible-intersection error from a plugin-synthesized __replace__ isn't specific to list comprehensions. In a plain if-statement, issubclass() narrowing hits the identical false Never, but the failure mode is silent instead of loud -- mypy marks the branch unreachable and skips type-checking it entirely, so no reveal_type note appears and a genuinely broken assignment inside the branch goes unreported. This adds a regression test for that minimal, comprehension-free reproduction alongside the existing ones.
Contributor
|
Diff from mypy_primer, showing the effect of this PR on open source code: steam.py (https://github.com/Gobot1234/steam.py)
- steam/profile.py:450: error: Definition of "__replace__" in base class "ProfileInfo" is incompatible with definition in base class "EquippedProfileItems" [misc]
- steam/profile.py:461: error: Definition of "__replace__" in base class "ProfileInfo" is incompatible with definition in base class "EquippedProfileItems" [misc]
dd-trace-py (https://github.com/DataDog/dd-trace-py)
- ddtrace/debugging/_signal/model.py:92: error: Subclass of "Probe" and "TimingMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_signal/model.py:93: error: Statement is unreachable [unreachable]
- ddtrace/debugging/_signal/model.py:96: error: Cannot determine type of "_timing" [has-type]
- ddtrace/debugging/_signal/model.py:101: error: Subclass of "Probe" and "ProbeConditionMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_signal/model.py:105: error: Statement is unreachable [unreachable]
- ddtrace/debugging/_signal/model.py:127: error: Subclass of "Probe" and "RateLimitMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_signal/model.py:131: error: Statement is unreachable [unreachable]
- ddtrace/debugging/_signal/model.py:182: error: Cannot determine type of "_timing" [has-type]
- ddtrace/debugging/_signal/model.py:215: error: Cannot determine type of "_timing" [has-type]
- ddtrace/debugging/_signal/log.py:41: error: Subclass of "Probe" and "LineLocationMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_signal/log.py:42: error: Statement is unreachable [unreachable]
- ddtrace/debugging/_signal/log.py:46: error: Subclass of "Probe" and "FunctionLocationMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_signal/log.py:47: error: Statement is unreachable [unreachable]
- ddtrace/debugging/_signal/log.py:54: error: Statement is unreachable [unreachable]
- ddtrace/debugging/_signal/snapshot.py:204: error: Need type annotation for "captures" (hint: "captures: dict[<type>, <type>] = ...") [var-annotated]
- ddtrace/debugging/_signal/snapshot.py:205: error: Subclass of "Probe" and "LogProbeMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_signal/snapshot.py:205: error: Right operand of "and" is never evaluated [unreachable]
- ddtrace/debugging/_signal/snapshot.py:206: error: Statement is unreachable [unreachable]
- ddtrace/debugging/_probe/registry.py:47: error: Subclass of "Probe" and "ProbeLocationMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_probe/registry.py:48: error: Statement is unreachable [unreachable]
+ ddtrace/debugging/_debugger.py:536: error: Redundant cast to "FunctionType" [redundant-cast]
- ddtrace/debugging/_debugger.py:374: error: Subclass of "Probe" and "LineLocationMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_debugger.py:376: error: Statement is unreachable [unreachable]
- ddtrace/debugging/_debugger.py:500: error: Subclass of "Probe" and "FunctionLocationMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_debugger.py:503: error: Statement is unreachable [unreachable]
- ddtrace/debugging/_debugger.py:619: error: Subclass of "Probe" and "LineLocationMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_debugger.py:620: error: Statement is unreachable [unreachable]
- ddtrace/debugging/_debugger.py:621: error: Subclass of "Probe" and "FunctionLocationMixin" cannot exist: would have incompatible method signatures [unreachable]
- ddtrace/debugging/_debugger.py:622: error: Statement is unreachable [unreachable]
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #21635.
What's going wrong
On Python >= 3.13,
@dataclasssynthesizes a__replace__(self, ...) -> Selfmethod to support
copy.replace(). When mypy tries to narrow atype[A]expression via
issubclass(cls, M)(orisinstance) against a second,unrelated dataclass
M, it builds an ad-hoc<subclass of A and M>type tocheck whether that narrowing is sound (
intersect_instancesinchecker.py). Building that ad-hoc type runscheck_multiple_inheritance,which sees
A's synthesized__replace__returningAandM'sreturning
M, and flags them as incompatible.That's a false positive: a real subclass of both (e.g.
class C(M, A),itself decorated with
@dataclass) gets its own freshly synthesized,mutually compatible
__replace__.The result
In an
ifstatement where there should have been type narrowing, the type resolves asNever, and so the block is ignored by the type checker. Type checking always silently succeeds without actually checking.In a comprehension, the
ifclause is assigned the typeNeverand so is ignored by the type checker. The object whose type should have been narrowed retains its original type while type checking the comprehension expression.The fix
check_compatibilityalready exempts__init__,__new__, and__init_subclass__from this cross-base check, because those are expectedto vary safely per concrete subclass.
__replace__has the same characterwhen it's plugin-generated (
Self-returning, regenerated fresh perconcrete class), so this adds it to that exemption -- but only when the
method is
plugin_generatedon both sides, so hand-written__replace__overrides with genuine incompatibilities are still caught.
Not included in this PR
TypeIsfailure mentioned in the description of Dataclasses fail to narrow withissubclass()in Python >= 3.13 #21635 turns out to be a completely different failure mode. I will open a separate issue for that.NamedTuple, but that also turns out to be a completely different failure mode. I will also open a separate issue for that.Tests
check-dataclasses.testwith the reproduction from the description of Dataclasses fail to narrow withissubclass()in Python >= 3.13 #21635check-dataclasses.testwith confirming that hand-written incompatible__replace__methods are still flagged.dataclass/isinstance/narrow/typeguard/comprehension/multiple-inheritance test suites locally; all pass.
black,ruff check, andcodespellall clean on the changed file.checker.pyis clean.LLM Disclosure
I was assisted by Claude (Sonnet 5), under my close guidance and supervision. The work is mine.