Return type op fast path results without the full fulfillment machinery [-0.83% avg]#30
Draft
xmakro wants to merge 2 commits into
Draft
Return type op fast path results without the full fulfillment machinery [-0.83% avg]#30xmakro wants to merge 2 commits into
xmakro wants to merge 2 commits into
Conversation
Equating a hidden type in insert_hidden_type registers region constraints on the inference context eagerly and returns only the non-region goals. When the equation fails partway, those partial region constraints stayed pending on the inference context: nothing drained them until the next query type op scraped and attributed them to itself, whichever op that happened to be. Roll them back instead. This makes it an invariant that no region state is pending on the inference context between type ops. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Query type ops check their fast paths (trivial sizedness proofs, alias free normalizations) inside the closure passed to scrape_region_constraints. Every fast path hit therefore still paid for an inference snapshot, an ObligationCtxt with a boxed fulfillment engine, RefCell take and reset cycles for region constraints, and the region constraint scraping itself, all over empty data. MIR type check issues several type ops per statement, and the overwhelming majority of these hit a fast path. Fast path results are final: they register no obligations and produce no region constraints, so the fast path is now checked once at the top of fully_perform and the result returned directly. The two inner fast path checks become dead and are removed. resolve_vars_if_possible is kept on the returned value, matching what scrape_region_constraints did before. Skipping the machinery also skips its role as the unconditional drain point for region state pending on the inference context. With the hidden type equation rollback in the previous commit, nothing registers such state outside a type op; the fast path debug-asserts that invariant (and not-in-snapshot, matching the machinery's own assertions) via the new InferCtxt::has_pending_region_state helper, which is documented to stay in sync with what scrape_region_constraints drains. Borrowck's leftover-constraints debug check polices the same invariant after every op. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
xmakro
force-pushed
the
perf/typeop-fastpath-hoist
branch
from
July 26, 2026 05:57
5190b9a to
03969d2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Query type ops check their fast paths (trivial sizedness proofs, alias free normalizations) inside the closure passed to scrape_region_constraints. Every fast path hit therefore still paid for an inference snapshot, an ObligationCtxt with a boxed fulfillment engine, RefCell take and reset cycles for region constraints, and the region constraint scraping itself, all over empty data. MIR type check issues several type ops per statement, normalizing the place and rvalue types and proving sizedness of the place type, and the overwhelming majority of these hit a fast path.
Fast path results are final. They register no obligations and produce no region constraints, so the fast path is now checked once at the top of fully_perform and the result returned directly, skipping all of that machinery. The two inner fast path checks become dead and are removed. resolve_vars_if_possible is kept on the returned value because the normalize fast path can return values that still contain inference variables, matching what scrape_region_constraints did before.
Local measurement: from scratch ThinLTO stage2 with jemalloc, instructions:u, full rustc-perf suite, both sides built from scratch.
tests/ui/borrowck, tests/ui/nll, tests/ui/sized-hierarchy, tests/ui/associated-types and tests/ui/higher-ranked pass (1752 tests, 0 failures). x check compiler is clean.
Update: review found that skipping
scrape_region_constraintsalso skipped its role as the unconditional drain point for region state pending on theInferCtxt. The old-solver hidden type equation ininsert_hidden_typeregisters region constraints eagerly and could leave partial state pending when it fails partway (the?inrelate_opaquesbails beforeregister_goalsscrapes). Reworked in two commits:relate_opaquesnow rolls back partial hidden type equations viacommit_if_ok, making "no region state pending between type ops" a real invariant.InferCtxt::has_pending_region_statehelper instead of relying on it silently. Release behavior and the perf numbers above are unchanged.Note: the next solver's
solver_region_constraint_storage(-Zassumptions-on-binders) is cross-op by design and intentionally outside both the scrape and the assertion.Full
tests/ui(21304 tests) passes on a stage1 build withdebug-assertions = true, exercising the new assertions.