perf: dep_graph: append-only save via wholesale carry of the record region [~-3.8% incr-unchanged, supersedes #42/#43]#44
Draft
xmakro wants to merge 4 commits into
Draft
Conversation
On a warm rebuild the previous dep graph is decoded and then almost entirely re-encoded: every green node is read back from the previous graph and written out again, byte-for-byte equivalent to what it already was. This decode-and-re-encode round trip dominates dep-graph serialization when little or nothing changed. Instead, keep each green node at its previous index. A green node only ever points to other green nodes, which also keep their previous indices, so its edge list is unchanged and its whole record is identical to the previous file's. Re-emit that record directly: rebuild the fixed header from the already-decoded fields (feeding the byte-width back in so the header packs the same way) and copy the edge bytes straight from their on-disk form in `edge_list_data`, skipping the per-edge max scan and the per-edge write of a full re-encode. New and red nodes get fresh indices above the carried range, so they never collide with a carried index; the two singleton nodes stay pinned at indices 0 and 1 and are colored up front so they are never promoted into a duplicate record. Local rustc-perf (primary crates, instructions:u): incr-unchanged -1.59%, incr-patched -0.92%, full and incr-full flat (the carry is inactive without a previous cache, which doubles as a control that no overhead is added), 28 cells improved by at least 0.25%, 0 regressed. tests/incremental passes in full (178 tests, 0 failures). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…egion Building on carrying unchanged nodes forward: instead of re-emitting every green node's record one by one, copy the previous file's entire record region into the new file in a single write, and give every node that existed in the previous session a stable index, red or green. A node re-verified green then needs no write at all: its carried record, whose edges point at other kept indices, is already exactly right. A re-executed node appends a record at its old index which overrides the carried one (later records win at decode time), and a node this session dropped is tombstoned via a dead list in the footer. Genuinely new nodes get indices above the previous index space. The save becomes proportional to what changed, not to the size of the graph, and the one large copy is a single kernel-side write from the retained mapping. Marking a promoted node green shrinks to just the color-map insert: no encoder borrow, no record write, no per-node bookkeeping in the hot try_mark_green path. Footer counts are computed as O(changed) deltas from the previous footer. After eight carried generations the file is rewritten fresh, compacting away dead records, superseded duplicates and their index slots; the same fresh rewrite serves sessions where a debugging feature needs every node to pass through the encoder. Local rustc-perf (primary crates, instructions:u), against the same base: incr-unchanged -3.88%, incr-patched -2.31%, full and incr-full flat (-0.04%/-0.05%, the carry is inactive without a previous cache), all 30 incremental cells improved, 0 regressed. Multi-generation runs: 12 generations of incr-unchanged and of alternating real edits, both crossing the compaction boundary, all clean; 12 generations under -Zincremental-verify-ich clean; tests/incremental passes in full (178 tests, 0 failures). dep-graph.bin grows by the appended delta per generation and shrinks back at the compaction rewrite, as intended. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With the record region carried forward, promoting a green node no longer consumes its edge list: the record is already in the new file. The marking walk still collected every dependency's index into a buffer purely to hand it to the encoder. Add a non-collecting variant of the walk for carried sessions that verifies dependency colors and promotes with just the color-map insert, eliminating a store per edge and the per-node edge-frame bookkeeping from the hottest incremental path. Compacting sessions keep the collecting walk, whose encoder still needs the edges. Local measurement on top of the region carry: the walk's self cost on a dep-graph-heavy incr-unchanged rebuild drops by about a quarter, and multi-generation medians improve by up to another 1.5% (match-stress) and 0.9% (hyper incr-patched). 12 generations of incr-unchanged, of alternating real edits, and of -Zincremental-verify-ich all pass; tests/incremental passes in full. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…atio Decoding used to copy every edge byte out of the file into a freshly allocated buffer, the largest allocation and copy of the load. The file stays mapped for the whole session anyway (the carry copies its record region forward), so serve the edge lists directly from the retained bytes: the on-disk varint encoding was already the in-memory representation, and each node's edge header now records a position in the file instead of a position in the copied buffer. Decoding a record shrinks to reading its fixed header and skipping over the edges. The fixed-size overread at the end of an edge list stays in bounds because the footer always follows the records. On Windows, where the mapping must not outlive the load because the save renames over the mapped file, the bytes are copied out once instead. Compaction is now also triggered by the dead-byte ratio: the file is rewritten fresh once the record region exceeds twice its live bytes, tracked exactly in the footer as an O(changed) delta like the other counts. High-churn graphs compact as often as before, while low-churn graphs carry for up to sixteen generations instead of eight, avoiding periodic compaction spikes the fixed cap forced on them. Local rustc-perf (primary crates, instructions:u) on top of the previous commits: incr-unchanged -0.49%, incr-patched -0.32%, full and incr-full flat, 26 cells improved by at least 0.25%, 0 regressed. 12 generations of incr-unchanged, of alternating real edits, and of -Zincremental-verify-ich all pass; tests/incremental passes in full. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
xmakro
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on carrying unchanged nodes forward (stacked on that branch; supersedes both carry variants if taken). Two follow-up optimizations were split into their own PRs so each is measurable independently: skip edge collection in the marking walk, and serve edge lists in place from the retained file bytes.
Two commits on top of the carry:
1. Append-only save via wholesale carry of the record region. Copy the previous file's entire record region into the new file in a single kernel-side write and give every node that existed in the previous session a stable index, red or green. A node re-verified green needs no write at all; a re-executed node appends a record at its old index which overrides the carried one (later records win at decode time); a dropped node is tombstoned via a dead list in the footer; new nodes get indices above the previous index space. Footer counts are O(changed) deltas from the previous footer.
2. Compact by dead-byte ratio. The file is rewritten fresh once the record region exceeds 2x its live bytes (tracked exactly in the footer as an O(changed) delta), instead of only on a fixed generation count. High-churn graphs compact as before; low-churn graphs carry up to 16 generations with no periodic compaction spike.
With the two follow-up PRs stacked on top, the full stack previously measured (local rustc-perf, primary crates, instructions:u): incr-unchanged -5.80%, incr-patched -3.55%, full/incr-full flat, all 30 incremental cells improved, 0 regressed. The follow-up PRs carry their own independent A/B numbers against this branch.
Validation on this branch alone: 20 generations each of incr-unchanged, of alternating real edits, and of alternating edits under
-Zincremental-verify-ich, crossing both the ratio-triggered and the generation-cap compaction boundaries: every generation clean. Compaction observably triggers by ratio under churn (gen 14) and by the generation cap without churn (gen 16), as intended.Known tradeoffs: the previous file stays mapped for the session; dead records accumulate between compactions, bounded at 2x live bytes; the marking walk remains O(verified nodes + edges) — making that change-proportional needs durability-style input masks, written up separately as future work.