UnloadChunk: bulk probe extraction for chunk batches

[DAlperin]

UnloadChunk is a second way to read a chunk, alongside NavigableChunk: given a sorted set of probe keys, copy the matching updates out into caller-owned staging. A cursor navigates a chunk in place and hands out references into it — Cursor::key<'a>(&self, storage: &'a Storage) -> Key<'a> lets the caller hold borrows for the batch's lifetime — so the only sound way to read a paged chunk under it is ColChunk's fetch-and-cache-forever OnceCell, and a probe-heavy consumer progressively re-materializes the trace it probes. Extraction is finished with a chunk's body when the call returns: any fetch a paged implementor needs is scoped inside one method, the cache stays empty, and probing a paged trace does not accrete it back into memory.

The trait keeps Chunk's discipline: key/val/time/diff opinions stay off it (Staging and Probes are opaque associated types), and the one comparison the batch driver needs is delegated to the chunk via locate, answered from resident metadata like len. Chunk boundaries are carried as a protocol rather than an index: a chunk consumes every probe strictly below its last key and extracts-but-does-not-consume a probe equal to it, so the driver re-offers that probe to the next chunk and the continuation lands in staging as a legal straddle — the same invariant chunk sequences carry everywhere else. Times are copied verbatim; since-advancement stays with the consumer, exactly as on the cursor path.

For the in-tree families, Staging is ChunkBatchBuilder<Self>: extraction emits its hit runs as resident chunks and the builder settles as it goes, so staged data is graded, coalesced, and — when a spiller is installed — paged under the same bounded-footprint discipline as any other chunk sequence. Extraction is thereby chunks-in, chunks-out, its results consumable by everything chunk-shaped (including, since done yields a ChunkBatch, the existing cursors — soundly, over storage the consumer owns). A probe run covering a whole resident chunk, and every fetch_into of one, stages as a handle push: a refcount bump, no copy.

Contents:

• trace/chunk: the UnloadChunk trait, a provided ChunkBatch driver (gallop the chunk list by locate, open only the chunks a probe touches, fetch_into as the scan path), and the VecChunk implementation.
• columnar: UpdatesBuilder::meld_keys, melding a key-index range of a borrowed UpdatesView by bulk column-range copies (meld now delegates to it). ColChunk's extraction melds each source chunk's partial hit runs into one staged chunk through it; because it works on views, one primitive serves typed tries and wire bytes, and the paged arm reads a zero-copy view over the loaded bytes (spill::read) for the scope of the call — no trie rebuild, no cache population.
• Tests: filter-oracle property tests over random chains for vec and col, resident and paged (asserting caches stay empty); exhaustive (chunk cut × probe placement) boundary coverage including a key spanning several chunks; a meld_keys property test against filtered form; paged extraction and scan round-trips with spill stats.
• chunk_bench: extraction vs the straddle cursor, both consuming into owned staging. At 1M updates: dense String probes 201ms cursor vs 1.8ms extraction (the whole-chunk handle path; the design doc's 21ms-class admission with two orders of headroom), sparse (1%) probes at 1.2x over cursor parity, fat 4x256B values staged as handle pushes. Paged cold reads: dense 23x, sparse 4.3x — where the cursor path must decode-and-cache while extraction views the spilled bytes.

This is the counterpart of #781 one layer down. That PR's backend presentations (present0, present_input) have exactly this contract — sorted probe keys in, an owned restricted run out, nothing borrowed across the boundary — and leave their implementation to the backend; extract_into and meld_keys are the machinery a chunked, spillable backend would implement them with. Both sit on the same two seams: tactics as the consumption dispatch point (#773, public since #789–#791), and cursor-less chunks as full trace citizens (#778). Rebased over all of the above.

🤖 Generated with Claude Code