Define RFC-026 Phase B implementation gates

[aaltshuler]

Summary

• update RFC-026 and the architecture canon after the implemented Phase A / internal-schema-v7 foundation
• split the next work into a private Phase B1 correctness slice and a separately gated public Phase B2 activation
• define the bounded one-generation writer, recovery-v11 fold, resource limits, format-v8/config-v2 boundary, and test ownership
• record pinned Lance 9.0.0-rc.1 hazards and the narrow public replay-watermark compatibility bridge
• keep RFC-024, RFC-025, and RFC-027 at their measured research no-go states

Why

Phase A made empty MemWAL enrollment recoverable, but it did not validate the row path. The RC.1 source audit found several load-bearing details that must be designed and tested before implementation:

• the durability watermark spans writer generations while batch positions reset
• put_no_wait may mutate before returning an error
• replay does not initialize the fresh BatchStore WAL-flush watermark, so plain replay/reseal can multiply WAL entries across crashes
• a late wait_for_flush_drain can miss a completed flush failure
• cancelling shutdown_all can detach an active handler
• same-key retry after AckUnknown is cardinality-safe but can overwrite a newer interleaving value

The documented B1 profile handles these inside the existing one-live-writer-process boundary without waiting for a new Lance release. Public B2 remains blocked on durable attribution, reclamation/orphan cleanup, strict correction, same-key sequencing/idempotency, and persistent operator controls.

Scope

Documentation and implementation planning only. This PR does not add a WAL row path, public stream API, schema syntax, or format-v8 code.

Validation

• cargo test -p omnigraph-engine --locked --test memwal_enrollment_gate -- --nocapture — 15 passed, including the explicit unconfigured-S3 skip path
• cargo test -p omnigraph-engine --locked --test lance_surface_guards — 23 passed
• cargo test -p omnigraph-engine --locked --test forbidden_apis — 16 passed
• git diff --check
• scripts/check-agents-md.sh — 69 links across 65 docs

Greptile Summary

This PR defines the next implementation gates for RFC-026. The main changes are:

• Splits private Phase B1 correctness work from public Phase B2 activation.
• Defines a bounded, single-generation writer and recovery-v11 fold.
• Documents Lance RC.1 watcher, replay, drain, and shutdown hazards.
• Adds public activation gates for attribution, reclamation, correction, sequencing, and operator controls.
• Aligns the architecture canon and supporting storage documentation.

Confidence Score: 5/5

This looks safe to merge.

• The updated RFC now keeps worker registry identity separate from the common table/ref admission lock.
• Enrollment, ordinary writes, and drain operations use the same lock domain.
• No blocking issue remains in the reviewed changes.

Important Files Changed

Filename	Overview
docs/rfcs/0026-memwal-streaming-ingest.md	Defines the Phase B1 and B2 gates and separates worker identity from the shared table/ref admission-lock domain.
docs/dev/canon.md	Updates the architecture canon and risk register for the bounded private streaming profile.
docs/dev/lance.md	Documents the Lance RC.1 behavior that constrains replay, durability, draining, and writer retirement.
WAL-thinking.md	Updates the WAL design overview with the private proof phase and later public activation requirements.

_{Reviews (2): Last reviewed commit: "docs: unify RFC-026 admission lock domai..." | Re-trigger Greptile}

Context used:

• Context used - AGENTS.md (source)
• Context used - CLAUDE.md (source)