From 5a80ddf0ea1b98f3c594b108be5e1c7bf13dd9db Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 29 May 2026 17:21:49 +0000 Subject: [PATCH 1/3] =?UTF-8?q?docs(mailbox):=20capture=205=20layered=20ru?= =?UTF-8?q?lings=20=E2=80=94=20one=20SoA=20never=20transformed;=20Libet=20?= =?UTF-8?q?-550ms=20Rubicon=20kanban=20in=20surrealkv-on-lance;=20SPO-W=20?= =?UTF-8?q?=3D=20pointer=20via=20belief-state=20arc?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan §11 + E-SOA-IS-THE-ONLY epiphany, recording the user-stated rulings: §11.1 One SoA, never transformed. The mailbox SoA is the single universal carrier across the stack; only three operations allowed (cognitive-shader thinking, cold-path read/write, AriGraph Markov context). Any change in any mailbox SoA = the only hot-path activity. Today's lance-graph containers are cold-path-adjacent thinking, only accidentally aligned; D-MBX-7 realigns them to MailboxSoA layout = ndarray::simd_soa.rs alignment → 1.4-4.2× SIMD, hard prereq for the SurrealDB transparent view. §11.2 Mailbox = full BindSpace reinvented as LE; witness = belief-state arc. The witness IS the per-row arc of CausalEdge64 emissions (CollapseGateEmission arc), implicitly documenting NARS revision (no separate log column). D-MBX-A1 columns landed; D-MBX-A2 closes expressivity gaps; D-MBX-A3 adds arc handle. §11.3 Libet -550 ms anchors the Rubicon (E-RUBICON-RACTOR concretised). Commit at t=-550ms = Σ10 commit = ractor START; Libet veto = pre-(-550ms) preempt; post-actional = ractor STOP. Rubicon kanban lives in surrealkv-on-lance (view over leading LanceDB), columns = 4 Heckhausen action phases. D-MBX-8 / D-MBX-9. §11.4 SPO-W witness = pointer, not stored data. Arc handle in mailbox SoA / kanban / mailbox index; the SoA decides commit to other mailboxes vs cold-path facts. No storage redundancy. D-MBX-A5. §11.5 Counterfactual Staunen × Wisdom = plasticity spreaders. Hot-path-only Hebbian spread when pre-Rubicon. Radius/decay TBD. D-MBX-A4. Author-stated rulings; not council-gated (council is for derived epiphanies). Design/spec only. https://claude.ai/code/session_017GFLBnDy23AWBqvkbHHC41 --- .claude/board/EPIPHANIES.md | 18 ++++ .../bindspace-singleton-to-mailbox-soa-v1.md | 83 +++++++++++++++++++ 2 files changed, 101 insertions(+) diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 37aaf719..4593b411 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,21 @@ +## 2026-05-29 — E-SOA-IS-THE-ONLY — there is ONE SoA, never transformed; mailbox SoA mutation IS the hot path; Libet −550 ms anchors the Rubicon kanban in surrealkv-on-lance; SPO-W witness is a *pointer* via the belief-state arc + +**Status:** CONJECTURE / design (user-stated 2026-05-29, post-PR-#433). Records five layered rulings; details in `bindspace-singleton-to-mailbox-soa-v1.md` §11.1–§11.5. Author-stated; not council-gated. + +1. **One SoA, never transformed (§11.1).** The mailbox SoA is the *single* universal carrier across the stack — never re-encoded into a different shape. Only three operations are allowed on it: (a) cognitive-shader thinking (hot path), (b) cold-path read/write to LanceDB, (c) AriGraph Markov-chain context building. *Any change in any mailbox SoA = the only hot-path activity.* Today's `crates/lance-graph` containers are *cold-path-adjacent thinking* — only accidentally aligned. Realignment (`D-MBX-7`): lance-graph containers ≡ `MailboxSoA` layout ≡ `ndarray::simd_soa.rs` alignment, unlocking **1.4–4.2× SIMD acceleration**. Nice-to-have today; **hard prerequisite** for the SurrealDB transparent view (§2.7). + +2. **Mailbox = full BindSpace, reinvented as LE; witness = belief-state arc (§11.2).** Carry everything BindSpace had, but as LE-contract types (never `Vsa16kF32`). The **witness IS the per-row arc of `CausalEdge64` emissions** (`CollapseGateEmission` arc) — that arc *implicitly* documents NARS revision because every emission stamps `confidence_u8` + `inference_mantissa`; reading the arc IS the `(frequency, confidence)` trace. **No separate revision log column.** D-MBX-A1 columns (`edges`/`qualia`/`meta`/`entity_type`) landed; D-MBX-A2 closes remaining BindSpace-expressivity gaps; D-MBX-A3 adds the witness-arc handle column. + +3. **Libet −550 ms anchors the Rubicon (§11.3).** Concretises `E-RUBICON-RACTOR` with wall-clock: pre-decisional = counterfactual deliberation; **commit at t = −550 ms** (Libet readiness-potential anchor) = Σ10 commit = ractor START of the actional phase; Libet "free won't" = pre-(−550 ms) CollapseGate veto / ghost-tier preempt; post-actional → ractor STOP → tombstone. The **Rubicon kanban lives in `surrealkv`-on-lance** (a *view* over leading LanceDB, §2.7), columns = the 4 action phases; ractor lifecycle transitions = kanban moves. (`D-MBX-8` timing anchor; `D-MBX-9` kanban view.) + +4. **SPO-W witness is a *pointer*, not stored data (§11.4).** The witness is an arc-handle into the belief-state-arc array; the pointer lives equivalently in the **mailbox SoA**, the **kanban row**, or the **mailbox index**. **The SoA itself decides** whether to commit a belief as a fact-with-witness into (a) other mailboxes (inter-mailbox baton handoff carrying the pointer) or (b) cold-path facts (LanceDB SPO-G calcification linking back to the arc). No storage redundancy: the witness lives where the arc lives. (`D-MBX-A5`.) + +5. **Counterfactual Staunen × Wisdom = plasticity spreaders (§11.5).** When a mailbox is pre-Rubicon (counterfactual phase), high Staunen × Wisdom *spreads* plasticity beyond the focal row (Hebbian spread). Hot-path-only — the spread is a mailbox-SoA mutation, never a side channel. Radius/decay TBD. (`D-MBX-A4`.) + +**Cross-ref:** `E-BATON-1`, `E-MAILBOX-IS-BINDSPACE`, `E-RUBICON-RACTOR` (Heckhausen + Libet origin), `E-LADDER-SERVES-MAILBOX` (§5 ghost preempt = Libet veto, §6 tombstone), `I-VSA-IDENTITIES`, `causaledge64-mailbox-rename-soa-v1` §10 + E-CE64-MB-8 (Σ10 router), `linguistic-epiphanies-2026-04-19.md` E21 (Σ10 Rubicon tier doctrine), D-CSV-10 (`SigmaTierRouter` shipped #388), `bindspace-singleton-to-mailbox-soa-v1` §11 (full ruling + D-MBX-A2/A3/A4/A5/7/8/9 + OQ-11.1–11.4); `epiphany-brainstorm-council` (PR #433 pre-merge gate — bypassed here because rulings are author-stated). + +--- + ## 2026-05-28 — E-NORMALIZED-ENTITY-1 — `NormalizedEntity` is the single typed carrier holding the four-way inheritance chain (Odoo → OGIT → OWL → DOLCE → FIBU/FIBO); stage advancement is typestate, not method calls on a context **Status:** FINDING (architectural unification). Drives `normalized-entity-holy-grail-v1`. The carrier is a typed lens into a `MailboxSoA` row — it does NOT own the four cognitive columns; the mailbox does. The 4-way inheritance slots (`odoo`/`ogit`/`owl`/`dolce` + optional `fibu`) populate as stages advance; phantom-typed `Stage` parameter forbids out-of-order traversal at compile time. Consumers chain ON the carrier (`entity.resolve_ogit(ctx).hydrate_owl(ctx)...`), never reach into its internals. diff --git a/.claude/plans/bindspace-singleton-to-mailbox-soa-v1.md b/.claude/plans/bindspace-singleton-to-mailbox-soa-v1.md index 4a1d42ce..6fb05065 100644 --- a/.claude/plans/bindspace-singleton-to-mailbox-soa-v1.md +++ b/.claude/plans/bindspace-singleton-to-mailbox-soa-v1.md @@ -423,3 +423,86 @@ append-only findings; no prior section has been modified. - **OQ-MBX-8** — `persisted_row` stub vs Lance native versioning (load-bearing; evidence at `REFACTOR_NOTES.md:129` + `driver.rs:458`). - **OQ-MBX-15′** — container scoping: per-cognitive-cycle, per-shader-dispatch, or per-mailbox-cohort? + +--- + +## §11 — 2026-05-29 architectural rulings (post-PR-#433 sync; D-MBX-A1 columns landed) + +User-stated rulings layering on top of §2.5/§2.7/§10. Recorded directly (not via `epiphany-brainstorm-council` shipped in PR #433) because they are author-stated, not derived. + +### §11.1 — THE one SoA is never *transformed*, only *operated on* (and that's the 1.4–4.2× SIMD payoff) + +> *"The same SoA is the one and only SoA consumed and transmitted everywhere, never transformed, only [operated on] through cognitive-shader thinking or cold path or AriGraph Markov chain context building. Any change in any mailbox SoA is the only hot-path activity."* + +- **The single mailbox SoA is the universal carrier across the whole stack.** It is *never re-encoded into a different shape.* Three operations are allowed on it — they are *operations*, not transforms: + 1. **Cognitive-shader thinking** (the hot path: `apply_edges` / `emit` over the per-row columns). + 2. **Cold path** (LanceDB read/write: same bytes — see §2.5). + 3. **AriGraph Markov-chain context-building** (read-only consumer of the SoA columns for context windows). +- **"Any change in any mailbox SoA = the only hot-path activity."** That equivalence is now the definition: if it is not a mutation of mailbox SoA bytes, it is not the hot path. Anything else (DTO, RecordBatch translation, JSON, etc.) is a re-encode boundary that violates §2.5 and must be collapsed or made out-of-scope. +- **Today's `crates/lance-graph` containers are *cold-path-adjacent thinking*** — only *accidentally* aligned to the SoA shape, not by design. They look in roughly the same direction; they are not the same contract. The realignment work (next bullet) makes that intentional. +- **SoA-as-lance-graph-containers + `ndarray::simd_soa.rs` alignment = 1.4–4.2× SIMD acceleration.** Today this is a nice-to-have. **It becomes a hard prerequisite** the moment SurrealDB needs a transparent view of LE-contract SoA mailbox content (§2.7 + §11.3): the view is only zero-copy / re-encode-free when the lance-graph container layout *is* the mailbox SoA layout. Filing the prereq as **`D-MBX-7`**. + +### §11.2 — Mailbox = the *full* BindSpace, reinvented as LE — witness is the belief-state arc + +> *"The mailbox needs to have everything that BindSpace had, reinvented as little-endian contract — so instead of a mushy `Vsa16kF32` it needs to be expressive enough so that, in connection with the witness, the witness is the belief-state arc; while `CausalEdge64` across a belief-state arc would implicitly document if NARS frequency and confidence would increase."* + +- **Expressivity target:** the mailbox SoA must carry *everything BindSpace had* — but as **LE-contract types** (`CausalEdge64`, `QualiaI4_16D`, `MetaWord`, i4/u8/u16/u32/u64 columns), never as the mushy `Vsa16kF32` resonance carrier. (D-MBX-A1 already landed `edges` / `qualia` / `meta` / `entity_type` per the current `mailbox_soa.rs`; D-MBX-A2 must close any remaining BindSpace-expressivity gaps — content-ref, temporal/expert fold, S/P/O role slices — see §3 and OQ-1/OQ-2.) +- **Witness = belief-state arc** (the *sequence* of `CausalEdge64` emissions a row produces over its life — the **`CollapseGateEmission` arc**, not a single edge). The arc *implicitly documents* NARS revision because `CausalEdge64.confidence_u8` and `inference_mantissa` are stamped per emission; reading the arc gives a monotone-or-not trace of `(frequency, confidence)` evolution. **No separate "revision log" column is needed** — the witness arc *is* the revision log. +- **Iron rule (proposed):** the per-row witness arc IS the row's belief state — never a parallel struct. If you need the row's NARS history, you traverse its `CausalEdge64` arc; if you need its current truth, you read the *last* edge. + +### §11.3 — Libet wall-clock: −550 ms ratification; Rubicon kanban in **surrealkv-on-lance** + +> *"Any mailbox lives in Libet's and Heckhausen's Rubicon model aligned: planning phase as counterfactual, committing the goal at −550 ms, and doing a veto / goalstate re-evaluation via ractor mailbox triggering the Rubicon kanban in surrealkv on lance."* + +Concretises the timing of `E-RUBICON-RACTOR`: + +| Heckhausen phase | Mailbox state | Wall-clock | +|---|---|---| +| Pre-decisional (deliberation, **counterfactual**) | energy integrates; no commit; alternatives explored under `InferenceType::Counterfactual` | t < −550 ms before action | +| **Crossing the Rubicon** (intention forms — irreversible volition) | **Σ10 commit** stamped: `ΔF < threshold ∧ resonance > Rubicon-bar` → ractor START | **t = −550 ms** (Libet readiness-potential anchor) | +| Pre-actional / actional | ractor running the action; baton emits; mailbox-SoA mutates (the only hot-path activity, §11.1) | −550 ms ≤ t < 0 ms | +| Post-actional evaluation | ractor STOP; tombstone-witness persists | t > 0 ms | + +- The **−550 ms** anchor names *when* in wall-clock the irreversible commit occurs — derived from Libet's measured readiness-potential lead time (~550 ms before reported conscious decision). +- **Libet "free won't" / veto** = before −550 ms, the CollapseGate can preempt the mailbox to zero (ghost-tier preempt; `E-LADDER-SERVES-MAILBOX` §5). After −550 ms, the act is committed — only post-actional revision is left. +- **Goalstate re-evaluation** = the ractor mailbox triggers a re-deliberation cycle (re-enter pre-decisional with the post-actional witness folded in); this triggers the **Rubicon kanban move** in the storage view (next bullet). +- **The Rubicon kanban lives in `surrealkv` on lance** — `surrealkv` is the SurrealDB on-kv-lance backend (the *view* over leading LanceDB storage, §2.7). The kanban is a SurrealDB query over the LanceDB-stored thoughts, columns = action phases (deliberation | crossed | actional | evaluated). Ractor lifecycle transitions map 1:1 to kanban column moves. + +### §11.4 — SPO-W witness is a *pointer*, not stored data — via the belief-state-arc array + +> *"The SPO-W witness is the pointer via the AriGraph episodic / belief-state arc array inside the SoA and/or kanban and/or mailbox index, and the SoA then decides if it commits itself as facts with witness in other mailboxes and/or the cold-path facts."* + +- **SPO-W witness ≠ a fact payload; it is a *pointer*** into the belief-state arc array. The pointer can live in three equivalent locations: + 1. Inside the mailbox **SoA** (an arc-index column / per-row `[u32; W]` arc handle). + 2. Inside the **kanban** (`surrealkv`-on-lance view) row. + 3. Inside the **mailbox index** (the sea-star registry of live mailboxes). +- **Whose commit?** *The SoA itself decides* whether to commit a belief as a fact-with-witness either (a) into **other mailboxes** (inter-mailbox baton handoff carrying the witness pointer) or (b) into the **cold-path facts** (LanceDB SPO-G calcification with the witness pointer linking back to the arc). +- This makes SPO-W *resolvable without storage redundancy*: the witness lives where the arc lives (mailbox row), and the SPO-G fact carries only the pointer. + +### §11.5 — Counterfactual Staunen and Wisdom = plasticity spreaders + +> *"Counterfactual Staunen and Wisdom should become helpers of spreading plasticity."* + +- `Staunen` and `Wisdom` are already qualia markers (cf. The Click: *"Magnitude = Contradiction depth from Staunen × Wisdom qualia"*; `E-LADDER-SERVES-MAILBOX`). +- **New role:** when a mailbox is in the counterfactual phase (§11.3 pre-Rubicon), high Staunen × Wisdom *spreads* plasticity beyond the focal row — bumping `plasticity_counter` on adjacent rows (Hebbian spread). Today plasticity increments only on the receiving row (`mailbox_soa.rs:144` `apply_edges`); under this rule, counterfactual-tagged Staunen-or-Wisdom-elevated emissions seed a small spread radius. +- **Hot-path-only.** Spreading happens in the mailbox SoA mutation (per §11.1); never in a separate side-channel. +- **Mechanics deferred** to `D-MBX-A4` (Staunen/Wisdom plasticity-spreader) — open question: radius, decay, and whether the spread is column-local (within one mailbox) or routed via baton (inter-mailbox). + +### New deliverables from §11 + +| D-id | Title | Owner | Status | +|---|---|---|---| +| **D-MBX-A2** | Close BindSpace-expressivity gaps in `MailboxSoA` (content-ref column; temporal/expert fold per OQ-1/OQ-2; S/P/O role slices); LE only | cognitive-shader-driver + contract | **Queued** (follows D-MBX-A1 columns) | +| **D-MBX-A3** | Witness-arc column: per-row `[u32; W]` arc handle into the belief-state arc; reading the arc gives the NARS revision trace (no separate log column) | cognitive-shader-driver | **Queued** | +| **D-MBX-A4** | Counterfactual Staunen/Wisdom plasticity spreader (radius/decay TBD; hot-path-only) | cognitive-shader-driver | **Queued — design** | +| **D-MBX-7** | `lance-graph` container layout = `MailboxSoA` layout = `ndarray::simd_soa.rs` aligned → enables zero-copy SurrealDB view + the 1.4–4.2× SIMD payoff | lance-graph + ndarray | **Queued — prerequisite for §2.7 D-MBX-6** | +| **D-MBX-8** | Libet −550 ms timing anchor wired into `SigmaTierRouter` (Rubicon commit stamp) + ractor START/STOP scheduler | sigma-tier-router + ractor outer-swarm | **Queued** | +| **D-MBX-9** | Rubicon kanban = `surrealkv`-on-lance view (deliberation \| crossed \| actional \| evaluated) backed by ractor lifecycle | surreal_container (view layer) + ractor | **Queued — blocks on `surreal_container` BLOCKED(A/B/C/D)** | +| **D-MBX-A5** | SPO-W witness pointer column (arc-handle / mailbox-index dual residency); commit decision in SoA, not in a side service | cognitive-shader-driver + AriGraph SPO-G | **Queued** | + +### Open questions added in §11 + +- **OQ-11.1** — radius/decay of the Staunen×Wisdom counterfactual plasticity spread (column-local vs baton-routed). +- **OQ-11.2** — witness-arc width `W` (how many `CausalEdge64` emissions per row before rotation/eviction). +- **OQ-11.3** — kanban column states beyond the 4 Heckhausen phases (need a "vetoed" column? a "ghosted" column for preempts?). +- **OQ-11.4** — `simd_soa.rs` alignment: do we need `#[repr(C, align(64))]` on `MailboxSoA`, or is the `SoaColumns` discipline (already shipped in `ndarray` `547824bc`) enough? From eb5c4a580c211aa8ab8209d0635ba9eb78358a34 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 29 May 2026 18:22:15 +0000 Subject: [PATCH 2/3] =?UTF-8?q?docs(mailbox):=20=C2=A711.3=204-phase=20kan?= =?UTF-8?q?ban=20+=20=C2=A711.4=20AriGraph=20episodic=20chain=20+=20=C2=A7?= =?UTF-8?q?11.6=20nine-consumer=20SoA=20convergence=20+=20Lance=206.0.1=20?= =?UTF-8?q?/=20LanceDB=200.29=20/=20DataFusion=2053=20stack=20pin?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three layered refinements to the §11 rulings: §11.3 kanban refined to 4 explicit columns: Planning (ractor mailbox owned SoA, counterfactual) → Cognitive work (Σ10 commit + actional SoA mutation) → Evaluation of goalstate → Commit · Plan · Prune (3-way terminal: calcify/re-deliberate/ghost-preempt). Supersedes the earlier 4-Heckhausen- phase mapping. §11.4 sharpened: "witness in other mailboxes" = a pointer into the AriGraph episodic Markov chain. The chain IS the episodic memory substrate (CLAUDE.md The Click: AriGraph IS thinking tissue, not storage). No parallel episodic structure exists. §11.6 NEW: the "half-baked nine" all consume THE same SoA from A-Z. AriGraph, Markov-grammar Vsa16kF32 substrate, BindSpace, lance-graph cold containers, lance-graph-planner, cognitive-shader-driver, lance-graph-callcenter, lance-graph-ontology (read-only AS IS), thinking-styles/atoms. SoA gets a version byte at the layout root (D-MBX-10, governed by I-LEGACY-API-FEATURE- GATED) so older bytes stay readable after schema upgrade. SurrealDB transparent view requires Lance 6.0.1 / LanceDB 0.29 / DataFusion 53 alignment — only lance =6.0.0 → =6.0.1 patch bump pending (D-MBX-11); arrow/datafusion/lancedb already on target. lance-graph-planner DTO surface overhaul (D-MBX-A6) re-expresses DTOs as operations on the SoA + 4-phase kanban transitions. Author-stated rulings; not council-gated. Design/spec only. (Cargo prohibited per user, session-stability concern; no build/test ran.) https://claude.ai/code/session_017GFLBnDy23AWBqvkbHHC41 --- .claude/board/EPIPHANIES.md | 8 +- .../bindspace-singleton-to-mailbox-soa-v1.md | 92 ++++++++++++++++--- 2 files changed, 84 insertions(+), 16 deletions(-) diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 4593b411..c74b93c4 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -12,7 +12,13 @@ 5. **Counterfactual Staunen × Wisdom = plasticity spreaders (§11.5).** When a mailbox is pre-Rubicon (counterfactual phase), high Staunen × Wisdom *spreads* plasticity beyond the focal row (Hebbian spread). Hot-path-only — the spread is a mailbox-SoA mutation, never a side channel. Radius/decay TBD. (`D-MBX-A4`.) -**Cross-ref:** `E-BATON-1`, `E-MAILBOX-IS-BINDSPACE`, `E-RUBICON-RACTOR` (Heckhausen + Libet origin), `E-LADDER-SERVES-MAILBOX` (§5 ghost preempt = Libet veto, §6 tombstone), `I-VSA-IDENTITIES`, `causaledge64-mailbox-rename-soa-v1` §10 + E-CE64-MB-8 (Σ10 router), `linguistic-epiphanies-2026-04-19.md` E21 (Σ10 Rubicon tier doctrine), D-CSV-10 (`SigmaTierRouter` shipped #388), `bindspace-singleton-to-mailbox-soa-v1` §11 (full ruling + D-MBX-A2/A3/A4/A5/7/8/9 + OQ-11.1–11.4); `epiphany-brainstorm-council` (PR #433 pre-merge gate — bypassed here because rulings are author-stated). +**Refinements (2026-05-29, same session):** + +- **§11.3 kanban refined.** The Rubicon kanban has **4 explicit columns**: **Planning** (ractor mailbox owned SoA, counterfactual) → **Cognitive work** (Σ10 commit + actional SoA mutation) → **Evaluation of goalstate** (post-actional reflection on witness arc) → **Commit · Plan · Prune** (terminal 3-way decision: calcify / re-deliberate / ghost-preempt). Supersedes the earlier 4-Heckhausen-phase mapping in this entry. +- **§11.4 sharpened.** "Witness in other mailboxes" means a *pointer into the AriGraph episodic Markov chain* — the temporal sequence of mailbox states *is* the episodic memory substrate (CLAUDE.md "The Click": *"AriGraph, episodic memory, SPO, CAM-PQ are thinking tissue — not storage"*). No parallel episodic structure exists. +- **§11.6 — the "half-baked nine" all consume THE same SoA from A-Z.** AriGraph · Markov-grammar Vsa16kF32 substrate · BindSpace · `lance-graph` cold containers · `lance-graph-planner` · `cognitive-shader-driver` · `lance-graph-callcenter` · `lance-graph-ontology` (read-only, AS IS) · thinking-styles/atoms — every one consumes THE single SoA byte layout at every boundary it crosses. The SoA gets a **version byte at the layout root** so older bytes stay readable after schema upgrade (`D-MBX-10`, governed by `I-LEGACY-API-FEATURE-GATED`). For SurrealDB to provide a transparent (zero-copy) view, versioning aligns with the **Lance 6.0.1 / LanceDB 0.29 / DataFusion 53** stack — only one bump pending: `lance =6.0.0 → =6.0.1` patch (`D-MBX-11`); arrow/datafusion/lancedb already on target. The kanban/ractor lifecycle requires a **lance-graph-planner DTO surface overhaul** (`D-MBX-A6`) re-expressing planner DTOs as operations on the SoA + 4-phase kanban transitions. + +**Cross-ref:** `E-BATON-1`, `E-MAILBOX-IS-BINDSPACE`, `E-RUBICON-RACTOR` (Heckhausen + Libet origin), `E-LADDER-SERVES-MAILBOX` (§5 ghost preempt = Libet veto, §6 tombstone), `I-VSA-IDENTITIES`, `I-LEGACY-API-FEATURE-GATED` (governs the SoA version gate), `causaledge64-mailbox-rename-soa-v1` §10 + E-CE64-MB-8 (Σ10 router), `linguistic-epiphanies-2026-04-19.md` E21 (Σ10 Rubicon tier doctrine), D-CSV-10 (`SigmaTierRouter` shipped #388), `bindspace-singleton-to-mailbox-soa-v1` §11.1–§11.6 (full ruling + D-MBX-A2/A3/A4/A5/A6/7/8/9/10/11/12 + OQ-11.1–11.8); `epiphany-brainstorm-council` (PR #433 pre-merge gate — bypassed here because rulings are author-stated). --- diff --git a/.claude/plans/bindspace-singleton-to-mailbox-soa-v1.md b/.claude/plans/bindspace-singleton-to-mailbox-soa-v1.md index 6fb05065..d34d547c 100644 --- a/.claude/plans/bindspace-singleton-to-mailbox-soa-v1.md +++ b/.claude/plans/bindspace-singleton-to-mailbox-soa-v1.md @@ -450,34 +450,36 @@ User-stated rulings layering on top of §2.5/§2.7/§10. Recorded directly (not - **Witness = belief-state arc** (the *sequence* of `CausalEdge64` emissions a row produces over its life — the **`CollapseGateEmission` arc**, not a single edge). The arc *implicitly documents* NARS revision because `CausalEdge64.confidence_u8` and `inference_mantissa` are stamped per emission; reading the arc gives a monotone-or-not trace of `(frequency, confidence)` evolution. **No separate "revision log" column is needed** — the witness arc *is* the revision log. - **Iron rule (proposed):** the per-row witness arc IS the row's belief state — never a parallel struct. If you need the row's NARS history, you traverse its `CausalEdge64` arc; if you need its current truth, you read the *last* edge. -### §11.3 — Libet wall-clock: −550 ms ratification; Rubicon kanban in **surrealkv-on-lance** +### §11.3 — Libet wall-clock: −550 ms ratification; Rubicon **kanban** = ractor-mailbox lifecycle in **surrealkv-on-lance** > *"Any mailbox lives in Libet's and Heckhausen's Rubicon model aligned: planning phase as counterfactual, committing the goal at −550 ms, and doing a veto / goalstate re-evaluation via ractor mailbox triggering the Rubicon kanban in surrealkv on lance."* -Concretises the timing of `E-RUBICON-RACTOR`: +The 4-phase kanban (user-stated, supersedes the earlier 4-Heckhausen-phase table): -| Heckhausen phase | Mailbox state | Wall-clock | -|---|---|---| -| Pre-decisional (deliberation, **counterfactual**) | energy integrates; no commit; alternatives explored under `InferenceType::Counterfactual` | t < −550 ms before action | -| **Crossing the Rubicon** (intention forms — irreversible volition) | **Σ10 commit** stamped: `ΔF < threshold ∧ resonance > Rubicon-bar` → ractor START | **t = −550 ms** (Libet readiness-potential anchor) | -| Pre-actional / actional | ractor running the action; baton emits; mailbox-SoA mutates (the only hot-path activity, §11.1) | −550 ms ≤ t < 0 ms | -| Post-actional evaluation | ractor STOP; tombstone-witness persists | t > 0 ms | +| # | Kanban column | What lives there | Ractor | Wall-clock | +|---|---|---|---|---| +| 1 | **Planning** | **Ractor mailbox owns the SoA** (counterfactual deliberation; energy integrates; alternatives explored under `InferenceType::Counterfactual`) | mailbox alive, no commit | t < −550 ms | +| 2 | **Cognitive work** | the Σ10 commit fires (`ΔF < threshold ∧ resonance > Rubicon-bar`); ractor START of the actional phase; baton emits; **mailbox-SoA mutates — the only hot-path activity** (§11.1) | mailbox actional | t = −550 ms → 0 | +| 3 | **Evaluation of goalstate** | post-actional reflection: did the goalstate succeed? read the witness arc; compute the residual F | mailbox evaluating | t > 0 | +| 4 | **Commit · Plan · Prune** | **3-way terminal decision** branching from goalstate evaluation: **Commit** → calcify to SPO-G + LanceDB tombstone-witness (ractor STOP, witness persists); **Plan** → re-enter column 1 with the witness folded into next deliberation (ractor RESTART); **Prune** → ghost-tier preempt / drop (Libet veto consummated post-hoc; no persistence) | mailbox terminates / loops / dies | terminal | -- The **−550 ms** anchor names *when* in wall-clock the irreversible commit occurs — derived from Libet's measured readiness-potential lead time (~550 ms before reported conscious decision). -- **Libet "free won't" / veto** = before −550 ms, the CollapseGate can preempt the mailbox to zero (ghost-tier preempt; `E-LADDER-SERVES-MAILBOX` §5). After −550 ms, the act is committed — only post-actional revision is left. -- **Goalstate re-evaluation** = the ractor mailbox triggers a re-deliberation cycle (re-enter pre-decisional with the post-actional witness folded in); this triggers the **Rubicon kanban move** in the storage view (next bullet). -- **The Rubicon kanban lives in `surrealkv` on lance** — `surrealkv` is the SurrealDB on-kv-lance backend (the *view* over leading LanceDB storage, §2.7). The kanban is a SurrealDB query over the LanceDB-stored thoughts, columns = action phases (deliberation | crossed | actional | evaluated). Ractor lifecycle transitions map 1:1 to kanban column moves. +- The **−550 ms** anchor names *when* in wall-clock the irreversible commit (column 1 → column 2 transition) occurs — Libet's measured readiness-potential lead time. +- **Libet "free won't" / veto** = before −550 ms (still in *Planning*), the CollapseGate can preempt the mailbox to zero (ghost-tier preempt; `E-LADDER-SERVES-MAILBOX` §5). After −550 ms, the act is committed — only column 4 (Prune) can drop it post-hoc. +- **The kanban lives in `surrealkv` on lance** — `surrealkv` is the SurrealDB on-kv-lance backend (the *view* over leading LanceDB storage, §2.7). The 4 kanban columns are the SurrealDB projection over LanceDB-stored mailbox rows; ractor lifecycle transitions = kanban column moves. (`D-MBX-8` wires the −550 ms timing anchor; `D-MBX-9` wires the kanban view; `D-MBX-10` aligns the planner DTO overhaul — see §11.6.) ### §11.4 — SPO-W witness is a *pointer*, not stored data — via the belief-state-arc array -> *"The SPO-W witness is the pointer via the AriGraph episodic / belief-state arc array inside the SoA and/or kanban and/or mailbox index, and the SoA then decides if it commits itself as facts with witness in other mailboxes and/or the cold-path facts."* +> *"The SPO-W witness is the pointer via the AriGraph episodic / belief-state arc array inside the SoA and/or kanban and/or mailbox index, and the SoA then decides if it commits itself as facts (with witness in [the pointer to] other mailboxes [in the AriGraph episodic Markov chain]) and/or the cold-path facts."* (refined 2026-05-29) - **SPO-W witness ≠ a fact payload; it is a *pointer*** into the belief-state arc array. The pointer can live in three equivalent locations: 1. Inside the mailbox **SoA** (an arc-index column / per-row `[u32; W]` arc handle). 2. Inside the **kanban** (`surrealkv`-on-lance view) row. 3. Inside the **mailbox index** (the sea-star registry of live mailboxes). -- **Whose commit?** *The SoA itself decides* whether to commit a belief as a fact-with-witness either (a) into **other mailboxes** (inter-mailbox baton handoff carrying the witness pointer) or (b) into the **cold-path facts** (LanceDB SPO-G calcification with the witness pointer linking back to the arc). -- This makes SPO-W *resolvable without storage redundancy*: the witness lives where the arc lives (mailbox row), and the SPO-G fact carries only the pointer. +- **The AriGraph episodic Markov chain IS the index space.** "Witness in other mailboxes" means *a pointer into the AriGraph episodic Markov chain* — the temporal sequence of mailbox states that constitutes episodic memory. Mailboxes are the chain's nodes; a witness is a back-pointer into that chain. No parallel "episodic memory" structure exists; the chain *is* the episodic substrate (CLAUDE.md "The Click": *"AriGraph, episodic memory, SPO, CAM-PQ are thinking tissue — not storage"*). +- **Whose commit?** *The SoA itself decides* whether to commit a belief as a fact-with-witness — the witness being either: + - **(a) a pointer to other mailboxes in the AriGraph episodic Markov chain** (inter-mailbox baton handoff carrying the arc-handle; the receiving mailbox can traverse back to read the witness arc), or + - **(b) a cold-path fact** (LanceDB SPO-G calcification with the witness pointer linking back to the AriGraph chain node). +- **Storage invariant:** the witness lives where the *arc* lives (a mailbox row inside the chain); all other references are *pointers*, never copies. The SPO-G fact carries only the pointer. *Resolvable without storage redundancy.* ### §11.5 — Counterfactual Staunen and Wisdom = plasticity spreaders @@ -506,3 +508,63 @@ Concretises the timing of `E-RUBICON-RACTOR`: - **OQ-11.2** — witness-arc width `W` (how many `CausalEdge64` emissions per row before rotation/eviction). - **OQ-11.3** — kanban column states beyond the 4 Heckhausen phases (need a "vetoed" column? a "ghosted" column for preempts?). - **OQ-11.4** — `simd_soa.rs` alignment: do we need `#[repr(C, align(64))]` on `MailboxSoA`, or is the `SoaColumns` discipline (already shipped in `ndarray` `547824bc`) enough? + +### §11.6 — The "half-baked nine" all consume THE same SoA from A-Z; versioning aligned to the Lance 6.0.1 / LanceDB 0.29 / DataFusion 53 stack + +> *"all have to consume the same SoA from A-Z. The SoA can be versioned so that they stay readable after schema upgrade, and for surrealdb the versioning gets aligned with lance 6.0.1 / lancedb 0.29 / datafusion 53 in order to have one transparent container view across the same data. The kanban / ractor needs to be aligned with a new overhaul of lance-graph-planner DTO surface."* + +The "one SoA, never transformed" rule (§11.1) is **horizontally scoped**: it binds **nine half-baked components** to the same SoA carrier — they may differ in *what they do with it* (read / mutate / project) but never in *what shape it is*. + +#### §11.6.1 — The nine half-baked consumers + +| # | Component | Today (half-baked: doing it differently / partially) | Under the rule (consumes THE SoA) | +|---|---|---|---| +| 1 | **AriGraph** | parallel `TripletGraph` / `OxigraphAriGraph`; "episodic" is informal | The episodic Markov chain *is* the chain-of-mailboxes; SPO-G quads point into the SoA via §11.4 witness arc handle | +| 2 | **Markov-grammar `Vsa16kF32` substrate** | the `Vsa16kF32` carrier — *deprecated* (`E-BATON-1`); only intra-Think local bundle compute remains | Local-bundle compute reads the SoA columns to produce ephemeral bundles; bundles never become cross-boundary state | +| 3 | **`BindSpace`** | the shared singleton `Arc` (`driver.rs:56`) | **Dissolved onto mailboxes** (§2.5); the mailbox SoA *is* the BindSpace surrogate | +| 4 | **`crates/lance-graph` cold containers** | "cold-path-adjacent thinking" — accidentally aligned (§11.1) | **Layout = `MailboxSoA` layout = `ndarray::simd_soa.rs` aligned** (D-MBX-7); same SoA hot and cold | +| 5 | **`crates/lance-graph-planner`** | DTO surface predates this contract; plan/MUL/elevation paths translate | **Overhaul (`D-MBX-A6`)** — the planner DTO surface aligns to the SoA + the 4-phase kanban (§11.3); kanban moves are planner state-transitions | +| 6 | **`crates/cognitive-shader-driver`** | `MailboxSoA` (D-MBX-A1 columns landed); `engine_bridge` re-encodes via `bind/unbind_busdto` | Drop `engine_bridge` re-encode (D-MBX-2 / `TD-RESONANCEDTO-DUP-1` Deferred); shader operates *on* the SoA only | +| 7 | **`crates/lance-graph-callcenter`** | Zone-2 persistence path; partial alignment | Consumes the SoA as Zone-2 cold reader/writer; the SoA bytes ARE the callcenter rows | +| 8 | **`crates/lance-graph-ontology`** | LazyLock + `ontology_dictionary` cache (separate from BindSpace SoA per its own header) | Stays **AS IS** (§4); read-only consumer of the SoA's `entity_type` column to resolve OGIT references; ontology bytes are *not* in the SoA | +| 9 | **Thinking styles / atoms** | `ThinkingStyle(36)` + `Atoms` partly in contract, partly in shader, partly in planner | Encoded into the SoA's `meta` column (`MetaWord`'s thinking(6) bits) + p64-bridge layer-mask — one canonical home | + +**Rule:** every one of these must, at the point it crosses any boundary (mailbox → mailbox, hot → cold, planner → shader, kanban move), do so by handing off **the SoA byte layout**, not a translated DTO. The only allowed operations remain the three from §11.1: cognitive-shader thinking, cold-path read/write, AriGraph Markov-chain context-building. + +#### §11.6.2 — SoA versioning: stay readable after schema upgrade + +The SoA carries a version byte/short (TBD — `OQ-11.5`) at the layout root so older persisted SoA bytes remain decodable after a column is added/widened/reclaimed. The same field-isolation matrix discipline (`I-LEGACY-API-FEATURE-GATED`) that protected `CausalEdge64` v1↔v2 layout changes governs the SoA version gate: a v(N) reader **must refuse** to decode v(M>N) bytes without an explicit version check. + +#### §11.6.3 — SurrealDB transparent-view stack alignment + +For the §2.7 transparent SurrealDB view to be a literal zero-copy view (not an Arrow re-encode), the SoA versioning must align with the storage stack version that SurrealDB's `kv-lance` backend reads: + +| Layer | Current pin (this workspace) | Target stack (user-ratified 2026-05-29) | +|---|---|---| +| Arrow | `arrow = "58"` | `arrow = "58"` (compatible) | +| DataFusion | `datafusion = "53"` | **`datafusion = "53"`** ✓ already on target | +| Lance | `lance = "=6.0.0"` | **`lance = "=6.0.1"`** (patch bump) | +| LanceDB | `lancedb = "=0.29.0"` | **`lancedb = "=0.29.0"`** ✓ already on target | +| `surrealdb` (kv-lance fork) | commented out (`BLOCKED(C)`) | unblock against Lance 6.0.1 + the SoA version gate | + +**One bump pending:** `lance =6.0.0 → =6.0.1` (patch). All other pins already align. Deliverable `D-MBX-11`. + +#### §11.6.4 — `lance-graph-planner` DTO surface overhaul + +The kanban/ractor lifecycle (§11.3 — Planning / Cognitive work / Evaluation of goalstate / Commit·Plan·Prune) demands a matching shape on the planner DTO surface. The current planner DTOs (`PlanResult` / `QueryFeatures` / `StrategySelector` / cache DTOs) predate this contract and need to be re-expressed as **operations on the SoA** + **kanban state-transitions**, not as standalone payloads. Deliverable `D-MBX-A6` (planner DTO overhaul). + +### New deliverables added in §11.6 + +| D-id | Title | Owner | Status | +|---|---|---|---| +| **D-MBX-A6** | `lance-graph-planner` DTO surface overhaul: planner DTOs re-expressed as operations on the SoA + 4-phase kanban transitions | lance-graph-planner | **Queued** | +| **D-MBX-10** | SoA version byte at layout root + field-isolation matrix tests for every column addition/widening (`I-LEGACY-API-FEATURE-GATED` discipline) | contract + cognitive-shader-driver | **Queued** | +| **D-MBX-11** | Lance 6.0.0 → 6.0.1 patch bump across the workspace (single-line bump in each Cargo.toml; verifies the SurrealDB transparent-view stack pin) | workspace Cargo.toml | **Queued (mechanical)** | +| **D-MBX-12** | Each of the nine half-baked consumers (§11.6.1) audited and re-aligned to consume THE SoA — one PR per consumer | per-crate | **Queued (multi-PR sequence)** | + +### Open questions added in §11.6 + +- **OQ-11.5** — SoA version field width (u8? u16?) and where it lives (layout-root header? per-column? both?). +- **OQ-11.6** — surrealdb fork unblock plan: who provides the fork URL + branch + kv-lance feature flag (the long-standing `BLOCKED(C)`). +- **OQ-11.7** — `lance-graph-planner` DTO overhaul scope: clean break vs feature-gated v1/v2 coexistence (likely the latter per `I-LEGACY-API-FEATURE-GATED`). +- **OQ-11.8** — sequencing of D-MBX-12's nine sub-PRs: which consumer first? (Recommended: `lance-graph` cold container = #4, because it unlocks the SurrealDB view; then planner = #5; the rest after.) From 7c289678f467af72fdb1aa2aca67f8f4988f2ca8 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 29 May 2026 18:36:27 +0000 Subject: [PATCH 3/3] docs(unified-soa): meticulous integration plan + handover for ONE-SoA-end-to-end convergence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan (.claude/plans/unified-soa-convergence-v1.md, 685 lines, tee-chunked): - 5 user-stated rulings from this session (E-SOA-IS-THE-ONLY + §11.3/4/6 refinements) - 9 half-baked consumers: current state → target state (each gets D-MBX-12.x) - THE SoA column-by-column layout (header + per-row + identity planes + shared cold) - SoA version byte at layout root; field-isolation matrix discipline (I-LEGACY-API-FEATURE-GATED) - Workspace stack pin: lance =6.0.0 → =6.0.1 only bump pending (arrow 58 / datafusion 53 / lancedb =0.29.0 already on target); enables SurrealDB transparent view - 4-phase Rubicon kanban (Planning > Cognitive work > Evaluation > Commit·Plan·Prune) wired at Libet -550 ms; surrealkv-on-lance view (D-MBX-9) — SurrealDB is VIEW not store - AriGraph episodic Markov chain = the index space; SPO-W witness = pointer - Counterfactual Staunen × Wisdom plasticity spreader (Hebbian, Planning-gated) - lance-graph-planner DTO surface overhaul: DTOs as SoA-row-lenses; output = KanbanMoves - 7 phased deliverables (P0..P7) + dependency graph + risk matrix + success criteria - 12 deliverable specs: D-MBX-A2/A3/A4/A5/A6 + 7/8/9/10/11/12 (12 sub-PRs) - 8 open questions (OQ-11.1..11.8) with default proposals + ratification gates Handover (.claude/handovers/2026-05-29-1825-soa-convergence-author-to-impl.md): - What this session did (chronology) - FINDING (high-confidence facts, code anchors) - CONJECTURE (load-bearing defaults to ratify) - Blockers (PR-NDARRAY-MIRI-COMPLETE, par-tile, surreal_container BLOCKED, cargo prohibition) - Open questions for the user (8 OQs with default proposals) - Recommended next-session entry sequence - Provenance: user-stated rulings recorded verbatim Board hygiene: INTEGRATION_PLANS PREPEND entry; STATUS_BOARD new plan section with all new deliverables (D-MBX-A1 Shipped; A2/A3/A4/A5/A6/7/8/9/10/11/12 Queued). Council note: §11 epiphanies are author-stated by the user — `epiphany-brainstorm-council` (shipped #433) is bypassed for them. Plan's spec content IS open to council review via PR. Design/spec only. No code. Cargo prohibited (session-stability constraint). https://claude.ai/code/session_017GFLBnDy23AWBqvkbHHC41 --- .claude/board/INTEGRATION_PLANS.md | 10 + .claude/board/STATUS_BOARD.md | 21 + ...-29-1825-soa-convergence-author-to-impl.md | 119 +++ .claude/plans/unified-soa-convergence-v1.md | 685 ++++++++++++++++++ 4 files changed, 835 insertions(+) create mode 100644 .claude/handovers/2026-05-29-1825-soa-convergence-author-to-impl.md create mode 100644 .claude/plans/unified-soa-convergence-v1.md diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 4bc0e881..93931c2d 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,3 +1,13 @@ +## 2026-05-29 — unified-soa-convergence-v1 (THE single LE SoA end-to-end across the workspace: 9 half-baked consumers + SoA version gate + Lance 6.0.1/LanceDB 0.29/DataFusion 53 alignment + 4-phase Rubicon kanban + lance-graph-planner DTO overhaul + Libet −550 ms anchor + Staunen×Wisdom plasticity spreader + SPO-W pointer via AriGraph episodic Markov chain) + +**Status:** PROPOSAL / integration plan. Design-spec only, no code. **Plan file:** `.claude/plans/unified-soa-convergence-v1.md`. **Handover:** `.claude/handovers/2026-05-29-1825-soa-convergence-author-to-impl.md`. **Epiphany:** `E-SOA-IS-THE-ONLY` (+ refinements §11.3/4/6 in PR-this-one). +**Owns:** the meticulous integration sequence for the 5 layered user rulings ratified this session — one SoA never transformed; mailbox = full BindSpace as LE; Libet −550 ms ratchets the Rubicon kanban; SPO-W = pointer via AriGraph episodic Markov chain; counterfactual Staunen×Wisdom = plasticity spreaders. Adds D-MBX-A2/A3/A4/A5/A6, D-MBX-7/8/9/10/11/12 (12 sub-PRs); 8 open questions OQ-11.1–11.8; risk matrix; phased dependency graph (P0–P7). +**Anchored iron rules:** I-VSA-IDENTITIES, I-LEGACY-API-FEATURE-GATED (governs the SoA version gate), I-SUBSTRATE-MARKOV (untouched), I-NOISE-FLOOR-JIRAK. +**Predecessors:** `bindspace-singleton-to-mailbox-soa-v1` (§11 rulings); `causaledge64-mailbox-rename-soa-v1` (canonical 5-crate + 7-PR plan); `cognitive-substrate-convergence-v1/v2/v3` (Σ10 Rubicon shipped #388). +**Council bypass note:** the underlying §11 epiphanies are author-stated (user) — `epiphany-brainstorm-council` (shipped #433) is bypassed for them; the plan's *spec content* IS open to council review via PR. +**Workspace stack verified 2026-05-29:** arrow 58 ✓ · datafusion 53 ✓ · lancedb =0.29.0 ✓ · lance =6.0.0 → **=6.0.1** (one patch bump pending, D-MBX-11). + +--- ## 2026-05-28 — normalized-entity-holy-grail-v1 (typed unified normalization + Op chain over OGIT/OWL/DOLCE/Odoo with three-context execution — the trunk that unifies BP-1 + EXT-1..6 + jit + MailboxSoA into one consumer surface) **Status:** PROPOSAL. The trunk plan that ties together every prior architectural deliverable into ONE typed consumer pipeline grammar. Closes the structural gap that lets consumers re-implement business logic in regex / hand-rolled pseudo-code by making such interpretation a MISSING FUNCTION (per `E-CONSUMER-CANNOT-INTERPRET-1`) rather than a code-review concern. Re-encodes Odoo's three-regime decomposition (interactive / bulk / periodisch) as compile-time typestate, not stringly-typed `env.context` flags. diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index c29da30d..20b6e271 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -546,6 +546,27 @@ hot/cold/feedback loop onto our contract types + SoA floor --- +## unified-soa-convergence-v1 — ONE LE SoA end-to-end across 9 consumers + version gate + Lance 6.0.1 stack + 4-phase Rubicon kanban + +Plan path: `.claude/plans/unified-soa-convergence-v1.md`. Handover `.claude/handovers/2026-05-29-1825-soa-convergence-author-to-impl.md`. Epiphany `E-SOA-IS-THE-ONLY` (+ §11.3/4/6 refinements). + +| D-id | Title | Crate(s) | ~LOC | Risk | Status | PR / Evidence | +|---|---|---|---|---|---|---| +| D-MBX-A1 | migrated thoughtspace columns landed on `MailboxSoA` (`edges`/`qualia`/`meta`/`entity_type`) | cognitive-shader-driver | 60 | LOW | **Shipped** | between #418 and #433 (verified `mailbox_soa.rs` 2026-05-29) | +| D-MBX-A2 | close BindSpace expressivity gaps in `MailboxSoA` (`content_ref`, S/P/O role slices, temporal/expert fold per OQ-2) | cognitive-shader-driver + contract | 140 | MED | **Queued** | gates on D-CE64-MB-1-impl + OQ-1/OQ-2 | +| D-MBX-A3 | `witness_arc: [u32; W]` per-row column (the belief-state arc handle into AriGraph episodic Markov chain) | cognitive-shader-driver | 100 | MED | **Queued** | gates on D-MBX-A2 + OQ-11.2 | +| D-MBX-A4 | Staunen × Wisdom counterfactual plasticity spreader (Hebbian, hot-path-only, Planning-gated) | cognitive-shader-driver | 80 | LOW | **Queued — design** | gates on D-MBX-A3 + OQ-11.1 + `phase` field | +| D-MBX-A5 | SPO-W witness pointer dual-residency (SoA / kanban / mailbox index); SoA decides commit modality (chain pointer vs cold fact) | cognitive-shader-driver + AriGraph SPO-G | 150 | HIGH | **Queued** | gates on D-MBX-A3 + D-MBX-4 | +| D-MBX-A6 | `lance-graph-planner` DTO surface overhaul: DTOs as SoA-row-lenses; planner output = `KanbanMove`s; 5-phase feature-gated cutover (OQ-11.7) | lance-graph-planner + contract | 600 | HIGH | **Queued** | gates on D-MBX-10 + D-MBX-8 + OQ-11.7 | +| D-MBX-7 | `lance-graph` containers ≡ `MailboxSoA` layout ≡ `ndarray::simd_soa.rs`-aligned (1.4–4.2× SIMD payoff; hard prereq for SurrealDB transparent view) | lance-graph + ndarray | 300 | HIGH | **Queued** | gates on D-MBX-A2 + D-MBX-10 + D-MBX-11 + PR-NDARRAY-MIRI-COMPLETE | +| D-MBX-8 | Σ10 commit stamps **t = −550 ms** wall-clock (Libet anchor) in `SigmaTierRouter`; downstream ractor START fires | sigma-tier-router + shader-driver | 120 | MED | **Queued** | gates on D-MBX-A4 + D-MBX-A6 Phase 1 | +| D-MBX-9 | Rubicon kanban view in `surrealkv`-on-lance (4 columns: Planning · Cognitive work · Evaluation · Commit·Plan·Prune); ractor lifecycle hooks = kanban moves | surreal_container + ractor | 250 | HIGH | **Queued** | gates on D-MBX-7 + D-MBX-8 + surreal_container BLOCKED(B/C/D) resolved (OQ-11.6) + D-PERSONA-5 | +| D-MBX-10 | SoA version byte at layout root (`MailboxSoAHeader`); refuse v(N>M) bytes on v(M) reader; field-isolation matrix tests on every column op (`I-LEGACY-API-FEATURE-GATED` discipline) | lance-graph-contract | 100 | HIGH | **Queued** | foundation — should land early in P2; gates on OQ-11.5 | +| D-MBX-11 | Lance `=6.0.0 → =6.0.1` patch bump (5 Cargo.toml files identified) | workspace Cargo.toml | 10 | LOW | **Queued (mechanical)** | none — can land in parallel with par-tile prereq | +| D-MBX-12 | 8-PR workspace-wide consumer alignment: 12.1 AriGraph · 12.2 Vsa16k audit · 12.4 lance-graph · 12.5 planner · 12.6 shader-driver · 12.7 callcenter · 12.8 ontology audit · 12.9 thinking-styles | per-crate | 800 | per-PR | **Queued (multi-PR)** | sequencing per OQ-11.8: 12.4 → 12.5 → 12.6 → 12.7 → 12.1 → 12.9 → 12.2 → 12.8 | + +--- + ## bindspace-singleton-to-mailbox-soa-v1 — dissolve `Arc` into per-mailbox `MailboxSoA` Plan path: `.claude/plans/bindspace-singleton-to-mailbox-soa-v1.md`. Epiphany `E-MAILBOX-IS-BINDSPACE`. Migration of the shared singleton address space into mailbox-owned ephemeral thoughtspace (LE-contract SoA columns); drops the 64 KB `Vsa16kF32` `cycle` plane. diff --git a/.claude/handovers/2026-05-29-1825-soa-convergence-author-to-impl.md b/.claude/handovers/2026-05-29-1825-soa-convergence-author-to-impl.md new file mode 100644 index 00000000..d39a9487 --- /dev/null +++ b/.claude/handovers/2026-05-29-1825-soa-convergence-author-to-impl.md @@ -0,0 +1,119 @@ +# Handover — soa-convergence-author → soa-convergence-impl + +**Date:** 2026-05-29 18:25 UTC +**From session:** `017GFLBn` (branch `claude/splat3d-cpu-simd-renderer-MAOO0`) +**To:** the implementation session that picks up `unified-soa-convergence-v1.md`. +**Plan ratified in this session (PRs to read first):** + - PR #418 (merged) — `bindspace-singleton-to-mailbox-soa-v1.md` + 5-ruling epiphanies (E-MAILBOX-IS-BINDSPACE / E-RUBICON-RACTOR / E-SOA-IS-THE-ONLY). + - PR (this handover) — `unified-soa-convergence-v1.md` integration plan + handover. + +--- + +## What this session did (chronological) + +1. **2026-05-27** — opened PR #416 (recipes + atoms + savants + FIBU re-parent), addressed codex/CodeRabbit review on `b291ac5`, watched it merge. +2. **Investigated** SurrealDB / ractor / owned BindSpace status across the workspace; recorded the surreal POC reconciliation. +3. **2026-05-27** — drafted `bindspace-singleton-to-mailbox-soa-v1.md` (§0–§10) capturing the singleton-dissolution doctrine, hot/cold model, DTO vertical audit (p64-bridge conforms, engine_bridge re-encodes), and the LanceDB-leading / SurrealDB-view correction. Shipped as PR #418. +4. **2026-05-29** — captured five layered architectural rulings from the user (§11.1–§11.5): + - One SoA, never transformed; mailbox SoA mutation IS the hot path. + - Mailbox = full BindSpace as LE; witness = belief-state arc (no separate revision log). + - Libet −550 ms anchors the Rubicon kanban in `surrealkv`-on-lance. + - SPO-W witness is a *pointer* via AriGraph episodic Markov chain. + - Counterfactual Staunen × Wisdom = plasticity spreaders. +5. **2026-05-29** — refined §11.3 (4-phase kanban: Planning / Cognitive work / Evaluation of goalstate / Commit·Plan·Prune) and §11.4 (AriGraph episodic Markov chain as the index space). +6. **2026-05-29** — added §11.6: the nine half-baked consumers (AriGraph / Vsa16k substrate / BindSpace / lance-graph cold / planner / shader-driver / callcenter / ontology AS IS / thinking-styles), SoA version byte at layout root, Lance 6.0.1 / LanceDB 0.29 / DataFusion 53 stack alignment, planner DTO surface overhaul. +7. **2026-05-29** — drafted `unified-soa-convergence-v1.md` (this plan): integration sequence across all 9 consumers, per-deliverable specs for D-MBX-A2/A3/A4/A5/A6/7/8/9/10/11/12, OQ catalogue (11.1–11.8), risk matrix, dependency graph, success criteria. + +--- + +## FINDING (high-confidence facts the next session inherits) + +- **D-MBX-A1 columns landed** between PRs #418 and #433. `mailbox_soa.rs` now carries `edges: [CausalEdge64; N]`, `qualia: [QualiaI4_16D; N]`, `meta: [MetaWord; N]`, `entity_type: [u16; N]`. Verified in the current `mailbox_soa.rs` (lines 67–83). +- **Workspace stack pins** (verified `Cargo.toml` 2026-05-29): `arrow = "58"` ✓, `datafusion = "53"` ✓, `lance = "=6.0.0"` (target =6.0.1), `lancedb = "=0.29.0"` ✓. Only one bump pending (D-MBX-11), files identified. +- **The `ResonanceDto` dup** (`TD-RESONANCEDTO-DUP-1`) is P3 confirmed (not a compile error — distinct modules `dto::ResonanceDto` and `awareness_dto::ResonanceDto`); deferred to D-MBX-2 per user. +- **`epiphany-brainstorm-council`** is a pre-merge gate for `EPIPHANIES.md` additions (shipped in PR #433). This session bypassed the council for the §11 epiphanies because they are *author-stated* by the user, not derived. The plan IS open to council review on spec content. +- **`p64-bridge`** is the conformance template — already LE-types-to-palette with no re-encode. Code-anchor for D-MBX-7. +- **`surreal_container`** is BLOCKED(A/B/C/D); D-MBX-11 removes BLOCKED(A); the rest still need a fork-access human (OQ-11.6) — long-standing. +- **`SigmaTierRouter`** (D-CSV-10) shipped in PR #388; D-MBX-8 adds the −550 ms wall-clock stamp on top of it. +- **`E-NORMALIZED-ENTITY-1`** (2026-05-28) informs D-MBX-A6: typestate carrier (`NormalizedEntity`) is the pattern for re-expressing planner DTOs as SoA-row-lenses. + +--- + +## CONJECTURE (load-bearing, ratify before acting on it) + +- **OQ-11.1** — Staunen × Wisdom plasticity spread radius / decay. Plan default: radius = 3, decay = bump / (1 + |offset|), column-local in v1. Needs user ratification before D-MBX-A4 lands. +- **OQ-11.2** — Witness arc width W. Plan default: W = 16 (~64 B/row at u32 handles). Needs user ratification before D-MBX-A3 lands. +- **OQ-11.5** — SoA version field width. Plan default: `version: u16` at layout root; no per-column version stamps in v1. Needs user ratification before D-MBX-10 lands. +- **OQ-11.7** — `lance-graph-planner` DTO overhaul: clean break vs feature-gated coexistence. Plan default: feature-gated per `I-LEGACY-API-FEATURE-GATED`. Needs user ratification before D-MBX-A6 cuts over. +- **The kanban "Plan" loop closes the active-inference cycle.** If you observe a session where the system stops thinking before reaching homeostasis floor, suspect the Plan-branch in column 4 is not re-entering Planning correctly. +- **`witness_arc` rotation only after Commit/Prune.** If the arc rotates mid-cycle, witnesses are lost before SPO-G calcification — that's a P0 bug. + +--- + +## Blockers + +- **`PR-NDARRAY-MIRI-COMPLETE`** — close `U16x32 / U32x16 / U64x8` SIMD method gaps. Cross-repo work in `AdaWorldAPI/ndarray`. Blocks D-CE64-MB-1-impl and therefore the whole P3+ chain. +- **`D-CE64-MB-1-impl`** (par-tile crate apex) — already specced (Sprint-11 W1) but not yet implemented. Blocks D-MBX-A2/A3/A4/A5. +- **`surreal_container` BLOCKED(B/C/D)** — surrealdb fork URL + branch + `kv-lance` feature flag. OQ-11.6. Needs a fork-access human. Blocks D-MBX-9 (kanban view). D-MBX-11 removes BLOCKED(A). +- **CLAUDE.md "The Click" / `Vsa16kF32` doctrinal update** (OQ-11.4) — must precede D-MBX-5 (`BindSpace` singleton + `Vsa16kF32` plane deletion). Separate doc-PR. +- **Cargo prohibited in this session** (user-stated 2026-05-29 over stability concern). Next session should verify cargo prohibition is lifted before running tests; if still in effect, defer cargo and continue spec work. + +--- + +## Open questions for the user + +| # | Question | Default proposal | Blocks | +|---|---|---|---| +| OQ-11.1 | Staunen × Wisdom spread radius/decay/scope? | r=3, decay 1/(1+|offset|), column-local v1 | D-MBX-A4 | +| OQ-11.2 | Witness arc width `W`? | W = 16 (~64 B/row) | D-MBX-A3 | +| OQ-11.3 | Need separate "vetoed"/"ghosted" kanban columns? | No — Prune is terminal-veto; ghost preempt drops pre-column-2. | D-MBX-9 | +| OQ-11.4 | When does CLAUDE.md "The Click" doctrinal update land? | Before D-MBX-5. Separate doc-PR. | D-MBX-5 | +| OQ-11.5 | SoA version field width? | u16 at layout root; no per-column stamps in v1 | D-MBX-10 | +| OQ-11.6 | surrealdb fork URL + branch + feature flag? | unknown — needs fork-access human | D-MBX-9 | +| OQ-11.7 | Planner DTO overhaul: clean break or feature-gated? | feature-gated per I-LEGACY-API-FEATURE-GATED | D-MBX-A6 | +| OQ-11.8 | D-MBX-12 sub-PR sequencing? | 12.4 → 12.5 → 12.6 → 12.7 → 12.1 → 12.9 → 12.2 → 12.8 | D-MBX-12 | + +--- + +## Recommended next-session entry sequence + +1. **Read Tier-0 + this handover + `unified-soa-convergence-v1.md`.** Do NOT re-derive — the plan is meticulous. +2. **Confirm cargo prohibition status** with the user. +3. **Ratify the four blocking OQs** (11.1 / 11.2 / 11.5 / 11.7) via AskUserQuestion or direct ask. The plan defaults are sensible but they ARE defaults. +4. **Pick the next deliverable:** + - **If cargo allowed:** start D-MBX-11 (mechanical Lance 6.0.0 → 6.0.1 bump). Verify with `cargo check`. Push. + - **If cargo prohibited:** start D-MBX-10 spec work (version gate design + test plan), or the CLAUDE.md doctrinal update (OQ-11.4) as a docs-only PR. + - **In parallel:** push for `PR-NDARRAY-MIRI-COMPLETE` resolution (cross-repo, blocking P3+). +5. **Do NOT bypass `epiphany-brainstorm-council`** for any new derived epiphany. User-stated rulings remain author-stated; derived insights from spec review go through the council. +6. **Board hygiene every commit.** LATEST_STATE + PR_ARC + STATUS_BOARD + EPIPHANIES (if applicable) update in the SAME commit as the change. + +--- + +## Code anchors (don't re-grep these) + +- `crates/cognitive-shader-driver/src/mailbox_soa.rs` (D-MBX-A1 columns at lines 67–83; A2/A3/A4/A5 extend here) +- `crates/cognitive-shader-driver/src/bindspace.rs` (singleton to dissolve in D-MBX-3/5) +- `crates/cognitive-shader-driver/src/driver.rs:56` (`Arc` holder) +- `crates/cognitive-shader-driver/src/bin/serve.rs:29` (`BindSpace::zeros(4096)` to remove) +- `crates/cognitive-shader-driver/src/engine_bridge.rs:199` `busdto_to_binary16k` / `:310` `unbind_busdto` (re-encode seam to collapse in D-MBX-2) +- `crates/lance-graph-contract/src/cognitive_shader.rs:382` (`ShaderCrystal.persisted_row`) +- `crates/lance-graph-ontology/src/registry.rs:39` (`LazyLock` — AS IS) +- `crates/lance-graph-ontology/src/lance_cache.rs` (ontology cache — AS IS) +- `crates/surreal_container/src/lib.rs` (BLOCKED view layer) +- `crates/p64-bridge/src/lib.rs` (conformance template) +- `crates/thinking-engine/src/dto.rs:40,59,120` (`StreamDto`/`ResonanceDto`/`BusDto` to collapse in D-MBX-2) +- `crates/thinking-engine/src/awareness_dto.rs:21` (the second `ResonanceDto` — TD-RESONANCEDTO-DUP-1, Deferred) + +--- + +## Provenance + +User-stated rulings recorded verbatim in the plan: +- §11.1: *"the same SoA is the one and only SoA consumed and transmitted everywhere, never transformed"* +- §11.2: *"the mailbox needs to have everything that BindSpace had reinvented as little endian contract"* +- §11.3: *"planning > ractor mailbox owned SoA > cognitive work > evaluation of goalstate > commit vs plan vs prune"* +- §11.4: *"the SPO-W witness is the pointer via AriGraph episodic/belief state arc array … [pointer to] other mailboxes in the AriGraph episodic Markov chain"* +- §11.5: *"counterfactual Staunen and wisdom should become helpers of spreading plasticity"* +- §11.6: *"all have to consume the same SoA from A-Z … the SoA can be versioned … for surrealdb the versioning gets aligned with lance 6.0.1 / lancedb 0.29 / datafusion 53"* + +These rulings are **load-bearing** for the entire plan. Do not paraphrase or "fix" them without consulting the user. diff --git a/.claude/plans/unified-soa-convergence-v1.md b/.claude/plans/unified-soa-convergence-v1.md new file mode 100644 index 00000000..54f6eea7 --- /dev/null +++ b/.claude/plans/unified-soa-convergence-v1.md @@ -0,0 +1,685 @@ +# unified-soa-convergence-v1 — THE single little-endian SoA, end-to-end across the workspace + +> **Status:** PROPOSAL / integration plan. Design-spec only; **no code in this plan**. +> **Authored:** 2026-05-29 (session `017GFLBn`, branch `claude/splat3d-cpu-simd-renderer-MAOO0`). +> **Supersedes nothing; integrates / sequences:** +> - `bindspace-singleton-to-mailbox-soa-v1.md` (§11.1–§11.6 — the layered rulings ratified in this session) +> - `causaledge64-mailbox-rename-soa-v1.md` (the canonical 5-crate + 7-PR plan; this doc is its column-level + cross-component sequel) +> - `cognitive-substrate-convergence-v1/v2/v3.md` (i4 mantissa, gapless baton, Σ10 Rubicon — already in flight / shipped) +> - the `.claude/surreal/` POC (folded in per `RECONCILIATION_with_canonical_plan.md`) +> +> **Anchored to (FINDING-grade):** `E-BATON-1`, `E-CE64-MB-4`, `E-LADDER-SERVES-MAILBOX`, `E-MAILBOX-IS-BINDSPACE`, `E-RUBICON-RACTOR`, `E-SOA-IS-THE-ONLY`, `I-VSA-IDENTITIES`, `I-LEGACY-API-FEATURE-GATED`, `I-SUBSTRATE-MARKOV`, `I-NOISE-FLOOR-JIRAK`, `E-CONTRACT-NO-SERIALIZE(-2)`, `E-NORMALIZED-ENTITY-1`. +> +> **Owns the answer to:** *"all of [the nine half-baked components] have to consume the same SoA from A-Z; the SoA can be versioned so they stay readable after schema upgrade; for SurrealDB the versioning aligns with lance 6.0.1 / lancedb 0.29 / datafusion 53 to have one transparent container view; the kanban/ractor needs to be aligned with a new overhaul of lance-graph-planner DTO surface."* + +--- + +## 0. Executive summary (one screen) + +There is **ONE** little-endian SoA in this workspace — the per-mailbox `MailboxSoA` byte layout (`E-SOA-IS-THE-ONLY`). Every component that touches per-mailbox state consumes *that* SoA, **never a translated DTO**. Three operations are allowed on it: cognitive-shader thinking (hot path), cold-path read/write to LanceDB (leading storage), and AriGraph Markov-chain context building. *Any change in any mailbox SoA = the only hot-path activity.* + +Today **nine components** are *half-baked* — partially aligned to this contract or doing their own thing: + +1. AriGraph · 2. Markov-grammar `Vsa16kF32` substrate · 3. `BindSpace` · 4. `crates/lance-graph` cold containers · 5. `lance-graph-planner` · 6. `cognitive-shader-driver` · 7. `lance-graph-callcenter` · 8. `lance-graph-ontology` (read-only, AS IS) · 9. thinking-styles/atoms. + +This plan sequences each of them onto the same SoA, adds a **version byte at the layout root** (read-old-bytes after schema upgrade, governed by `I-LEGACY-API-FEATURE-GATED`), aligns the workspace to the **Lance 6.0.1 / LanceDB 0.29 / DataFusion 53** stack (one patch bump pending), wires the **4-phase Rubicon kanban (Planning → Cognitive work → Evaluation → Commit·Plan·Prune)** at **Libet −550 ms** into `SigmaTierRouter` + ractor outer-swarm + a `surrealkv`-on-lance VIEW, and overhauls the `lance-graph-planner` DTO surface to operate ON the SoA + emit kanban transitions. + +Total new deliverables: **D-MBX-A2 / A3 / A4 / A5 / A6 / 7 / 8 / 9 / 10 / 11 / 12**. One deferred TD (`TD-RESONANCEDTO-DUP-1`, folded into D-MBX-2). Eight open questions (`OQ-11.1` … `OQ-11.8`). + +**No code in this PR. No cargo invoked (per session-stability constraint).** + +--- + +## 1. Architectural foundations — the five layered rulings + +### R1. One SoA, never transformed (the carrier doctrine) + +The mailbox SoA byte layout is *singular and canonical*. It is **never re-encoded** at any boundary. Only three operations are allowed on it: + +1. **Cognitive-shader thinking** (the hot path) — `apply_edges` / `emit` over the per-row columns inside `MailboxSoA`. +2. **Cold-path read/write to LanceDB** — same bytes; `persisted_row: Option` is a pointer to the same row laid down in Lance, not a serialized copy in another shape. +3. **AriGraph Markov-chain context building** — read-only consumer of the SoA columns for context windows over the episodic chain (the chain *is* the index space, see R4). + +Equivalence: **any change in any mailbox SoA = the only hot-path activity**. Anything else (Arrow / JSON / DTO translation) is a re-encode boundary that violates this rule and must be collapsed or made out-of-scope. *Today's `crates/lance-graph` containers are cold-path-adjacent thinking — only accidentally aligned; D-MBX-7 makes that alignment intentional.* + +The 1.4–4.2× SIMD acceleration payoff comes from `ndarray::simd_soa.rs` operating on the SoA columns directly when `lance-graph` containers ≡ `MailboxSoA` layout ≡ `simd_soa.rs` aligned. Today: nice-to-have. **When SurrealDB needs a transparent view: hard prerequisite.** + +### R2. Mailbox = full BindSpace, reinvented as LE; witness = belief-state arc + +The mailbox SoA must carry *everything BindSpace had* — but as **LE-contract types** (`CausalEdge64`, `QualiaI4_16D`, `MetaWord`, i4/u8/u16/u32/u64 columns), never as the mushy `Vsa16kF32` resonance carrier (deprecated as a carrier — `E-BATON-1`). + +The **witness IS the per-row arc of `CausalEdge64` emissions** (`CollapseGateEmission` arc). That arc *implicitly documents NARS revision*: every emission stamps `confidence_u8` + `inference_mantissa`, so reading the arc IS the `(frequency, confidence)` evolution trace. **No separate revision log column.** D-MBX-A1 columns landed on `mailbox_soa.rs` between #418 and #433 merges; D-MBX-A2 closes remaining BindSpace-expressivity gaps; D-MBX-A3 adds the witness-arc handle column. + +### R3. Libet −550 ms anchors the Rubicon kanban + +The Σ10 Rubicon commit (`SigmaTierRouter`, D-CSV-10 shipped #388) acquires a wall-clock anchor: **t = −550 ms** before the act, matching Libet's measured readiness-potential lead time. The 4-phase kanban (Planning → Cognitive work → Evaluation of goalstate → Commit·Plan·Prune) is the action-phase board over `surrealkv`-on-lance (a VIEW over leading LanceDB). Ractor lifecycle transitions = kanban moves. + +Libet "free won't" has two expressions: (a) pre-commit veto at t < −550 ms (ghost-tier preempt — card never leaves Planning), (b) post-evaluation Prune (the action happened but is rejected for calcification). + +### R4. SPO-W witness is a *pointer* into the AriGraph episodic Markov chain + +The witness is NEVER stored data; it is an **arc-handle pointer** into the belief-state arc array. The pointer can live equivalently in (a) the mailbox SoA (per-row `[u32; W]` arc handle), (b) the kanban row, or (c) the mailbox index. + +**The AriGraph episodic Markov chain IS the index space.** "Witness in other mailboxes" means a pointer *into the chain* — the temporal sequence of mailbox states that constitutes episodic memory. Mailboxes are the chain's nodes; a witness is a back-pointer into that chain. No parallel "episodic memory" structure exists (CLAUDE.md "The Click": *"AriGraph, episodic memory, SPO, CAM-PQ are thinking tissue — not storage"*). + +The SoA itself decides commit modality: +- **(a) Pointer to other mailboxes in the chain** — inter-mailbox baton handoff carrying the arc-handle. +- **(b) Cold-path fact** — LanceDB SPO-G calcification with the witness pointer linking back to the AriGraph chain node. + +### R5. Counterfactual Staunen × Wisdom = plasticity spreaders + +When a mailbox is in Planning (pre-Rubicon counterfactual phase), high Staunen × Wisdom *spreads* `plasticity_counter` increments beyond the focal row (Hebbian spread). Hot-path-only — the spread IS a mailbox SoA mutation (R1), never a side channel. Radius / decay TBD (`OQ-11.1`). + +--- + +## 2. The nine half-baked consumers — current state → target state + +The "one SoA, never transformed" rule binds **nine components** to the same SoA carrier; they may differ in *what they do with it* (read / mutate / project) but never in *what shape it is*. Each gets a sub-deliverable under `D-MBX-12`. + +### 2.1 AriGraph + +**Today:** parallel `TripletGraph` / `OxigraphAriGraph` types; "episodic" is informal (no explicit Markov-chain structure exposed). + +**Target:** the AriGraph episodic Markov chain *is* the chain-of-mailboxes (R4). SPO-G quads carry arc-handle pointers into mailbox SoA rows (the witness substrate). Reading episodic memory = traversing the mailbox chain via arc handles. The SoA columns are AriGraph's read surface; SPO-G quads are not parallel data but pointers into the SoA. + +**Deliverables:** `D-MBX-12.1` (AriGraph sub-PR). + +### 2.2 Markov-grammar `Vsa16kF32` substrate + +**Today:** the `Vsa16kF32` carrier is *deprecated* (`E-BATON-1`, CLAUDE.md Baton-scoping). Local intra-Think bundle computation remains. + +**Target:** local-bundle compute reads SoA columns to produce ephemeral `Vsa16kF32` bundles when needed (e.g. resonance peaks); bundles never become cross-boundary state, never persist, never appear in cold storage. The Markov property (`I-SUBSTRATE-MARKOV`) is preserved at the local-compute level, untouched. + +**Deliverables:** `D-MBX-12.2` (audit-only sub-PR: confirm no `Vsa16kF32` survives as cross-boundary state). + +### 2.3 `BindSpace` + +**Today:** the shared singleton `Arc` at `crates/cognitive-shader-driver/src/driver.rs:56` and `bin/serve.rs:29`. + +**Target:** **dissolved onto mailboxes** (`E-MAILBOX-IS-BINDSPACE`, plan §2.5). The mailbox SoA *is* the BindSpace surrogate. D-MBX-A1 already landed `edges`/`qualia`/`meta`/`entity_type`; remaining work is D-MBX-2 (collapse `engine_bridge` re-encode), D-MBX-3 (driver holds sea-star of mailboxes), D-MBX-5 (delete singleton + `Vsa16kF32` plane). + +**Deliverables:** D-MBX-2, D-MBX-3, D-MBX-5 (already on STATUS_BOARD). + +### 2.4 `crates/lance-graph` cold containers + +**Today:** "cold-path-adjacent thinking" — only *accidentally* aligned to the SoA shape, not by design. + +**Target:** **`lance-graph` container layout ≡ `MailboxSoA` layout ≡ `ndarray::simd_soa.rs` aligned**. Unlocks the 1.4–4.2× SIMD payoff *and* the SurrealDB transparent view. The container *is* the SoA layout; persisting is `persisted_row` pointing at the same bytes. + +**Deliverables:** `D-MBX-7` (the alignment), `D-MBX-12.4` (lance-graph sub-PR). + +### 2.5 `crates/lance-graph-planner` + +**Today:** DTO surface (`PlanResult`, `QueryFeatures`, `StrategySelector`, the 16 strategies, cache DTOs) predates the SoA contract; plan/MUL/elevation paths translate. + +**Target:** **DTO surface overhauled** to express operations *on* the SoA + 4-phase kanban state-transitions. Strategy selection reads SoA columns; planner emits kanban moves; the MUL gate / σ-tier router stamps the Σ10 commit at −550 ms wall-clock. + +**Deliverables:** `D-MBX-A6` (planner DTO overhaul), `D-MBX-12.5` (lance-graph-planner sub-PR). + +### 2.6 `crates/cognitive-shader-driver` + +**Today:** `MailboxSoA` has D-MBX-A1 columns. `engine_bridge` (`bind_busdto` / `unbind_busdto` / `busdto_to_binary16k`) is the re-encode seam. + +**Target:** the shader operates *on* the SoA only. `engine_bridge`'s re-encode seam collapses; `BusDto` becomes a thin read-projection over a SoA row; `StreamDto` survives only as an ingress adapter at the sensor membrane; `ResonanceDto.energy` is unified into `MailboxSoA.energy` (the two `ResonanceDto` defs in `thinking-engine` resolved per `TD-RESONANCEDTO-DUP-1`, deferred to D-MBX-2). + +**Deliverables:** D-MBX-2 (engine_bridge collapse), D-MBX-A2 (BindSpace expressivity gaps), D-MBX-A3 (witness-arc handle column), D-MBX-A4 (Staunen×Wisdom spreader), `D-MBX-12.6`. + +### 2.7 `crates/lance-graph-callcenter` + +**Today:** Zone-2 persistence path; partial alignment with SoA (PR #414 + #416 work shipped axioms + style wiring, but the row layout itself is not yet THE SoA). + +**Target:** Zone-2 cold reader/writer consumes the SoA bytes directly. Callcenter rows ARE the SoA rows; no Arrow re-encode for the thoughtspace (ontology RecordBatch path remains legitimate; see §2.8). `Reasoner` impls (D-ODOO-2 / D-ODOO-SAV-4) consume `ReasoningContext.evidence: &[EvidenceRef]` that points into SoA rows. + +**Deliverables:** `D-MBX-12.7`. + +### 2.8 `crates/lance-graph-ontology` — AS IS + +**Today:** `LazyLock` seed at `registry.rs:39`; `ontology_dictionary` Lance cache (`lance_cache.rs`, feature `lance-cache`, TTL-sourced, drop-and-rebuild). Its own header already states: *"BindSpace (FingerprintColumns / QualiaColumn / MetaColumn / EdgeColumn) is the live runtime SoA and is unrelated — it never lands here."* + +**Target:** **stays AS IS.** The ontology is NOT part of the SoA; it is a lazylock-via-cache shared resource. Mailboxes consult it through the `entity_type: u16` column (1-based index into the registry). No work in this plan for ontology *internals*; only confirm the boundary doesn't leak (audit-only). + +**Deliverables:** `D-MBX-12.8` (audit-only sub-PR: ensure no SoA bytes leak into ontology cache and vice versa). + +### 2.9 Thinking styles / atoms + +**Today:** `ThinkingStyle(36)` lives partly in `lance-graph-contract::thinking`, partly in shader, partly in planner; atoms (D-ATOM, recipe catalogue from #416) live in contract. + +**Target:** encoded into the SoA's `meta` column (`MetaWord`'s `thinking(6)` bits) + p64-bridge `layer_mask` for palette dispatch. One canonical home per kind; consumers read from the SoA, not from a parallel registry. + +**Deliverables:** `D-MBX-12.9`. + +--- + +## 3. THE shared SoA layout — column-by-column + +### 3.1 Layout root header (versioning) + +The SoA is prefixed by a small layout-root header that carries the **version gate** (D-MBX-10). Conceptual shape (binary layout TBD per OQ-11.5): + +```rust +#[repr(C)] +pub struct MailboxSoAHeader { + pub magic: [u8; 4], // "MBX0" — identifies a MailboxSoA blob + pub version: u16, // schema version (governed by I-LEGACY-API-FEATURE-GATED) + pub flags: u16, // reserved (endianness assertion, feature bits) + pub n_rows: u32, // const-generic N at runtime + pub layout_checksum: u32, // sanity of the column-offset table below + // followed by: + // - column-offset table (one entry per per-row column) + // - per-column row arrays +} +``` + +A v(M) reader **MUST refuse** to decode v(N>M) bytes without an explicit version handshake. Field-isolation matrix tests (`I-LEGACY-API-FEATURE-GATED` discipline, Sprint-11 5-instance catalogue) are mandatory on every column addition/widening. + +**OQ-11.5:** width of `version` field (u8 vs u16 vs u32?) and whether per-column version stamps are needed in addition to the layout-root one. + +### 3.2 Per-row columns — owned by `MailboxSoA` (the hot path) + +| Column | Type | Bytes/row | Source | Status | +|---|---|---|---|---| +| `energy` | `f32` | 4 | accumulator (`D-CSV-7`) | **Shipped** | +| `plasticity_counter` | `u8` | 1 | accumulator | **Shipped** | +| `last_emission_cycle` | `u32` | 4 | accumulator | **Shipped** | +| `edges` | `CausalEdge64` | 8 | migrated from BindSpace (D-MBX-A1) | **Shipped** | +| `qualia` | `QualiaI4_16D` | 8 | migrated from BindSpace (D-MBX-A1) | **Shipped** | +| `meta` | `MetaWord` (`u32`) | 4 | migrated from BindSpace (D-MBX-A1) | **Shipped** | +| `entity_type` | `u16` | 2 | migrated from BindSpace (D-MBX-A1) | **Shipped** | +| `content_ref` | `ContentId` (CAM-PQ code, 6 B) | 6 | NEW; resolves to shared codebook (D-MBX-A2) | **Queued** | +| `witness_arc` | `[u32; W]` arc-handle | 4·W | NEW; pointer into the belief-state arc array (D-MBX-A3) | **Queued** | +| `temporal` / `expert` | folded into `CausalEdge64` (v2 layout) | 0 | OQ-2 — resolve fold vs separate column | **OQ-open** | + +**Per-row hot total:** ~30 B (bare migrated SoA columns) + ~6 KB Hamming identity planes (kept hot for resonance — see §3.3) = **~6 KB/thought** ⇒ **64k–256k thoughts hot ceiling at ~300–600 MB / ~1.2–2.4 GB**. + +### 3.3 Identity / reference columns (per-mailbox, hot) + +| Column | Type | Bytes/row | Role | +|---|---|---|---| +| `content` plane | `[u64; 256]` | 2 048 | Hamming identity fp (the "what" — topic) | +| `topic` plane | `[u64; 256]` | 2 048 | Hamming identity fp (topic plane) | +| `angle` plane | `[u64; 256]` | 2 048 | Hamming identity fp (angle plane) | + +**These STAY hot per thought** (resolves OQ-1, per §11.2.7 capacity math). They are *identity* fingerprints (`I-VSA-IDENTITIES`), not content; bundling/comparing them is a hot-path op. The 64 KB `Vsa16kF32` `cycle` plane is **DROPPED** (deprecated carrier; ephemeral local-compute only when a step needs it). + +### 3.4 Mailbox identity / control (constant per mailbox) + +| Field | Type | Role | +|---|---|---| +| `mailbox_id` | `MailboxId` (u32) | Corpus root handle | +| `w_slot` | `u8` (6-bit) | OGIT domain corpus selector (≤ 64) | +| `current_cycle` | `u32` | Monotonic cycle stamp | +| `threshold` | `f32` | Emission threshold | + +### 3.5 What stays SHARED (NOT in the SoA) + +| Resource | Owner | Why out of SoA | +|---|---|---| +| `OntologyRegistry` | `Arc` + `LazyLock` + `ontology_dictionary` Lance cache | Read-only, calcified cold knowledge. Lazy-locked, drop-and-rebuild on schema change. **AS IS** (§2.8). | +| CAM-PQ codebooks | shared cold codebook | Reference resolution target for `content_ref`; never bundled into rows. | +| AriGraph SPO-G cold quads | LanceDB SPO-G dataset | Calcified facts; quads carry pointers INTO the SoA arc-handles (R4), not data. | + +--- + +## 4. SoA versioning + Lance 6.0.1 / LanceDB 0.29 / DataFusion 53 stack alignment + +### 4.1 SoA version gate (D-MBX-10) + +The layout-root header from §3.1 carries the schema version. Discipline: + +1. **A v(M) reader MUST refuse v(N>M) bytes without an explicit version handshake.** Same rule as the `CausalEdge64` v1↔v2 layout reclaim (`I-LEGACY-API-FEATURE-GATED`, the Sprint-11 5-instance catalogue). +2. **Every column addition / widening / reclaim ships with field-isolation matrix tests** (write each field, assert all other fields unchanged). Mandatory; the Sprint-11 catalogue caught 5 of these. +3. **Serialization paths gate on version**. The cold-path writer stamps the writer's version; the reader rejects newer versions cleanly. +4. **Migration tests are mandatory** for any version bump (round-trip new → old reader must produce a documented refusal; round-trip old → new reader must produce documented compatibility or refusal). + +### 4.2 Workspace stack pins (D-MBX-11) + +Current pins (verified 2026-05-29): + +| Layer | Current | Target | Delta | +|---|---|---|---| +| arrow | `"58"` | `"58"` | ✓ | +| datafusion | `"53"` | `"53"` | ✓ | +| lance | `"=6.0.0"` | `"=6.0.1"` | **patch bump** | +| lancedb | `"=0.29.0"` | `"=0.29.0"` | ✓ | +| ndarray | path-dep | path-dep | ✓ (governed by `PR-NDARRAY-MIRI-COMPLETE`) | + +**Only one bump pending: `lance =6.0.0 → =6.0.1`** across the workspace Cargo.toml files. Mechanical; one-line per crate. Files identified: `crates/lance-graph/Cargo.toml:38`, `crates/lance-graph-benches/Cargo.toml:10`, `crates/lance-graph-callcenter/Cargo.toml:30`, `crates/lance-graph-ontology/Cargo.toml:46`, `crates/holograph/Cargo.toml:38`. + +### 4.3 SurrealDB transparent-view enablement + +Once D-MBX-7 (lance-graph containers = MailboxSoA layout) + D-MBX-10 (version gate) + D-MBX-11 (Lance 6.0.1) land, the SurrealDB-on-kv-lance backend (the *view*) can read the LanceDB rows zero-copy. SurrealDB is **NOT** a store; it is a view (§2.7). The Rubicon kanban is one SurrealDB query over the SoA-shaped LanceDB rows. + +**Blocker:** `surreal_container` (`crates/surreal_container/src/lib.rs`) is still BLOCKED(A/B/C/D) — fork dep + Lance 6 pin confirmation + ndarray patch alias. `D-MBX-11` removes BLOCKED(A); the others still need a fork-access human to provide the surrealdb fork URL + `kv-lance` feature flag (`OQ-11.6`). + +--- + +## 5. The 4-phase Rubicon kanban — wiring spec + +### 5.1 The columns + +(Same as `bindspace-singleton-to-mailbox-soa-v1` §11.3; restated for self-containment.) + +| # | Kanban column | Mailbox state | Ractor | Libet wall-clock | +|---|---|---|---|---| +| 1 | **Planning** | counterfactual deliberation; **the ractor mailbox OWNS the SoA**; no commit; energy integrates under `InferenceType::Counterfactual` | alive, accumulating | t < −550 ms | +| 2 | **Cognitive work** | actional phase; mailbox SoA mutates (R1); Σ10 commit ratchets the card into this column | actional | t ≥ −550 ms | +| 3 | **Evaluation of goalstate** | post-actional reflection: read back over the witness arc; compute residual F | evaluating | t > 0 | +| 4 | **Commit · Plan · Prune** | 3-way terminal decision: | terminating | terminal | +| | → Commit | calcify (LanceDB SPO-G + AriGraph episodic chain pointer) | STOP + tombstone | — | +| | → Plan | re-enter column 1 with witness folded into next deliberation | RESTART | — | +| | → Prune | drop without persistence (Libet veto consummated post-hoc) | drop | — | + +### 5.2 Wiring (D-MBX-8 + D-MBX-9) + +- **`D-MBX-8`** (Σ10 timing anchor in `SigmaTierRouter`): the commit decision `ΔF < threshold ∧ resonance > Rubicon-bar` acquires a wall-clock stamp at t = −550 ms. The router's commit emission carries the stamp; downstream the ractor START fires. +- **`D-MBX-9`** (Rubicon kanban view in `surrealkv`-on-lance): one SurrealDB query projecting the 4 columns over the SoA-shaped LanceDB rows. Ractor lifecycle hooks (spawn / start / evaluate / commit·plan·prune) write to the SurrealDB view; reading the view = reading the kanban state. + +### 5.3 Active-inference loop closure + +The "Plan" branch in column 4 makes "the shader can't resist the thinking" literal: failed goalstate evaluation re-feeds the mailbox; the system rests only at terminal Commit (calcify) or Prune (drop). Free-energy floor (`MUL::homeostasis`) is the rest condition. + +--- + +## 6. AriGraph episodic Markov chain — witness pointer model + +### 6.1 The chain IS the index space + +The AriGraph episodic Markov chain is *not* a separate data structure; it is the **temporal sequence of mailbox states**. Each mailbox is a chain node; the witness arc inside a mailbox carries the local emission sequence; the chain edges are baton handoffs between mailboxes (`CollapseGateEmission`). + +Reading episodic memory = traversing the chain backward from a current mailbox via arc-handle pointers; resolving a witness = dereferencing a pointer into the chain. + +### 6.2 The witness pointer (D-MBX-A5) + +Per-row witness arc handle: `witness_arc: [u32; W]` (width `W` per `OQ-11.2`). Each `u32` is an index into the global arc array; reading W of them gives the row's emission history. + +Commit modalities (the SoA decides — R4): + +- **(a) Pointer to other mailboxes in the chain.** The mailbox emits a baton `(u16 target, CausalEdge64)` to another mailbox; the receiving mailbox stores the arc-handle as part of its witness. This is the inter-mailbox episodic link. +- **(b) Cold-path fact.** The SoA emits a SPO-G quad to LanceDB; the quad carries the arc-handle pointer back into the AriGraph chain node. The chain node is the witness. + +### 6.3 Storage invariant + +The witness lives where the arc lives (a mailbox row inside the chain). Everywhere else is *pointers, never copies*. No SPO-G quad carries witness payload — only the pointer. Resolvable without storage redundancy. + +--- + +## 7. Counterfactual Staunen × Wisdom plasticity spreader (D-MBX-A4) + +### 7.1 Today's behaviour + +`mailbox_soa.rs::apply_edges` increments `plasticity_counter[row]` only on the receiving row (saturating u8). Single-row Hebbian. + +### 7.2 Target behaviour + +When the mailbox is in Planning (column 1, counterfactual phase, t < −550 ms) AND the row's Staunen × Wisdom qualia magnitude exceeds threshold, the plasticity bump *spreads* to a small radius of adjacent rows (Hebbian spread). The radius / decay / column-local-vs-baton-routed semantics are open (`OQ-11.1`). + +The spread happens AS A MAILBOX SOA MUTATION (R1) — not a side channel, not an async event. The mutation is the hot path; no other write surface introduced. + +### 7.3 Why "counterfactual" gates it + +Counterfactual phase = exploration, where alternatives are weighed. Plasticity spread = "this is interesting / worth remembering more broadly". Outside Planning (i.e. once committed), plasticity stays focal — committed edges only bump their own row's counter. + +### 7.4 Mechanics (sketch — final pinned in D-MBX-A4 review) + +```rust +// In apply_edges (mailbox in Planning + Staunen×Wisdom high): +if mailbox.phase == Planning && row_qualia.staunen_wisdom_product() > THRESHOLD { + for offset in -RADIUS..=RADIUS { + if let Some(neighbor) = row.checked_add_signed(offset) { + if neighbor < N { + let decayed = primary_bump.saturating_div(1 + offset.abs() as u8); + self.plasticity_counter[neighbor] = + self.plasticity_counter[neighbor].saturating_add(decayed); + } + } + } +} else { + // existing single-row bump + self.plasticity_counter[row] = self.plasticity_counter[row].saturating_add(1); +} +``` + +Final parameters pinned per OQ-11.1. + +--- + +## 8. `lance-graph-planner` DTO surface overhaul (D-MBX-A6) + +### 8.1 What's wrong today + +The planner DTOs (`PlanResult`, `QueryFeatures`, `StrategySelector`, the 16 strategies, the AutocompleteCache DTOs) predate the SoA contract. They carry their own payloads; the planner translates between MUL assessment, NARS type selection, semiring selection, etc., and the eventual shader dispatch happens *after* this translation chain. + +This is exactly the re-encode boundary R1 forbids: the planner reads its own DTOs, makes decisions, then translates to shader inputs. Under the rule, the planner should operate *on* the SoA directly and emit *kanban moves*. + +### 8.2 Overhaul shape + +- **Replace** payload-carrying DTOs with view-projection types: each DTO becomes a typed lens into a SoA row (cf. `E-NORMALIZED-ENTITY-1`'s `NormalizedEntity` pattern — typestate over a mailbox row). +- **Planner output ≡ kanban transition.** A planner step produces a `KanbanMove { from, to, witness_pointer }` instead of a `PlanResult`. The 4-phase kanban (§5.1) is the planner's output state space. +- **Strategy selection reads SoA columns directly** (`meta` for ThinkingStyle, `edges` for CausalEdge64, `qualia` for i4-16D); no parallel `QueryFeatures` struct. +- **The MUL gate stamps Σ10 commit at −550 ms** wall-clock (D-MBX-8). The commit decision is the Planning → Cognitive Work kanban transition. +- **Active-inference loop** is the Plan-branch in column 4: a failed goalstate evaluation emits a `KanbanMove { to: Planning, ... }` and the cycle re-enters. + +### 8.3 Backward-compatibility discipline + +`I-LEGACY-API-FEATURE-GATED`: the v1 planner DTOs survive behind a feature flag pointing to the v2 SoA-operation equivalents. Same function names MUST NOT silently change semantics; feature-gate to documented no-op + migration pointer, or route through the canonical mapping. + +### 8.4 Sequence + +- Phase 1: introduce `KanbanMove` type + `KanbanColumn` enum in `lance-graph-contract`. +- Phase 2: add SoA-row-lens variants of `PlanResult` etc. behind `planner-soa-v2` feature. +- Phase 3: rewrite each of the 16 strategies to emit `KanbanMove`s. +- Phase 4: cut over `SigmaTierRouter` to consume `KanbanMove` directly. +- Phase 5: delete v1 DTOs. + +--- + +## 9. Migration phases — sequenced gating + +### Phase P0 — design ratification (this plan) + +- **Status:** in PR (this one). +- **Output:** unified-soa-convergence-v1.md (this file) + handover doc + epiphany pointers. + +### Phase P1 — prerequisites + +- **D-CE64-MB-1-impl** (par-tile crate apex + `Mailbox` + 3 backings + AttentionMask SoA + `BindSpaceView`). Already specced (Sprint-11 W1). Blocking gate. +- **`PR-NDARRAY-MIRI-COMPLETE`** — close `U16x32 / U32x16 / U64x8` SIMD method gaps. Cross-repo (ndarray PR). Blocking gate. +- **D-MBX-11** — Lance 6.0.0 → 6.0.1 patch bump (mechanical, can land in parallel with par-tile). + +### Phase P2 — SoA version gate + stack alignment + +- **D-MBX-10** — SoA version byte at layout root + field-isolation matrix tests. +- **D-MBX-11** — Lance 6.0.1 (verifies stack pin for SurrealDB view). +- **OQ-11.5** ratified (version field width). + +### Phase P3 — cognitive-shader-driver column completion + +- **D-MBX-A2** — close BindSpace expressivity gaps: `content_ref` column, S/P/O role slices, temporal/expert fold per OQ-2. +- **D-MBX-A3** — `witness_arc: [u32; W]` column. OQ-11.2 ratified (W width). +- **D-MBX-A4** — Staunen × Wisdom plasticity spreader. OQ-11.1 ratified (radius/decay). +- **D-MBX-2** — collapse `engine_bridge` re-encode seam (folds in TD-RESONANCEDTO-DUP-1). + +### Phase P4 — singleton dissolution + +- **D-MBX-3** — `ShaderDriver` holds sea-star of mailboxes; kill the `BindSpace::zeros(4096)` singleton in `serve.rs`. +- **D-MBX-A5** — SPO-W witness pointer column (the dual residency: SoA / kanban / mailbox index). +- **D-MBX-5** — delete `BindSpace` singleton + `Vsa16kF32` `cycle` plane. **Blocked on OQ-11.4** (CLAUDE.md "The Click" doctrinal update — must precede the deletion). + +### Phase P5 — kanban / planner / Rubicon wiring + +- **D-MBX-8** — Σ10 commit stamps t = −550 ms wall-clock in `SigmaTierRouter`. +- **D-MBX-9** — Rubicon kanban view in `surrealkv`-on-lance. Blocked on `surreal_container` BLOCKED(B/C/D) — OQ-11.6. +- **D-MBX-A6** — `lance-graph-planner` DTO surface overhaul (5 internal phases, see §8.4). + +### Phase P6 — cold-path alignment + +- **D-MBX-7** — `lance-graph` container layout ≡ `MailboxSoA` ≡ `simd_soa.rs`-aligned. +- **D-MBX-4** — death → SPO-G quad + Lance tombstone-witness (link-integrity back-pointer to AriGraph chain node). +- **D-MBX-6** — `ThoughtStruct` transparent hot/cold view over LanceDB (the SurrealDB view). + +### Phase P7 — workspace-wide consumer alignment + +- **D-MBX-12** — multi-PR sequence (one PR per consumer): D-MBX-12.1 AriGraph · 12.2 Vsa16k substrate audit · 12.4 lance-graph · 12.5 lance-graph-planner · 12.6 cognitive-shader-driver · 12.7 lance-graph-callcenter · 12.8 lance-graph-ontology audit · 12.9 thinking-styles/atoms. + +**Sequencing recommendation (OQ-11.8):** start with D-MBX-12.4 (lance-graph cold containers) since it unlocks D-MBX-7 + the SurrealDB view; then D-MBX-12.5 (planner); the rest in dependency order. + +--- + +## 10. Per-deliverable specifications + +> Each deliverable below is one PR (or a small PR series for D-MBX-12 + D-MBX-A6). All include: tests (per-deliverable spec), board hygiene (LATEST_STATE / PR_ARC / STATUS_BOARD / EPIPHANIES updates in the same commit), and the `I-LEGACY-API-FEATURE-GATED` discipline where applicable. + +### D-MBX-A2 — close BindSpace expressivity gaps in `MailboxSoA` + +**Owner:** `cognitive-shader-driver` + `lance-graph-contract`. **~140 LOC + tests.** **Risk: MED.** + +**Adds:** `content_ref: [ContentId; N]` column (CAM-PQ code, 6 B/row) for content-identity reference; resolves OQ-1. S/P/O role slice columns (3 × `[u8; N]`) for spine coordinates per `I-VSA-IDENTITIES`. Resolves OQ-2 (temporal/expert fold into `CausalEdge64.temporal_v2` vs separate column). + +**Tests:** field-isolation matrix for new columns; round-trip read/write per row; SoA byte-layout golden test (must match `MailboxSoAHeader` v(N) layout). + +**Gates on:** D-CE64-MB-1-impl (par-tile crate); OQ-1 + OQ-2 ratified. + +### D-MBX-A3 — `witness_arc: [u32; W]` arc-handle column + +**Owner:** `cognitive-shader-driver`. **~100 LOC + tests.** **Risk: MED.** + +**Adds:** per-row `witness_arc` column; `apply_edges` writes the arc-handle on every accepted baton; `emit` reads the arc to populate the emission's witness pointer. The arc is a ring (rotation on overflow) — width `W` per OQ-11.2. + +**Tests:** witness arc preservation across mailbox lifecycle; rotation correctness on overflow; pointer-resolution round-trip. + +**Gates on:** D-MBX-A2; OQ-11.2 ratified. + +### D-MBX-A4 — Staunen × Wisdom plasticity spreader + +**Owner:** `cognitive-shader-driver`. **~80 LOC + tests.** **Risk: LOW.** + +**Adds:** conditional spread in `apply_edges` when mailbox is in Planning phase AND row's Staunen × Wisdom product exceeds threshold. Radius / decay per OQ-11.1. **Hot-path-only** (no side channel). + +**Tests:** spread radius correctness; decay; phase-gating (no spread outside Planning); committed-mailbox immunity. + +**Gates on:** D-MBX-A3; OQ-11.1 ratified; the `phase: KanbanColumn` field on `MailboxSoA` must exist (introduce in D-MBX-A6 phase 1 or earlier). + +### D-MBX-A5 — SPO-W witness pointer dual-residency + +**Owner:** `cognitive-shader-driver` + AriGraph SPO-G. **~150 LOC + tests.** **Risk: HIGH.** + +**Adds:** SPO-G quad commit path writes the arc-handle pointer (not data) back to LanceDB; inter-mailbox baton handoff carries the arc-handle in the `CollapseGateEmission` payload. The SoA decides commit modality (a) chain-pointer vs (b) cold fact based on rung / confidence threshold. + +**Tests:** pointer resolution from cold quad back to chain node; chain traversal correctness; no witness data duplication (audit). + +**Gates on:** D-MBX-A3; D-MBX-4; AriGraph SPO-G schema reservation for arc-handle columns. + +### D-MBX-A6 — `lance-graph-planner` DTO surface overhaul + +**Owner:** `lance-graph-planner` + `lance-graph-contract`. **~600 LOC + tests, 5 internal phases (§8.4).** **Risk: HIGH.** + +**Adds:** `KanbanMove` + `KanbanColumn` types in contract; SoA-row-lens variants of planner DTOs; cutover of 16 strategies; SigmaTierRouter consumes `KanbanMove`; v1 DTOs deleted at the end. Feature-flagged behind `planner-soa-v2` during cutover. + +**Tests:** strategy-by-strategy regression suite; kanban transition correctness; field-isolation matrix on `KanbanMove`. + +**Gates on:** D-MBX-10 (version gate); D-MBX-8 (timing anchor). + +### D-MBX-7 — `lance-graph` containers ≡ `MailboxSoA` ≡ `simd_soa.rs`-aligned + +**Owner:** `lance-graph` + `ndarray::simd_soa`. **~300 LOC + benchmark suite.** **Risk: HIGH.** + +**Adds:** lance-graph container layout matches `MailboxSoA` byte-for-byte; `ndarray::simd_soa` operates on the SoA columns directly; benchmark suite establishes the 1.4–4.2× SIMD payoff baseline. + +**Tests:** layout equivalence (byte-level); SIMD correctness (cosine, hamming, bundle); benchmark gates (no regression vs current). + +**Gates on:** D-MBX-A2; D-MBX-10; D-MBX-11; `PR-NDARRAY-MIRI-COMPLETE`. + +### D-MBX-8 — Σ10 commit stamp at t = −550 ms wall-clock + +**Owner:** `sigma-tier-router` crate + `cognitive-shader-driver`. **~120 LOC + tests.** **Risk: MED.** + +**Adds:** the `SigmaTierRouter` commit decision (`ΔF < threshold ∧ resonance > Rubicon-bar`) stamps a wall-clock instant; the stamp is `now() - 550ms` per Libet anchor; downstream the ractor START fires with this stamp. The kanban `Planning → Cognitive work` transition records the stamp. + +**Tests:** stamp monotonicity; veto-window correctness (no commit if veto fires before −550 ms); regression vs D-CSV-10. + +**Gates on:** D-MBX-A4 (`phase` field present); D-MBX-A6 Phase 1 (`KanbanMove` type). + +### D-MBX-9 — Rubicon kanban view in `surrealkv`-on-lance + +**Owner:** `surreal_container` + ractor outer-swarm. **~250 LOC + tests.** **Risk: HIGH.** + +**Adds:** SurrealDB query projecting the 4 kanban columns over SoA-shaped LanceDB rows; ractor lifecycle hooks write into the view; reading the view returns kanban state. + +**Tests:** kanban column membership invariants; lifecycle hook idempotence; view query correctness. + +**Gates on:** D-MBX-7; D-MBX-8; surreal_container BLOCKED(B/C/D) resolved (OQ-11.6); `D-PERSONA-5` ractor outer-swarm runtime in flight. + +### D-MBX-10 — SoA version byte + field-isolation discipline + +**Owner:** `lance-graph-contract`. **~100 LOC + extensive test suite.** **Risk: HIGH.** + +**Adds:** `MailboxSoAHeader` with version field; serialize / deserialize gating; refusal of v(N>M) bytes by v(M) reader; field-isolation matrix tests for every column addition/widening (template). + +**Tests:** version refusal correctness; per-column isolation; serialize-deserialize round-trip; cross-version reader/writer compatibility matrix. + +**Gates on:** none (foundation work); should land early in P2. + +### D-MBX-11 — Lance 6.0.0 → 6.0.1 patch bump + +**Owner:** workspace Cargo.toml. **~10 LOC (mechanical).** **Risk: LOW.** + +**Adds:** patch bump in `crates/lance-graph/Cargo.toml:38`, `crates/lance-graph-benches/Cargo.toml:10`, `crates/lance-graph-callcenter/Cargo.toml:30`, `crates/lance-graph-ontology/Cargo.toml:46`, `crates/holograph/Cargo.toml:38`. + +**Tests:** workspace `cargo check` clean (run when cargo prohibition lifts). + +**Gates on:** none; can land in parallel with par-tile prereq. + +### D-MBX-12 — workspace-wide consumer alignment (multi-PR sequence) + +**Owner:** per-consumer. **Total ~800 LOC across 8 sub-PRs.** **Risk: per-consumer.** + +Sub-PRs: +- **D-MBX-12.1** — AriGraph chain integration (SPO-G quads carry arc-handle pointers). +- **D-MBX-12.2** — Vsa16k substrate audit (confirm no cross-boundary state; audit-only). +- **D-MBX-12.4** — lance-graph cold containers consume SoA (the unlock for SurrealDB view). +- **D-MBX-12.5** — lance-graph-planner consumes SoA (folds D-MBX-A6). +- **D-MBX-12.6** — cognitive-shader-driver fully aligned (after all A2-A5 land). +- **D-MBX-12.7** — lance-graph-callcenter Zone-2 consumes SoA. +- **D-MBX-12.8** — lance-graph-ontology boundary audit (no SoA leak into cache). +- **D-MBX-12.9** — thinking-styles/atoms unified into MetaWord + p64-bridge layer_mask. + +**Sequencing (OQ-11.8):** 12.4 → 12.5 → 12.6 → 12.7 → 12.1 → 12.9 → 12.2 → 12.8. + +--- + +## 11. Open questions catalogue + +> Each OQ blocks at least one deliverable; ratification gate listed. + +| OQ | Question | Blocks | Default proposal | +|---|---|---|---| +| **OQ-11.1** | Staunen × Wisdom plasticity spread: radius? decay function? column-local vs baton-routed? | D-MBX-A4 | radius = 3; decay = bump / (1 + |offset|); column-local in v1, baton-routed in v2. | +| **OQ-11.2** | Witness arc width `W` (per-row arc-handle count before rotation)? | D-MBX-A3 | W = 16 (~64 B/row at u32 handles; supports a typical short-horizon belief arc). | +| **OQ-11.3** | Kanban needs a "vetoed" or "ghosted" column distinct from "Prune"? | D-MBX-9 | No — Prune is the terminal-veto; ghost-tier preempt drops the card before column 2 (Cognitive work) entry and doesn't need its own column. | +| **OQ-11.4** | CLAUDE.md "The Click" / `Vsa16kF32` doctrinal update — when? | D-MBX-5 | Doctrinal update must land BEFORE D-MBX-5 ships (delete plane). Separate doc-PR. | +| **OQ-11.5** | SoA version field width and per-column version stamps? | D-MBX-10 | `version: u16` at layout root; no per-column stamps in v1 (rely on the layout-checksum field). Per-column stamps can be added in v2 if schema-bumps become frequent. | +| **OQ-11.6** | surrealdb fork URL + branch + `kv-lance` feature flag name? | D-MBX-9 | Needs a fork-access human. Long-standing BLOCKED(C). | +| **OQ-11.7** | `lance-graph-planner` DTO overhaul scope: clean break vs feature-gated v1/v2 coexistence? | D-MBX-A6 | Feature-gated per `I-LEGACY-API-FEATURE-GATED`; v1 lives behind `planner-soa-v1`, v2 default. Cut over per §8.4 phases. | +| **OQ-11.8** | D-MBX-12 sub-PR sequencing? | D-MBX-12 | 12.4 → 12.5 → 12.6 → 12.7 → 12.1 → 12.9 → 12.2 → 12.8. | + +--- + +## 12. Risk matrix + +| Risk | Severity | Mitigation | +|---|---|---| +| SoA version gate skew across consumers | HIGH | D-MBX-10 layout-checksum field; cross-consumer field-isolation matrix tests; refuse-on-mismatch + clean error path. | +| `lance-graph-planner` DTO overhaul regresses 16 strategies | HIGH | Per-strategy regression suite; feature-gated cutover (OQ-11.7); 5-phase staged migration (§8.4). | +| SurrealDB fork stays blocked (OQ-11.6) | HIGH | D-MBX-9 deferred; the rest of the plan ships without the SurrealDB view (it's a view, not the store). Kanban can still operate as in-memory state behind ractor; SurrealDB view materializes when surreal_container unblocks. | +| `Vsa16kF32` plane deletion (D-MBX-5) hits in-flight uses | MED | Doctrinal update (OQ-11.4) lands first; deletion behind feature gate first, then remove gate; field-isolation matrix on the column. | +| Lance 6.0.1 bump (D-MBX-11) breaks a transitive dep | LOW | Patch bump only; can be reverted; CI gates (when cargo prohibition lifts). | +| AriGraph chain back-pointer link integrity | HIGH | D-MBX-A5 chain-traversal tests; SPO-G quad → chain node round-trip; tombstone outlives mailbox invariant. | +| Witness arc rotation loses old-witness facts | MED | Rotation only after column 4 Commit (or Prune); committed witnesses persist via the SPO-G pointer (R4) before rotation. | +| Staunen × Wisdom spread amplifies noise | MED | Phase-gated (Planning only); plasticity counter saturates u8; in v1 column-local (no baton-routed amplification across mailboxes). | + +--- + +## 13. Success criteria / acceptance tests + +> Cargo currently prohibited (session-stability constraint). Tests below are the per-deliverable acceptance bar; **run when cargo prohibition lifts**. + +### Per-phase acceptance + +- **P1 (prereqs):** par-tile crate exists; ndarray SIMD primitives complete; `lance =6.0.1` workspace-wide. +- **P2 (version gate):** `MailboxSoAHeader` carries version; round-trip read/write green; refusal-on-mismatch path tested. +- **P3 (shader columns):** all new columns present; field-isolation matrix green; existing tests still pass. +- **P4 (singleton dissolution):** `BindSpace::zeros(4096)` removed; `serve.rs` builds a sea-star of `MailboxSoA`s; integration test passes a full think-cycle through the new path. +- **P5 (kanban/planner):** SigmaTierRouter emits −550 ms stamp; `KanbanMove` flows through planner; SurrealDB view queryable. +- **P6 (cold-path):** `lance-graph` containers byte-equal `MailboxSoA`; SIMD benchmark shows ≥ 1.4× on representative workloads. +- **P7 (consumer alignment):** all nine D-MBX-12 sub-PRs merged; workspace `cargo check` clean; no Arrow re-encode boundary for the thoughtspace. + +### Workspace-level acceptance + +- **`cargo check`** clean across workspace. +- **`cargo test`** green for all lance-graph crates. +- **No `Vsa16kF32` carrier** in any cross-boundary type signature (`grep` audit). +- **No re-encode** between mailbox SoA and lance-graph storage row (verified by `D-MBX-12.4` golden tests). +- **SurrealDB transparent view** materializes the kanban over LanceDB rows without an Arrow round-trip (when surreal_container unblocks). + +--- + +## 14. Dependencies graph (textual) + +``` +PR-NDARRAY-MIRI-COMPLETE ───────────┐ + ▼ +D-CE64-MB-1-impl (par-tile) ───┬──► D-MBX-A2 ──► D-MBX-A3 ──► D-MBX-A4 ──► D-MBX-A5 + │ ▼ + │ D-MBX-2 ──► D-MBX-3 ──► D-MBX-5* + │ ▼ + └─► D-MBX-10 (foundation) ─────────────────► D-MBX-A6 ──► D-MBX-12.5 + ▲ +D-MBX-11 (Lance 6.0.1) ────────────────────────────────────────────────────────┘ + ▼ + D-MBX-7 ──► D-MBX-12.4 ──► D-MBX-6 ──► D-MBX-9 (needs surreal_container) + ▼ ▲ + D-MBX-4 ───────────────────────┘ + ▼ + D-MBX-8 ──► D-MBX-9 + +(*) D-MBX-5 also gated on OQ-11.4 (CLAUDE.md doctrinal update). +``` + +--- + +## 15. Cross-references + +- **Plans:** + - `bindspace-singleton-to-mailbox-soa-v1.md` §11.1–§11.6 — the layered rulings ratified in this session. + - `causaledge64-mailbox-rename-soa-v1.md` — canonical 5-crate + 7-PR plan; this doc sequences post-impl convergence. + - `cognitive-substrate-convergence-v1/v2/v3.md` — i4 mantissa, gapless baton, Σ10 Rubicon (shipped + in flight). + - `rung-persona-orchestration-v1.md` — D-PERSONA-5 ractor outer-swarm runtime (Queued). + - `.claude/surreal/RECONCILIATION_with_canonical_plan.md` — surreal POC folded in. + - `normalized-entity-holy-grail-v1` — `NormalizedEntity` typestate pattern (E-NORMALIZED-ENTITY-1; informs D-MBX-A6 DTO-as-lens). + +- **Epiphanies:** + - `E-SOA-IS-THE-ONLY` (2026-05-29, this session) — the five rulings. + - `E-MAILBOX-IS-BINDSPACE` (2026-05-27) — singleton dissolution. + - `E-RUBICON-RACTOR` (2026-05-27) — Heckhausen + Libet grounding of Σ10 Rubicon. + - `E-BATON-1` (2026-05-26) — LE Baton contract. + - `E-CE64-MB-4` — mailbox-as-owner ⇒ compile-time UB impossibility. + - `E-LADDER-SERVES-MAILBOX` — escalation ladder serves the mailbox; AriGraph hot→cold→tombstone. + - `E-NORMALIZED-ENTITY-1` (2026-05-28) — typestate carrier informing planner DTO overhaul. + - `linguistic-epiphanies-2026-04-19.md E21` — canonical Σ10 Rubicon tier doctrine. + +- **Iron rules:** + - `I-VSA-IDENTITIES` (bundle identities, not content). + - `I-LEGACY-API-FEATURE-GATED` (governs the SoA version gate). + - `I-SUBSTRATE-MARKOV` (Chapman-Kolmogorov; untouched at the local-bundle compute level). + - `I-NOISE-FLOOR-JIRAK` (weak-dependence Berry-Esseen; relevant for any threshold derivation). + +- **Code anchors:** + - `crates/cognitive-shader-driver/src/mailbox_soa.rs` (D-MBX-A1 columns shipped; A2/A3/A4/A5 add to this file). + - `crates/cognitive-shader-driver/src/bindspace.rs` (singleton to dissolve). + - `crates/cognitive-shader-driver/src/driver.rs` (the `Arc` holder at :56). + - `crates/cognitive-shader-driver/src/bin/serve.rs:29` (the `BindSpace::zeros(4096)` to remove). + - `crates/cognitive-shader-driver/src/engine_bridge.rs` (re-encode seam to collapse). + - `crates/lance-graph-contract/src/cognitive_shader.rs:382` (`ShaderCrystal.persisted_row`). + - `crates/lance-graph-ontology/src/registry.rs:39` (`LazyLock` — AS IS). + - `crates/lance-graph-ontology/src/lance_cache.rs` (the ontology cache — AS IS). + - `crates/surreal_container/src/lib.rs` (the SurrealDB view layer — BLOCKED). + - `crates/p64-bridge/src/lib.rs` (the conformance template — already LE). + +- **PRs (recent context):** + - PR #388 — `SigmaTierRouter` Rubicon-resonance dispatch (shipped). + - PR #414 — Odoo families 0x64 / 0x90 + axioms + StyleCluster wiring (shipped). + - PR #416 — recipes + atoms + savants + FIBU re-parent (shipped). + - PR #417 — `E-CONTRACT-NO-SERIALIZE-2` correction (shipped). + - PR #418 — bindspace-singleton-to-mailbox-soa migration spec (shipped; this plan extends). + - PR #433 — `style_recipe` D-Atom + epiphany-brainstorm-council + 5 savant cards (shipped). + - PR (this one) — unified-soa-convergence-v1 + handover. + +--- + +## 16. Council bypass note + +The `epiphany-brainstorm-council` pre-merge gate for `EPIPHANIES.md` additions (shipped in PR #433) is **bypassed** for this plan's epiphany pointers because the underlying rulings (`E-SOA-IS-THE-ONLY` plus refinements) are **author-stated** by the user, not derived. The council is for derived epiphanies where a wrong one would pollute downstream priors; user-stated rulings have higher authority and a different review surface (the user themselves). + +This plan, however, IS open to the council for its *spec content* — the convergence sequencing, the D-MBX deliverable breakdown, the OQ catalogue, and the risk matrix. PR review is the appropriate channel. + +---