CodeWithJuber · CodeWithJuber · Jul 7, 2026 · Jul 7, 2026
@@ -8,6 +8,17 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ### Documentation
 
+- **Substrate v2 plan: the whitepaper, completed (`docs/plans/substrate-v2/`).** Nine specs
+  + two ADRs mapping every remaining paper faculty/mechanism to a concrete algorithm, unified
+  by the **Proof-Carrying Memory (PCM) protocol**: every stored unit (lesson, fact, cached
+  artifact, graph edge, design fingerprint, diagnosis) becomes a content-addressed claim whose
+  confidence is a decayed Beta posterior over independent-oracle outcomes — retrieval implements
+  the paper's Eq. 3, team memory is a conflict-free CRDT ledger merged over git, code reuse is a
+  proof-carrying artifact cache, context assembly is a token-budget knapsack with a set-cover
+  completeness gate, and generated-UI quality is a measurable slop-distance/conformance gate.
+  ADR-0005 relaxes the zero-dependency rule to selective optional deps with stdlib fallbacks;
+  ADR-0006 converges all persistence on the PCM ledger. `ROADMAP.md` now carries the P1–P8
+  phase plan. Docs only — no runtime behavior changes.
 - **Visual flow diagrams in the entry-point docs.** A "one source → every tool + pre-action gate"
   mermaid in `README.md` and a "your day with Forge" loop in `ONBOARDING.md` (alongside the
   propose→verify diagram in the substrate README) — making the model easier to grasp at a glance,

@@ -13,19 +13,42 @@ governor (`forge cost`), spec-as-contract drift (`forge spec`), goal-drift check
 [docs/cognitive-substrate/](./docs/cognitive-substrate/). See
 [CHANGELOG.md](./CHANGELOG.md) `[Unreleased]`.
 
-## Next
-- **Substrate calibration fixtures** — expand assumption/routing/impact fixtures beyond
-  the original paper prototypes while keeping research-edge claims advisory.
-- **AST-backed impact** — an optional exact call graph behind the regex-approximate atlas
-  (weighed against the zero-runtime-dependency constraint).
-- **Testing** — Playwright MCP/agents opt-in + a coverage Stop-hook gate.
-- **MCP hygiene** — enforce the ~6-server cap + a registry resolver in `forge doctor`.
+## Next — Substrate v2 (the whitepaper, completed)
+The full plan lives in [docs/plans/substrate-v2/](./docs/plans/substrate-v2/00-overview.md):
+every paper faculty and mechanism mapped to an algorithm, unified by the
+**Proof-Carrying Memory (PCM) protocol** (ADR-0006) — every stored thing is a claim that
+carries its evidence, earns confidence only from independent oracles, and merges across
+teammates conflict-free (git-native CRDT ledger).
+
+Phases (dependency-ordered; acceptance gates in the overview):
+- **P1 Ledger core** — claim store, Beta-posterior confidence, Eq. 3 retrieval;
+  cortex + recall migrate onto claim kinds.
+- **P2 Team memory** — `.forge/ledger/` union-merge over git; `forge ledger`.
+- **P3 Reuse cache** — proof-carrying artifact cache (`forge reuse`): verified code is
+  served with its evidence, revalidated against the atlas, demoted by outcomes.
+- **P4 Context assembly** — knapsack selection under a token budget + a set-cover
+  completeness gate that *computes* what's missing (`forge context`).
+- **P5 Loop closure** — outcome write-back band; doom-loop diagnosis + escalation;
+  imagination dry-run (impacted-test set cover + worktree sandbox); AST-tier atlas with
+  a mandatory impact gate; M3 partition planning, M4 CUSUM drift, M5 footprint metric,
+  M6 checkpoint cadence; outcome-calibrated routing.
+- **P6 UI quality gate** — design fingerprints, slop-distance + project-conformance
+  scoring, scale/palette conformance checks (`forge uicheck` v2, machine-readable taste).
+- **P7 `forge dash`** — local dashboard over the ledger, cost stages, cache, blast radius.
+- **P8 Evaluation** — measured per-stage cost report (target ~90 % composed; only
+  measured numbers get claimed — see [05-cost-model.md](./docs/plans/substrate-v2/05-cost-model.md)).
+
+Carried forward from the previous roadmap into these phases: calibration fixtures (P8),
+AST-backed impact (P5), Playwright testing (P6), MCP hygiene in `forge doctor` (P7).
 
 ## Later / exploring
-- Optional vector-memory backend (Hindsight / Mem0) wiring.
+- Optional embedding/vector backends for retrieval and cache near-match (ADR-0005 tier).
 - `forge verify` independent-review wired into CI with provenance gating.
 - Formal/semantic verification — documented as out-of-scope for now.
+- Parametric learning channel (LoRA distillation) — deliberately out of scope (ADR-0006).
 
 ## Non-goals
-Adding runtime dependencies. Reimplementing mature tools (skills installer, memory engine,
-SDD framework, SAST) — we wire the best existing ones. Bundling a whole IDE.
+Unbounded dependency growth — deps are selective, optional, and always backed by a
+stdlib fallback path (ADR-0005). Reimplementing mature tools (skills installer,
+subagent orchestration, SDD framework, SAST) — we wire the best existing ones. Bundling
+a whole IDE. A hosted memory server — git *is* the sync (see 02-team-memory.md).
@@ -0,0 +1,41 @@
+# ADR 0005: Allow selective runtime dependencies
+
+- Status: accepted (supersedes the blanket rule in ADR-0001)
+- Date: 2026-07-07
+
+## Context
+ADR-0001 committed the CLI to zero production dependencies, sized for a config emitter
+plus advisory gates. Substrate v2 (docs/plans/substrate-v2/) adds capabilities where the
+stdlib ceiling has real costs: AST-accurate impact analysis (TypeScript compiler API),
+higher-recall semantic matching for the reuse cache and retrieval (embedding backends),
+fast ledger indexing at monorepo scale (SQLite), and the UI visual loop (Playwright).
+Writing these from scratch would be reinventing mature tools — the exact waste
+ARCHITECTURE.md's reuse ledger forbids. Meanwhile the symlink install channel that
+motivated ADR-0001 still exists and must not break.
+
+## Decision
+Runtime dependencies are permitted under four conditions, each enforced in review:
+
+1. **Core stays stdlib.** Every substrate feature has a pure-JS/stdlib reference path
+   (MinHash instead of embeddings, JSON-file scan instead of SQLite, regex atlas tier
+   instead of AST). Dependencies buy *better*, never *at all*.
+2. **Graceful absence.** A missing optional dependency degrades to the stdlib path with
+   a `forge doctor` note — never a crash. The symlink/plugin channels ship the stdlib
+   paths and remain dependency-free.
+3. **Named tiers.** `dependencies` stays empty; enhancements live in
+   `optionalDependencies` or documented opt-in installs, each mapped to the feature it
+   unlocks.
+4. **Vetting.** Each new dependency passes `forge scan`-grade review (provenance,
+   install scripts, transitive count) and is recorded in the reuse ledger with its
+   rationale.
+
+## Consequences
+- (+) AST-tier atlas, embedding retrieval, SQLite indexes, and the Playwright visual
+  loop become buildable without forking the architecture.
+- (+) The zero-dep guarantee survives where it matters (plugin/symlink channels, `npx`
+  cold start) because it is now a per-path property, not a package-wide one.
+- (−) Two code paths per enhanced feature — the fallback must be tested as a first-class
+  path (CI runs the suite with optional deps absent *and* present).
+- (−) Supply-chain surface grows; bounded by conditions 3–4.
+- CI's "dependencies is empty" assertion is kept; a new assertion checks every
+  optionalDependency has a registered fallback test.
@@ -0,0 +1,39 @@
+# ADR 0006: Proof-Carrying Memory is the one store
+
+- Status: accepted
+- Date: 2026-07-07
+
+## Context
+The whitepaper's sharpest novelty is validity-anchored memory: stored knowledge scored by
+whether external oracles later confirmed it (Eq. 3's `val` term), never by the model's
+say-so. v0.4 approximates this in two parallel stores with different rules — cortex
+lessons (α/β evidence, decay) and recall facts (plain files, no validity) — and the
+planned subsystems (reuse cache, context summaries, design fingerprints, diagnoses,
+team sharing) would each need their own persistence, trust, and merge story. Divergent
+stores mean divergent trust semantics and no coherent team-sync path.
+
+## Decision
+All substrate persistence converges on one protocol: the **Proof-Carrying Memory (PCM)
+claim ledger** (docs/plans/substrate-v2/01-pcm-protocol.md). Every stored unit is a
+content-addressed claim carrying provenance and a grow-only evidence set; confidence is
+a decayed Beta posterior over independent-oracle outcomes; retrieval implements the
+paper's Eq. 3; team sharing is the ledger's CRDT set-union merge over git
+(02-team-memory.md). Cortex lessons and recall facts become claim kinds behind their
+existing CLIs; new subsystems (artifacts, edges, fingerprints, diagnoses, summaries,
+decisions) mint claims from day one. The parametric learning channel (LoRA distillation,
+paper §7.2) remains explicitly out of scope.
+
+## Consequences
+- (+) One trust model everywhere: anything the substrate believes can answer "why?"
+  with machine-readable evidence, and anything contradicted by outcomes demotes itself —
+  including cached code and context summaries, not just lessons.
+- (+) Team memory falls out of the storage choice (semilattice merge ⇒ conflict-free by
+  construction) instead of being a bolted-on sync feature.
+- (+) The write-back band (Eq. 2) has one place to write, so outcome-validated learning
+  covers every subsystem at once.
+- (−) A migration: lessons/recall move onto adapters; their tests must pass unchanged
+  (the P1 acceptance gate) and the old on-disk stores need a one-shot import.
+- (−) The ledger becomes load-bearing infrastructure — its canonicalization and merge
+  invariants require property-level tests, a higher bar than the stores it replaces.
+- (−) Repo history carries the ledger; bounded by canonical-JSON compactness, sharding,
+  and the attic-prune policy.
@@ -0,0 +1,87 @@
+# Substrate v2 — completing the whitepaper, and the Proof-Carrying Memory protocol
+
+> Status: **P0 — specification** · Owner: forgekit core · Companion to
+> [docs/cognitive-substrate/](../../cognitive-substrate/) (the paper this plan completes)
+
+ForgeKit v0.4 implements roughly half of the committed whitepaper — the two prototyped
+mechanisms (M1 routing, M2 assumption gate), an approximate impact atlas, and an advisory
+memory (cortex + recall). This plan closes the rest, and it closes the pains the paper's
+build-opportunity map (§10) ranks highest: validity-anchored memory, outcome-validated
+learning, team/shared memory, code reuse, context completeness, doom-loop diagnosis — plus
+two pains named by the project owner that the paper only gestures at: **generated-UI
+template convergence** and a **~90 % cost-reduction target**.
+
+One idea unifies all of it: the **Proof-Carrying Memory (PCM) protocol**
+([01-pcm-protocol.md](./01-pcm-protocol.md)). Every unit of knowledge the system holds —
+a lesson, a cached code artifact, a dependency edge, a design fingerprint, a diagnosis —
+becomes a *claim* that carries its own evidence, earns confidence only from independent
+oracles (tests, typecheck, CI, human accept/revert), decays without review, and merges
+across teammates without conflicts. This takes the paper's sharpest novelty — the `val`
+term of Eq. 3, "validity-anchored memory" (§11) — and turns it from one scoring term into
+the storage, trust, and wire protocol for the whole substrate.
+
+## 1. Gap analysis — paper vs. `src/`
+
+| Paper capability (§10 map) | ForgeKit v0.4 | Gap | Closed by |
+|---|---|---|---|
+| M1 routing + M2 assumption gate (opp. #1) | `src/route.js`, `src/preflight.js` — 62 % measured saving | ✅ shipped; residue: outcome-calibrated weights | [06](./06-faculties-and-mechanisms.md) §7 |
+| Impact oracle, mandatory gate (opp. #3) | `src/atlas.js` regex graph; gate opt-in (`FORGE_ENFORCE=1`) | precision; gate not mandatory | [06](./06-faculties-and-mechanisms.md) §1 |
+| Validity-anchored memory (opp. #2, Eq. 3) | cortex confidence exists (`src/lessons.js` already keeps α/β evidence) | no Eq. 3 retrieval, no forget/consolidate policy, flat store | [01](./01-pcm-protocol.md) |
+| Outcome-validated learning (opp. #4, Eq. 2) | outcomes update lessons, but not *the memories that informed an action* | write-back band incomplete | [01](./01-pcm-protocol.md) §6, [06](./06-faculties-and-mechanisms.md) §6 |
+| Team / shared memory | single-repo file recall (`src/recall.js`, `src/brain.js`) | no merge semantics, no attribution | [02](./02-team-memory.md) |
+| Code reuse / generation cache | `reuse-first` skill is prose only | no artifact cache | [03](./03-reuse-cache.md) |
+| Context compression / completeness | none — "incomplete context" is unaddressed | whole module missing | [04](./04-context-assembly.md) |
+| Doom-loop root cause (opp. #5) | `doom-loop.sh` detects; no diagnosis, no escalation | diagnosis + escalation | [06](./06-faculties-and-mechanisms.md) §5 |
+| Imagination (faculty, §3) | atlas traversal only — no dry-run of consequences | test selection + sandbox | [06](./06-faculties-and-mechanisms.md) §2 |
+| M3/M4/M5/M6 (decomposition, drift, lean, inline verify) | `scope.js`/`anchor.js`/`lean.js`/`verify.js` heuristics | each gets its algorithm | [06](./06-faculties-and-mechanisms.md) §3–§6 |
+| Generated-UI quality (owner pain; M5-shaped) | `src/uicheck.js` WCAG contrast only; taste is prose | anti-template gate | [07](./07-ui-quality-gate.md) |
+| Cost to ~90 % (owner target) | routing alone: 62 % measured | cache + context + gate stages unmeasured | [05](./05-cost-model.md) |
+| ForgeKit's own UX | CLI only | `forge dash` dashboard | [08](./08-dashboard-ux.md) |
+
+## 2. The 11-capability master table
+
+Every faculty (paper §3) and mechanism (paper §6) with the math, algorithm, or data
+structure this plan assigns it. Nothing is left as prose-only discipline.
+
+| # | Capability | Mechanism | Math / algorithm / data structure | Spec |
+|---|---|---|---|---|
+| 1 | Memory (faculty) | PCM claim ledger; Eq. 3 retrieval; ʿilm→fahm→ḥikma layers | Content-addressed (Merkle-keyed) claim store; Beta(α,β) posterior with exponential decay; MinHash sketches; consolidation = union-find clustering over Jaccard ≥ τ, promoting episodes → patterns → decision rules | [01](./01-pcm-protocol.md) |
+| 2 | Learning (faculty) | Outcome write-back band (Eq. 2): oracle results update the `val` of every claim that informed the action | Bayesian evidence update; rubric-weight calibration by logistic regression over outcome claims (the paper's own "learn the rubric weights" note, §9.3) | [01](./01-pcm-protocol.md) §6 |
+| 3 | Imagination (faculty) | Consequence simulator `g` (paper Eq. 4): blast radius → impacted-test selection → sandboxed dry-run | Reverse-dependency traversal with hop-decay; test selection as bipartite set cover (greedy ln n-approx); dry-run result becomes evidence on the prediction claim | [06](./06-faculties-and-mechanisms.md) §2 |
+| 4 | Self-correction (faculty) | External-oracle cascade, never self-prompting (paper §3 honest negative, C12) | Cost-ordered oracle chain (types → impacted tests → independent reviewer); verdict requires ≥ 2 signals external to fθ | [06](./06-faculties-and-mechanisms.md) §4 |
+| 5 | Impact-awareness (faculty) | Atlas hardened; pre-edit gate becomes **mandatory** (hook-enforced) | Incremental dep graph keyed by file content hash; reverse-edge index → O(deg) "who depends on X" | [06](./06-faculties-and-mechanisms.md) §1 |
+| 6 | M1 routing | Shipped; add auditable per-task rubric surface + outcome-calibrated weights | Additive transparent rubric; escalation only on verified failure; online weight calibration from ledger | [06](./06-faculties-and-mechanisms.md) §7 |
+| 7 | M2 assumption gate | Shipped; questions become **computed missing-sets**, not pattern matches | `s(x) < τ` halt rule + set difference `R(edit) \ covered(selection)` as the question generator | [04](./04-context-assembly.md) §4 |
+| 8 | M3 decomposition | Automatic partition-boundary detection (the paper's flagged residue, §5.3) | Constrained graph partition on the task-dependency graph: greedy modularity / min-cut, each part's working set ≤ window budget | [06](./06-faculties-and-mechanisms.md) §3 |
+| 9 | M4 goal-anchoring | Continuous drift control, not a static anchor (§5.4) | Drift `D(y_t,g) = 1 − sim(goal, rolling summary)`; **CUSUM control chart** triggers mandatory re-anchor | [06](./06-faculties-and-mechanisms.md) §5 |
+| 10 | M5 anti-over-engineering | `src/lean.js` footprint made a defined metric (§5.5) | `φ(y) − φ*(x)` over files/abstractions/LOC; MDL tie-break: smallest description that passes the oracle | [06](./06-faculties-and-mechanisms.md) §6 · [07](./07-ui-quality-gate.md) for UI |
+| 11 | M6 inline verification | Checkpoint scheduling during generation (§5.6) | Optimal-stopping threshold: check when hazard × tokens-at-risk > check cost → deterministic cadence per tier | [06](./06-faculties-and-mechanisms.md) §6 |
+
+## 3. Phase roadmap
+
+Phases are dependency-ordered; each has an acceptance gate. P1 is the keystone — every
+later phase stores its state as PCM claims.
+
+| Phase | Delivers | Depends on | Acceptance |
+|---|---|---|---|
+| **P0** (this PR) | Specs 00–08, ADR-0005, ADR-0006, ROADMAP update | — | Docs merged; referenced paths resolve; no source changes |
+| **P1 Ledger core** | `src/ledger.js` (claim store, canonical hashing, Beta confidence, Eq. 3 retrieval, decay/prune); migrate `src/lessons.js` + `src/lessons_store.js` + `src/recall.js` onto claim kinds | P0 | All existing cortex/recall tests green on the new store; property tests: id stability, decay monotonicity |
+| **P2 Team sync** | `.forge/ledger/` git layout, union-merge driver, `forge ledger merge\|verify\|blame` | P1 | Three-way merge fuzz: any interleaving of two ledgers converges byte-identically (semilattice test) |
+| **P3 Reuse cache** | `forge reuse` — fingerprint, exact/near lookup, atlas revalidation, eviction | P1 | Cache hit returns artifact + evidence; stale-interface artifact refused; hit/miss metrics emitted |
+| **P4 Context assembly** | `forge context` — candidate scoring, knapsack selection, required-set completeness gate; wired into `src/substrate.js` + hooks | P1 | Gate emits computed missing-set on incomplete context; token budget never exceeded |
+| **P5 Loop closure** | Outcome write-back band; doom-loop diagnosis + escalation; imagination dry-run; M3/M4/M5/M6 extensions | P1, P4 | Revert/test outcomes visibly move `val` of informing claims; repeated failure signature halts with a diagnosis claim |
+| **P6 UI quality gate** | `forge uicheck` v2: design fingerprints, slop distance, scale conformance; machine-readable taste constraints | P1 | Known-template fixture flagged; project-conformant fixture passes; zero LLM calls in the gate |
+| **P7 Dashboard** | `forge dash` — local server + self-contained HTML over `.forge/` stores | P1–P6 (reads their stores) | Renders ledger, cost meter, cache rate, blast radius offline |
+| **P8 Evaluation** | Extend `src/eval.js`: cost-stage measurement, cache-hit-rate harness, honest report | P3–P5 | A measured (not asserted) end-to-end cost figure per stage, published in reports/ |
+
+## 4. Honesty register (the paper's own discipline, applied to this plan)
+
+- **Measured:** 62 % routing saving (paper §9, live tokens); atlas recall/precision method
+  (paper §8). Everything else in [05-cost-model.md](./05-cost-model.md) is a **target** until
+  P8 measures it — the ~90 % figure is a composition argument, not a result.
+- **Solved-elsewhere, not rebuilt:** subagent orchestration (M3's mechanics), model
+  gateways (M1's plumbing) — per paper §10 "do not rebuild".
+- **Research-edge, shipped as advisory first:** consolidation quality (ʿilm→fahm promotion),
+  M4 drift similarity, M6 hazard estimates. They enter as advisory signals and only become
+  blocking gates once P8 gives them fixtures — the same guard-over-prose discipline as
+  ADR-0003.