Roadmap "Next" wave: ledger read-path flip, optional embeddings tier, Playwright visual gate

CHANGELOG.md

@@ -6,6 +6,48 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+### Added
+
+- **Optional embeddings tier** (`src/embed.js`, ADR-0005; ROADMAP "Next"): set
+  `FORGE_EMBED=cmd:<command>` (stdin/stdout JSON protocol — any local model or script)
+  or `FORGE_EMBED=http:<url>` (OpenAI-compatible, `$FORGE_EMBED_MODEL` /
+  `$FORGE_EMBED_KEY`, key never logged) and `forge reuse query` + `forge ledger query`
+  replace the MinHash `rel` term with embedding cosine (near/adapt ≥ 0.85/0.7 — a
+  higher bar than Jaccard's 0.8/0.6 to match dense cosine's noise floor), fixing the
+  documented weak spot on very short specs. Vectors are disk-cached
+  (`.forge/embed-cache.jsonl`, content-hash keyed, corrupt-tolerant, truncate-oldest);
+  both commands print the backend that served (`sim: minhash` / `sim: embed(cmd)`);
+  any provider failure degrades silently to MinHash. `dependencies` stays empty —
+  the tier is configuration, not a package; the pure ledger core never imports it.
+- **`forge uicheck visual <file-or-url>`** — the Playwright visual loop
+  (07-ui-quality-gate §5): renders the page headless at two viewports, fingerprints
+  the **computed** styles of every visible element (what the cascade and runtime
+  theming actually painted, with used `auto`-margins and never-painted UA noise
+  filtered out), and runs the identical `design` gate over that rendered vector —
+  screenshots land in `.forge/ui/`. Playwright stays an optional tier (ADR-0005):
+  `package.json` gains no dependency, absence degrades to a "skipped (no browser
+  runtime)" note with exit 0 (`npm i -D playwright-core` or `FORGE_PLAYWRIGHT=…` to
+  enable), and non-loopback http(s) targets are refused by default (`--remote` to
+  override) — a gate that fetches arbitrary URLs is an exfiltration hazard.
+
+### Changed
+
+- **Ledger read-path flip (P2).** Reads are now a merged view (legacy ∪ ledger) via the
+  new `src/ledger_read.js`, so teammate knowledge that arrives with `forge ledger merge`
+  actually reaches injection and retrieval: cortex lesson surfaces
+  (`lessonsForContext`, `startupBlock`, `summary`, the substrate advisory and routing
+  past-mistake density) map ledger `lesson` claims onto the legacy lesson shape with an
+  evidence-derived status (tombstoned → retired, val ≥ 0.6 → active, val < 0.45 with a
+  contradiction → quarantined, else candidate), and fact surfaces (`recall list`/
+  `MEMORY.md`, brain's `AGENTS.md` index) include live ledger `fact` claims — always
+  deduped by legacy id/slug with the local file winning, and best-effort (a missing or
+  corrupt ledger degrades to legacy-only). Write paths (`recordMistake`'s
+  confirm-vs-create lookup, `recordContradiction`, `applyDistillation`) deliberately
+  keep reading the legacy store they edit; convergence comes from content-addressed
+  claim ids. `reconcileFacts` now only tombstones locally-authored claims, so a merged
+  teammate fact survives `forge recall consolidate`. Legacy formats are still written —
+  full retirement is the next step.
+
 ## [0.6.0] - 2026-07-07
 
 ### Changed

README.md

@@ -166,8 +166,9 @@ Forge states its own ceiling everywhere. In short: **guards reduce, don't elimin
 the "ignored my rules" problem; `recall`/`cortex` are file memory, **not** weight-level
 learning; the `atlas`/`impact` graph is regex-approximate (conservative, not a sound
 call graph — the impact numbers above are n = 6 hand-labeled cases on one JavaScript
-repo); the substrate's rubrics are heuristic; `forge reuse`'s near-match is weak on very
-short specs; and `forge cost --stages` reports **measured stages only** — a stage with
+repo); the substrate's rubrics are heuristic; the MinHash near-match is weak on very short
+specs (an optional embeddings backend — `FORGE_EMBED` — lifts this; MinHash stays the
+zero-dependency default); and `forge cost --stages` reports **measured stages only** — a stage with
 no events says "no data", never a default. What's *asserted* is safe to gate on (repo
 grounding, graph traversal, routing arithmetic, test commands); everything else is
 *advisory*. **Tests and human corrections always win.** Full list:

ROADMAP.md

@@ -26,14 +26,19 @@ confidence only from independent oracles, and merges across teammates conflict-f
 (git-native CRDT ledger).
 
 ## Next
-- **Ledger read-path flip** — the ledger is the convergent *write* store today while
-  the legacy stores (`lessons/`, `recall`, `brain`) still serve reads; flip reads to be
-  ledger-first, then retire the legacy formats.
+- **Legacy store retirement** — the read-path flip has shipped: every read surface
+  (cortex injection/status, the substrate advisory, routing, `recall list`, brain's
+  AGENTS.md index) is now a merged view (legacy ∪ ledger) via `src/ledger_read.js`,
+  so teammate knowledge from `forge ledger merge` reaches injection. The legacy
+  formats (`lessons/*.md`, recall/brain fact files) are still written as the canonical
+  local state; the remaining step is retiring them so the ledger is the only store.
 - **Embeddings tier** — optional vector backend (ADR-0005 dependency tier, stdlib
   fallback kept) for Eq. 3 retrieval and `forge reuse` near-match, where MinHash is
   weak on short specs.
-- **Playwright loop** — drive a real browser in the UI gate: screenshot + interaction
-  checks feeding `forge uicheck` verdicts back as oracle evidence on design claims.
+- **Playwright loop** — shipped as `forge uicheck visual` (rendered computed-style
+  fingerprint through the design gate + 2-viewport screenshots, optional-tier
+  playwright); still open: interaction checks and feeding verdicts back as oracle
+  evidence on design claims.
 - **Advisory → gated promotions** — outcome-calibrated routing weights, consolidation
   promotion (ʿilm→fahm), M6 hazard estimates: advisory today, become blocking only
   once fixtures measure them (overview §4 honesty register).

docs/GUIDE.md

@@ -330,10 +330,12 @@ holds** — confidence above the 0.6 floor and every declared dependency still i
 
 ```console
 $ forge reuse query "debounce user input before firing search"
+  sim: minhash
   NEAR hit (similarity 0.87) — module at src/lib/debounce.js
     claim 9c41d2ab77e0 — `forge ledger blame 9c41d2ab` for its proof
 
 $ forge reuse query "quantum blockchain"
+  sim: minhash
   miss — nothing verified matches; generate, then `forge reuse mint` it
 ```
 
@@ -348,7 +350,32 @@ $ forge reuse mint "debounce helper for search input" --file src/lib/debounce.js
 
 `forge reuse stats` shows lookups by outcome + estimated tokens saved (from
 `.forge/metrics.jsonl`). Honest limit: the MinHash near-match is weak on very short
-specs — a few words hash to too few shingles to rank reliably.
+specs — a few words hash to too few shingles to rank reliably. The optional embeddings
+tier below lifts exactly this.
+
+### `FORGE_EMBED` — the optional embeddings tier (ADR-0005)
+
+MinHash is the always-working, zero-dependency default. Set `FORGE_EMBED` and
+`forge reuse query` + `forge ledger query` swap the lexical similarity for embedding
+cosine — no new flags, the env var is the switch, and every query prints which backend
+served it (`sim: minhash` / `sim: embed(cmd)` / `sim: embed(http)`):
+
+- `FORGE_EMBED=cmd:<shell-command>` — the universal escape hatch (any local model, any
+  script): forge writes `{"texts":[...]}` to its stdin and reads
+  `{"vectors":[[...]]}` from its stdout.
+- `FORGE_EMBED=http:<url>` (or a bare `https://` URL) — OpenAI-compatible
+  `POST {input, model: $FORGE_EMBED_MODEL}` with
+  `Authorization: Bearer $FORGE_EMBED_KEY` (the key is passed via environment only —
+  never logged, never in argv).
+
+Thresholds move with the scale: near/adapt are cosine ≥ 0.85/0.7 instead of Jaccard's
+0.8/0.6, because dense cosines have a much higher noise floor (unrelated sentences
+commonly score 0.4–0.6) while unrelated shingle sets sit near 0. Vectors are cached in
+`.forge/embed-cache.jsonl` (content-hash keyed, corrupt-tolerant, size-capped
+truncate-oldest) so repeated queries never re-pay the provider. Any provider failure —
+crash, timeout, garbage output — silently degrades to the MinHash path
+(`FORGE_EMBED_TIMEOUT_MS` caps the wait, default 15000). Per ADR-0005:
+`dependencies` stays empty; this tier is configuration, not a package.
 
 ### `forge context "<task>"` — budgeted assembly + completeness gate
 
@@ -424,7 +451,8 @@ tree (your uncommitted changes wouldn't be in the run); commit/stash first or pa
 
 ### `forge uicheck` — deterministic UI checks
 
-Three subcommands, all static parsing — no LLM, no screenshots.
+Four subcommands: three are static parsing — no LLM, no screenshots — and `visual`
+optionally drives a real browser.
 
 **`contrast <fg> <bg>`** — exact WCAG math, asserted, never guessed (bare
 `forge uicheck <fg> <bg>` still works):
@@ -457,6 +485,32 @@ Failures are actionable per-feature edits, never a bare score. Honest limit: the
 fingerprint doesn't resolve CSS `var()` indirection yet — fully tokenized palettes are
 partially invisible to it.
 
+**`visual <file-or-url> [--taste <name>] [--json] [--remote]`** — the Playwright
+visual loop: renders the page headless at two viewports (1280×800, 390×844),
+fingerprints the **computed** styles of every visible element — what the cascade,
+`var()` resolution, and runtime theming actually produced — and runs the exact same
+design gate as `design` (exit 1 on fail). Screenshots land in `.forge/ui/` for human
+review. Playwright is an *optional tier* (ADR-0005): `package.json` stays
+dependency-free; without a browser runtime the command prints a "skipped (no browser
+runtime)" note and exits 0 — enable it with `npm i -D playwright-core` or point
+`FORGE_PLAYWRIGHT` at an existing install (e.g.
+`FORGE_PLAYWRIGHT=/path/to/node_modules/playwright-core`). Security default: http(s)
+targets must be loopback (`localhost`, `127.*`, `[::1]`) — fetching arbitrary URLs is
+an exfiltration hazard, so non-local URLs are refused unless you pass `--remote`.
+
+```console
+$ forge uicheck visual src/dash.html
+Forge uicheck visual — rendered fingerprint + design gate
+
+  rendered:      file:///…/src/dash.html (80 visible element style(s))
+  screenshots:   .forge/ui/dash-1280x800.png, .forge/ui/dash-390x844.png
+  slop distance: 0.516  (need ≥ 0.25 — farther from generic is better)
+  ...
+  ✓ spacing-scale: 100% of 6 spacing value(s) on the 4px base (ε 0.5px)
+
+  ✓ PASS
+```
+
 ### `forge dash [--port N]` — the local dashboard
 
 A read-only lens over `.forge/` — stdlib `node:http`, localhost-only, one
@@ -683,7 +737,8 @@ Edit `brand.json` (`FORGE_BRAND`), the `bin` key in `package.json`, and `name` i
 - **The atlas graph is regex-approximate** — conservative, not a sound call graph;
   dynamic dispatch and generated code can be missed.
 - **`forge reuse`'s MinHash near-match is weak on very short specs** — a few words hash
-  to too few shingles to rank reliably; write a sentence, not a keyword.
+  to too few shingles to rank reliably; write a sentence, not a keyword — or configure
+  the optional `FORGE_EMBED` embeddings tier, which replaces exactly this term.
 - **The UI fingerprint doesn't resolve CSS `var()` indirection yet** — a fully
   tokenized palette is partially invisible to the design gate.
 - **`forge cost --stages` reports measured stages only** — a stage with no events says

docs/plans/substrate-v2/07-ui-quality-gate.md

@@ -99,6 +99,10 @@ the other gates).
 
 ## 5. The visual loop (the only non-static part)
 
+> Status: shipped as `forge uicheck visual <file-or-url>` (`src/uivisual.js`) — rendered
+> computed-style fingerprint through the same gate, 2-viewport screenshots to
+> `.forge/ui/`, optional-tier playwright with graceful absence (ADR-0005).
+
 Static analysis can't see rendered composition. Under ADR-0005, the
 `frontend-verifier` crew agent gets Playwright:
 

src/brain.js

@@ -5,19 +5,25 @@
 // by construction: the inlined index is capped; overflow stays in fact files, never
 // silently truncated the way Claude's native 200-line MEMORY.md is (#39811).
 import { existsSync, mkdirSync, readdirSync, readFileSync, writeFileSync } from "node:fs";
-import { join } from "node:path";
-import { add as recallAdd, list as recallList } from "./recall.js";
+import { dirname, join } from "node:path";
+import { ledgerFacts, mergeFactSlugs } from "./ledger_read.js";
+import { listStored, add as recallAdd } from "./recall.js";
 
 export const brainStore = (targetRoot = process.cwd()) => join(targetRoot, ".forge", "brain");
 
+// Brain facts shadow into the REPO ledger (.forge/ledger — the sibling of this store,
+// see `forge remember`), so that is where merged teammate facts arrive.
+const brainLedger = (store) => join(dirname(store), "ledger");
+
 /** Store one fact (secret-refused by recall) and rebuild the inlined index. */
 export function remember(store, name, body) {
   const res = recallAdd(store, name, body);
   if (res.ok) buildIndex(store);
   return res;
 }
 
-export const list = recallList;
+/** Merged read (P2 read flip): file facts ∪ live repo-ledger facts, file wins on slug. */
+export const list = (store) => mergeFactSlugs(listStored(store), brainLedger(store));
 
 const gistOf = (text) =>
   text
@@ -26,25 +32,35 @@ const gistOf = (text) =>
     .map((l) => l.trim())
     .find(Boolean) || "";
 
-/** Build the capped, cliff-safe index inlined into AGENTS.md. Overflow → a pointer. */
+/** Build the capped, cliff-safe index inlined into AGENTS.md. Overflow → a pointer.
+ *  Merged view (P2 read flip): teammate facts that arrived in the repo ledger via
+ *  `forge ledger merge` join the index; a local file wins on name collision. */
 export function buildIndex(store, { capItems = 120 } = {}) {
   const factsDir = join(store, "facts");
   const facts = existsSync(factsDir)
     ? readdirSync(factsDir)
         .filter((f) => f.endsWith(".md"))
         .sort()
     : [];
+  const entries = facts.map((file) => ({
+    name: file.replace(/\.md$/, ""),
+    gist: gistOf(readFileSync(join(factsDir, file), "utf8")),
+  }));
+  const seen = new Set(entries.map((e) => e.name));
+  for (const f of ledgerFacts(brainLedger(store))) {
+    if (seen.has(f.slug)) continue;
+    seen.add(f.slug);
+    entries.push({ name: f.slug, gist: gistOf(f.text) });
+  }
+  entries.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0));
   const rows = [];
   let overflow = 0;
-  for (const file of facts) {
+  for (const e of entries) {
     if (rows.length >= capItems) {
       overflow += 1;
       continue;
     }
-    const name = file.replace(/\.md$/, "");
-    rows.push(
-      `- **${name}** — ${gistOf(readFileSync(join(factsDir, file), "utf8")).slice(0, 140)}`,
-    );
+    rows.push(`- **${e.name}** — ${e.gist.slice(0, 140)}`);
   }
   const indexed = rows.length; // count real facts before adding the overflow pointer
   if (overflow)