feat: end-to-end alignment — docs↔code drift gating, math over keyword heuristics, persistent memory

[CodeWithJuber]

What & why

Three systemic problems, fixed at the root and gated so they can't recur:

1. Docs and code were misaligned, and nothing noticed. A full audit found 56 real drift issues: the entire forge config command family undocumented, 18 env vars read but documented nowhere, a phantom env var in ROADMAP (LITELLM_GATEWAY_URL — never existed in code), MCP tool counts disagreeing across four docs (5 vs 15 vs 3 vs actual 19), and two completely empty CHANGELOG release sections. This PR fixes all of them and adds forge docs check (new CI job + doctor check): commands, env vars, MCP tool names/counts, and CHANGELOG are now machine-reconciled against the code — the sources of truth are the commands.js table, actual process.env reads (src + shell guards), and the mcp_tools.js registry. bump.mjs now refuses to rotate an empty [Unreleased].

2. Hardcoded regex made semantic decisions. Every keyword-list decision is replaced with math (data tables like exemplars stay data):

• Routing: similarity-weighted k-NN over a labeled exemplar bank (overlap coefficient on stopword-filtered, plural-folded unigram+bigram sets, credibility-shrunk toward a prior) replaces the four topic keyword regexes and their additive magic weights. Unseen phrasings route by resemblance; every score returns its neighbors. Tune by adding labeled rows, not weights.
• Secret detection: src/secrets.js is the single source of truth — format grammars + a Shannon-entropy gate (new src/math.js) for unknown-vendor tokens. The secret-redact guard imports the same module, ending the JS/shell regex divergence.
• Lessons: graded token-overlap keyword matching (same-module partial credit) instead of all-or-nothing string equality.
• Goal drift: goalDrift returns a graded driftScore — the magnitude the existing CUSUM detector was designed to accumulate but never received.
• Substrate minimality warnings derive from computed preflight/route signals instead of a second keyword copy; skillgate gains an entropy detector for obfuscated base64 payloads; provider status probes /health for behavioral gateway evidence.

3. Changes didn't propagate end-to-end, and sessions forgot.

• The atlas now parses markdown into doc nodes with references edges — forge impact src/foo.js lists the docs that go stale, and the pre-edit hook says so before the edit.
• forge anchor set/show/clear persists the active goal in .forge/goal.md; SessionStart re-injects it every session.
• The Stop hook auto-repairs a drifted managed AGENTS.md (FORGE_AUTOSYNC=0 to disable) — drift used to be detected by doctor and repaired by nobody.

Adversarial review before push: an 8-angle review (line-by-line, removed-behavior, cross-file, reuse/simplification/efficiency, altitude, conventions) surfaced 10 verified defects in the new code — most critically, entropy false positives on file paths/camelCase identifiers that would have corrupted tool output, and a lost routing floor on short concurrency phrasings. All fixed with regression tests in the final commit.

Checklist

• npm test passes (Node 18/20/22) — 543 tests (was 500), 0 failures
• npm run check passes (Biome lint + format)
• New public functions have a test
• Conventional commit message (feat:/fix:/docs: …)
• CHANGELOG.md updated under ## [Unreleased] (and the empty 0.8.0/0.8.1 sections backfilled)
• No new runtime dependency (dev deps ok)
• Substrate/docs updated if this changes forge substrate, forge impact, router/gate, or MCP substrate tools — GUIDE routing/config/anchor/MCP/env sections rewritten; forge docs check passes clean on this tree (dogfooded)

Risk & rollback

• Risk level: medium — routing scores and secret detection change behavior; both are advisory-by-default layers, all existing behavior contracts are pinned by tests, and the LLM band rails/floors are re-verified against review counterexamples.
• Rollback plan: revert the branch commits; each wave is an independent commit (math → secrets → routing → wiring → docs machinery → memory → docs → hardening) so a single subsystem can be reverted alone.

Extra checks (tick if applicable)

• npm run typecheck passes
• Input validated at boundaries; errors handled (no swallowing — fail-safe paths log under FORGE_DEBUG=1)
• Logs contain no secrets/PII (secret detection itself is under test both directions)
• If AI-assisted: I understand it, verified the package APIs, and it has tests

---

Generated by Claude Code