Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,11 +231,15 @@ parses) swept against every doc artifact → UPDATED / STALE (file:line hits) /
VERIFIED-UNAFFECTED with the reason recorded. Pure reporter; the gate provides the teeth.

**Docs-check now guards more than names (`src/docs_check.js`).** Beyond
commands/env/MCP-tools/CHANGELOG, three reconcilers close the blind spots behind recurring
commands/env/MCP-tools/CHANGELOG, five reconcilers close the blind spots behind recurring
"docs rot" complaints: `checkDiagrams` scans every `mermaid` block across all Markdown for
the branded `%%{init` theme and literal-`\n` node breaks; `checkModelTiers` reconciles doc
prose prices against `src/model_tiers.json`; `checkBenchmarks` reconciles bolded `N ms`
README claims against the measured table in `reports/benchmarks.md`. The two public pages
README claims against the measured table in `reports/benchmarks.md`; `checkLinks` resolves
every intra-repo Markdown anchor (`#x` and `path.md#x`) against the target's real headings
(GitHub-exact slugs — em-dashes yield `--`, never collapsed), killing the dead-anchor class;
and `checkRoadmap` fails when the ROADMAP's "Now" marker trails the shipped `package.json`
version. The two public pages
(`landing/index.html` + the `build-pages.mjs` status page) share one set of design tokens,
enforced for parity (plus non-empty changes list, no phantom webfont) by
`test/pages.test.js` — so neither the docs' numbers nor the site's look can silently drift.
Expand All @@ -252,6 +256,21 @@ confidence gate, NOT a keyword DFA. Note `intentGrams` ≠ `contentGrams`: route
generic task verbs (`fix`/`add`/`build`) as complexity noise, but they are exactly the
intent signal — same math, different stop-set data.

**Graded goal-drift & completeness (`src/anchor.js`, `src/preflight.js`).** Two decisions that
were the last hand-static holdouts are now formulas. Goal-drift no longer classifies a changed
file by a binary path-substring match; `onGoalScore` is a **noisy-OR** (`1 − (1 − p)^hits`, the
same estimator `lessons.js` uses) over how many distinct goal concepts the file exhibits in its
path **and** its atlas-defined identifiers, thresholded at the single-hit floor — so a file that
implements the goal without naming it in its path is still classed on-goal. `driftScore` stays the
off-goal fraction (the `cusum` operating point is unchanged; an on-goal checkpoint scores 0 and
drains the chart); the grading sharpens _which_ files count as drift, not the detector's tuning. The M2
completeness score `s(x)` is a **logistic** over its features (concreteness, named specifics,
vagueness, a smooth `tanh` length term) instead of an additive rubric with magic coefficients and
discontinuous word-count steps — the `sigmoid` bounds it to (0,1) with no clamp, every feature's
pull stays attributable, and a labeled bank could refine the weights via `predictor.js`'s
`trainLogistic`. The calibrated prior still lands the paper's own examples where they were
(a bare "make the auth better" ≈ 0.23 → ask; a concrete verifyToken edit ≈ 0.63 → proceed).

**The evidence trail (preflight).** Once a goal is anchored, every prompt appends its
graded `driftScore` to the session log; `cusum` (until now test-only math) accumulates
the series and a sustained alarm rides the gate's block reason. Proceeding under
Expand Down
36 changes: 36 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,42 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Changed

- **Goal-drift classification is graded and identifier-aware (`src/anchor.js`).** A changed
file's on-goal/off-goal call is now a **noisy-OR** (`1 − (1 − p)^hits`) over how many
distinct goal concepts it exhibits in its path **and** the identifiers it defines (via the
atlas), thresholded at the single-hit floor — replacing the binary path-substring match, so
a file that implements the goal but never names it in its path is caught deterministically,
not just by the opt-in LLM pass. `driftScore` stays the off-goal fraction, so the CUSUM
detector's operating point is unchanged (an on-goal checkpoint scores 0 and drains the chart);
the sharper classification is what improves the signal.
- **Specification completeness is a logistic estimator (`src/preflight.js`).** The M2
assumption gate's `s(x)` completeness score is now a logistic over its features
(concreteness, named specifics, vagueness, a smooth `tanh` length term) — replacing the
additive scorer's magic coefficients and discontinuous word-count steps. The `sigmoid`
bounds it to (0,1) with no clamp, each feature's pull stays attributable, and a labeled
bank could refine the weights via `predictor.js`'s `trainLogistic`. Calibrated to keep the
documented examples (a bare "make the auth better" ≈ 0.23 → ask; a concrete verifyToken
edit ≈ 0.63 → proceed).

### Added

- **`forge docs check` now guards intra-repo links and roadmap freshness** — two more
reconcilers close recurring "docs rot" classes: `checkLinks` resolves every Markdown
anchor (`#x` and `path.md#x`) against the target file's real headings using
GitHub-exact slugs (an em-dash yields `--`, never collapsed), catching dead anchors like
a renamed `#install`; `checkRoadmap` fails when ROADMAP's "Now" marker trails the shipped
`package.json` version.

### Fixed

- **Dead and fabricated docs** — a fabricated `forge route` example in `docs/GUIDE.md`
(an impossible `Fable 5 / Opus` / "premium tier" output) now shows the real routed
verdict; broken `#install` anchors in `ONBOARDING.md` and the substrate README now point
to `#60-second-quickstart`; a dead `#use-it-in-a-script` self-link resolves; and ROADMAP's
"Now" marker is current (v0.11.0). All now enforced by the new docs-check guards.

## [0.11.0] - 2026-07-11

### Added
Expand Down
6 changes: 3 additions & 3 deletions ONBOARDING.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Onboarding — five minutes to productive

**One brain for every AI coding agent.** A language model is *stateless* — one
**One brain for every AI coding agent.** A language model is _stateless_ — one
context window, wiped every call — so it has no memory of what your team learned, no
foresight about what an edit breaks, and no enforced guardrails. forgekit is the
**cognitive substrate** that supplies exactly those three things, and it delivers them
Expand Down Expand Up @@ -39,7 +39,7 @@ forge doctor # everything green?
```

Full matrix (no-registry `github:` install, symlink dev setup) →
[README → Install](README.md#install).
[README → 60-second quickstart](README.md#60-second-quickstart).

## 2. Configure a repo (once per repo)

Expand All @@ -64,7 +64,7 @@ forge sync # recompiles into every tool; idempotent (only rewrit

## 3. Use the cognitive substrate

The substrate is the layer that runs *before* the model edits code. One command runs
The substrate is the layer that runs _before_ the model edits code. One command runs
the whole pre-action gate:

```bash
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,8 @@ an assumption until measured_.
real models (paper §9; that's the paper's measurement, not this repo's — `forge cost
--stages` reports only _your_ measured stages).
- **Conflict-free team memory** — merging two 500-claim ledger replicas takes **158 ms**; the
merge is a property-tested join-semilattice, so teammate ledgers converge in any order over
plain git.
merge is order-independent and property-tested, so teammate ledgers converge to the same state
no matter who syncs first, over plain git.

## 60-second quickstart

Expand Down
25 changes: 17 additions & 8 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,20 @@ every tool. This is where that brain is headed.
Direction, not promises — shaped by the two field reports this project is grounded in
(the SDLC pain-point map and the ecosystem landscape). Open a Discussion to weigh in.

## Now (`master`, v0.8.1+)
Everything from 0.8.0 plus gateway-environment support end to end — `ANTHROPIC_AUTH_TOKEN`
recognized everywhere, `ANTHROPIC_MODEL`/`FORGE_MODEL` model override, LiteLLM-gateway
auto-classification of `ANTHROPIC_BASE_URL`, direct-HTTP LLM calls when the `claude` CLI
is absent (`src/llm.js`) — plus decision math replacing keyword heuristics (exemplar k-NN
routing, entropy secret detection), docs↔code drift gating (`forge docs check`, in CI),
docs in the impact graph, and a persistent goal (`forge anchor set`).
## Now (`master`, v0.11.0)

The substrate is fully graded — decision math replaces every keyword heuristic: exemplar k-NN
routing, entropy secret detection, noisy-OR goal-drift over paths **and** the identifiers a file
defines, and a logistic specification-completeness gate. Around it: docs↔code drift gating
(`forge docs check`, in CI) that now also reconciles diagrams, model prices, benchmark numbers,
intra-repo links, and roadmap freshness; a completion gate (Stop hook); and auto-release on merge.
Gateway environments are supported end to end — `ANTHROPIC_AUTH_TOKEN` recognized everywhere,
`ANTHROPIC_MODEL`/`FORGE_MODEL` model override, LiteLLM-gateway auto-classification of
`ANTHROPIC_BASE_URL`, and direct-HTTP LLM calls when the `claude` CLI is absent (`src/llm.js`).
See [CHANGELOG.md](./CHANGELOG.md).

## Shipped — Substrate v2 (all phases P0–P8, v0.5.0)

The plan lives in [docs/plans/substrate-v2/](./docs/plans/substrate-v2/00-overview.md)
(phase dependency graph + acceptance gates, all marked done): every paper faculty and
mechanism mapped to an algorithm, unified by the **Proof-Carrying Memory (PCM)
Expand All @@ -25,6 +29,7 @@ confidence only from independent oracles, and merges across teammates conflict-f
(git-native CRDT ledger).

## Shipped — 0.7.0

- **Zero-config provider auto-detection** — `autoDetectProvider()` probes env vars
for LiteLLM (local + hosted gateway), OpenRouter, and Anthropic (key, auth token,
or custom base URL); `forge init` reports what it found, no manual config needed.
Expand All @@ -39,6 +44,7 @@ confidence only from independent oracles, and merges across teammates conflict-f
event timeline, and ledger health from `.forge/` data.

## Shipped — 0.6.0

- **Embeddings tier** — optional vector backend (`src/embed.js`, ADR-0005 dependency
tier, stdlib fallback kept): `FORGE_EMBED=cmd:<command>` or
`FORGE_EMBED=http:<url>` (OpenAI-compatible), disk-cached at
Expand All @@ -53,6 +59,7 @@ confidence only from independent oracles, and merges across teammates conflict-f
`/status/`.

## Next

- **OpenAI + Gemini provider detection** — extend `autoDetectProvider()` beyond
Anthropic/OpenRouter/LiteLLM (`OPENAI_API_KEY`, `GEMINI_API_KEY`) with the same
zero-config contract.
Expand All @@ -70,12 +77,14 @@ confidence only from independent oracles, and merges across teammates conflict-f
once fixtures measure them (overview §4 honesty register).

## Later / exploring

- `forge verify` independent-review wired into CI with provenance gating.
- Formal/semantic verification — documented as out-of-scope for now.
- Parametric learning channel (LoRA distillation) — deliberately out of scope (ADR-0006).

## Non-goals

Unbounded dependency growth — deps are selective, optional, and always backed by a
stdlib fallback path (ADR-0005). Reimplementing mature tools (skills installer,
subagent orchestration, SDD framework, SAST) — we wire the best existing ones. Bundling
a whole IDE. A hosted memory server — git *is* the sync (see 02-team-memory.md).
a whole IDE. A hosted memory server — git _is_ the sync (see 02-team-memory.md).
16 changes: 10 additions & 6 deletions docs/GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,9 @@ $ forge route "write an is_prime function"
driven by: similar to "check if a number is prime" (sim 1.00, complexity 0.08)

$ forge route "design and implement a distributed rate limiter with sliding windows across 3 services"
→ Fable 5 / Opus (premium tier)
driven by: similar to "implement a rate limiter with a token bucket" (sim 0.71, complexity 0.78)
→ Opus 4.8 (complex, $5/$25 per M tok)
architecture, cross-module refactor, novel algorithms, multi-layer debugging
complexity 0.73 · driven by: similar to "implement a rate limiter with a token bucket" (sim 0.43, complexity 0.78)
```

Unseen phrasings route by resemblance — "two threads deadlock when the queue is full"
Expand Down Expand Up @@ -274,10 +275,13 @@ Forge anchor — goal-drift check
```

`src/auth.js` maps to the goal (named file + where `verifyToken` lives); `src/report.js`
doesn't — so it's surfaced as drift to confirm or undo. Coarse and advisory by design
(path/keyword match, not semantic). `forge substrate` folds this in automatically. The
result also carries `driftScore` (the off-goal fraction per checkpoint) — the graded
signal the `cusum` change-point detector accumulates to catch _sustained_ small drift.
doesn't — so it's surfaced as drift to confirm or undo. Advisory by design. The on-goal/off-goal
call is graded and identifier-aware: each file's on-goal confidence is a noisy-OR over how many
goal concepts it exhibits in its path **and** the identifiers it defines (via the atlas), so a
file that implements the goal but never spells it in its path is still classed on-goal. `forge
substrate` folds this in automatically. The result carries `driftScore` — the fraction of the
checkpoint's changes classed off-goal — the signal the `cusum` change-point detector accumulates
to catch _sustained_ small drift (an on-goal checkpoint scores 0 and drains the chart).

**The goal persists.** `forge anchor set "<goal>"` stores it in `.forge/goal.md`; every
new session re-injects it at SessionStart, a bare `forge anchor` checks against it, and
Expand Down
4 changes: 2 additions & 2 deletions docs/cognitive-substrate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ In Claude Code it runs **automatically**. In other tools you (or the agent) run

## Install

Install Forge (plugin · npm · `github:`) per [README → Install](../../README.md#install),
Install Forge (plugin · npm · `github:`) per [README → 60-second quickstart](../../README.md#60-second-quickstart),
then inside any project:

```bash
Expand Down Expand Up @@ -159,7 +159,7 @@ Code, and agent-invoked everywhere else.

---

## Use it in a script & extend it
## Use it in a script

Add `--json` to any command for machine-readable output — gate your agent's next step on
`okToProceed`, feed `route.tier` to your model picker, read `impact.impactedFiles` before
Expand Down
2 changes: 1 addition & 1 deletion docs/cognitive-substrate/deliverable-package.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

### Theory → Evidence → Build-Map edition (v2)

**One-line thesis:** The faculties a coding agent lacks — memory, learning, imagination, self-correction, impact-awareness — are not gaps in the model's _knowledge_ but structural consequences of what a frozen transformer _is_ (a stateless map `y = f_θ(x)`, fixed weights, bounded window). They cannot be prompted or tooled away; they can only be supplied by **re-wrapping the input→process→output loop** into a closed, stateful cycle around the frozen model.
**One-line thesis:** The faculties a coding agent lacks — memory, learning, imagination, self-correction, impact-awareness — are not gaps in the model's _knowledge_ but structural consequences of what a frozen transformer _is_: a system with fixed weights and a bounded context window that keeps no state between calls (formally, a stateless map `y = f_θ(x)`). They cannot be prompted or tooled away; they can only be supplied by **re-wrapping the input→process→output loop** into a closed, stateful cycle around the frozen model.

**What v2 adds.** The first edition argued the five faculties from first principles and prototyped the one that is buildable today. This edition (1) **grounds the argument in the field's own evidence** — twelve load-bearing pain-point statistics independently re-grounded from primary sources and graded _confirmed / vendor-reported / unverifiable_; (2) adds **six metacognitive mechanisms** the frozen loop also lacks (routing, assumption gate, decomposition, goal-anchoring, anti-over-engineering, inline verification); (3) **maps all eleven capabilities against the real 2026 Claude-Code stack**, marking each solved / partial / residual-gap so we say clearly _what not to build_; and (4) ships a **second runnable prototype** — a complexity-aware router + assumption gate, evaluated live on real models.

Expand Down
Loading
Loading