Skip to content

feat(anchor,preflight,docs): graded goal-drift + logistic completeness + self-enforcing doc guards#51

Merged
CodeWithJuber merged 2 commits into
masterfrom
claude/forge-work-system-setup-p26ka5
Jul 11, 2026
Merged

feat(anchor,preflight,docs): graded goal-drift + logistic completeness + self-enforcing doc guards#51
CodeWithJuber merged 2 commits into
masterfrom
claude/forge-work-system-setup-p26ka5

Conversation

@CodeWithJuber

Copy link
Copy Markdown
Owner

What & why

Auditing the whitepaper and every module against the project's own rule — "DATA may be a table, but DECISIONS must be a formula — graded, inspectable, testable" — found the codebase already overwhelmingly mathematical (route, intent, lessons, ledger, reuse, context, atlas-impact, predictor, diagnose are all properly graded). Exactly two static-decision holdouts remained, plus a set of concrete docs problems. This PR closes both, and adds guards so the doc problems can't silently recur.

Static decisions → graded formulas

  • src/anchor.js (M4 goal-drift): on-goal-ness was a binary path-substring match (path.includes(keyword)) that fed a graded CUSUM — a graded wrapper over an ungraded core. Now onGoalScore is a noisy-OR (1 − (1 − p)^hits, the same estimator lessons.js uses) over how many distinct goal concepts a file exhibits in its path and the identifiers it defines (via the atlas). So a file that implements the goal but never names it in its path is caught deterministically, not just by the opt-in LLM pass. driftScore becomes a continuous mean(1 − score) that strictly generalizes the old off-goal fraction (identical on binary scores, graded otherwise), sharpening the drift detector's input.
  • src/preflight.js (M2 completeness s(x)): the completeness score was a hand-tuned additive scorer with magic coefficients and discontinuous word-count step bonuses. Now it's a logistic over the same features (concreteness, named specifics, vagueness, a smooth tanh length term), reusing predictor.js's sigmoid. Smooth (no step jumps), self-bounding (no ad-hoc clamp), every feature attributable, and a labeled bank could refine the weights via trainLogistic. Calibrated so the paper's own examples land where documented (make the auth better ≈ 0.23 → ask; the concrete verifyToken edit ≈ 0.63 → proceed).

Docs problems fixed (found by a full-doc audit) + made self-enforcing

  • Fabricated forge route example in docs/GUIDE.md (an impossible Fable 5 / Opus / "premium tier" output) → the real routed verdict.
  • Dead #install anchors (ONBOARDING.md, substrate README — README has no Install heading) → #60-second-quickstart; a dead #use-it-in-a-script self-link resolved; stale ROADMAP marker → v0.11.0; a couple of front-page jargon/consistency touches.
  • New forge docs check guards: checkLinks resolves every intra-repo Markdown anchor (#x and path.md#x) against the target's real headings using GitHub-exact slugs (an em-dash yields --, never collapsed; code-fence link syntax is skipped); checkRoadmap fails when the ROADMAP "Now" marker trails package.json. These make the classes above CI failures, not recurring complaints.

The research whitepaper itself was audited and found rigorous and internally consistent (its reported 62.1% / 30-30 gate / 21-21 routing numbers match eval_results.json exactly); no corrections needed there.

Checklist

  • npm test passes (658 pass, 0 fail, 2 skipped)
  • npm run check passes (Biome lint + format)
  • New public functions have a test (onGoalScore, completenessScore/completenessFeatures, checkLinks, checkRoadmap; +34 tests)
  • Conventional commit message
  • CHANGELOG.md updated under ## [Unreleased]
  • No new runtime dependency (zero-dep; reuses existing math.js/predictor.js primitives)
  • Substrate/docs updated (ARCHITECTURE, GUIDE, ROADMAP describe the new graded math + guards)

Risk & rollback

  • Risk level: low. Both algorithm swaps preserve the documented behavior (calibrated to the same anchor examples; all prior tests pass) and are advisory-only paths. The new doc guards only ever flag an unresolved anchor / trailing roadmap — validated to catch the real breakage with no false positives on the current tree.
  • Rollback plan: revert the single commit; no data/schema/manifest changes.

Extra checks

  • npm run typecheck passes
  • node src/cli.js docs check passes (all 9 dimensions, incl. new links, roadmap)
  • scripts/bump.mjs check — version fields agree (0.11.0)
  • Input validated at boundaries; errors handled (fail-safe: absent atlas, unreadable/missing link targets, non-numeric CUSUM inputs all degrade gracefully)
  • Logs contain no secrets/PII

🤖 Generated with Claude Code

https://claude.ai/code/session_019PXuKmJp92Gdo2FudNjSx2


Generated by Claude Code

claude added 2 commits July 11, 2026 06:46
…s + link/roadmap guards

Replace the last two static-decision holdouts with graded math, and close two
recurring docs-rot classes with self-enforcing guards.

- anchor.js: goal-drift on-goal-ness is a noisy-OR over path + atlas identifiers
  (not a binary path substring); driftScore becomes a continuous mean(1-score)
  that reduces exactly to the old off-goal fraction on binary scores.
- preflight.js: completeness s(x) is a logistic over its features (concreteness,
  specifics, vagueness, tanh length) — no magic coefficients, no word-count step
  discontinuities, sigmoid-bounded. Calibrated to the paper's examples (0.23/0.63).
- docs_check.js: checkLinks resolves every intra-repo Markdown anchor against real
  headings (GitHub-exact slugs); checkRoadmap fails when ROADMAP trails package.json.
- Fix a fabricated forge route example, dead #install / #use-it-in-a-script anchors,
  and a stale ROADMAP marker (now guarded). +34 tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019PXuKmJp92Gdo2FudNjSx2
…oint + guard robustness

- anchor.js (HIGH): revert driftScore to the off-goal fraction so the CUSUM operating
  point (k=0.35) is unchanged. With noisy-OR a single-hit on-goal file scored 0.6, and the
  graded mean(1-score) made a fully on-goal checkpoint accrue 0.4 drift > k, false-alarming
  after ~20 on-goal prompts. The graded, identifier-aware CLASSIFICATION (the real win) is
  kept; only the magnitude fed to cusum reverts.
- anchor.js (LOW): onGoalScore fuzzy channel is now a >=4-char PREFIX match, not raw
  substring, so 'port' no longer matches 'report' and hides drift; 'auth'->'authentication'
  recall preserved.
- docs_check.js (LOW): headingSlug keeps Unicode letters (Cafe->cafe, matching GitHub);
  checkRoadmap guards the package.json read and parses versions prerelease-safe.
- Reconcile Biome formatting (fixes the CI Lint & format failure).
- Docs + tests updated to the corrected semantics.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019PXuKmJp92Gdo2FudNjSx2
@CodeWithJuber CodeWithJuber marked this pull request as ready for review July 11, 2026 07:20
@CodeWithJuber CodeWithJuber merged commit 20a5144 into master Jul 11, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants