feat(anchor,preflight,docs): graded goal-drift + logistic completeness + self-enforcing doc guards#51
Merged
Conversation
…s + link/roadmap guards Replace the last two static-decision holdouts with graded math, and close two recurring docs-rot classes with self-enforcing guards. - anchor.js: goal-drift on-goal-ness is a noisy-OR over path + atlas identifiers (not a binary path substring); driftScore becomes a continuous mean(1-score) that reduces exactly to the old off-goal fraction on binary scores. - preflight.js: completeness s(x) is a logistic over its features (concreteness, specifics, vagueness, tanh length) — no magic coefficients, no word-count step discontinuities, sigmoid-bounded. Calibrated to the paper's examples (0.23/0.63). - docs_check.js: checkLinks resolves every intra-repo Markdown anchor against real headings (GitHub-exact slugs); checkRoadmap fails when ROADMAP trails package.json. - Fix a fabricated forge route example, dead #install / #use-it-in-a-script anchors, and a stale ROADMAP marker (now guarded). +34 tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019PXuKmJp92Gdo2FudNjSx2
…oint + guard robustness - anchor.js (HIGH): revert driftScore to the off-goal fraction so the CUSUM operating point (k=0.35) is unchanged. With noisy-OR a single-hit on-goal file scored 0.6, and the graded mean(1-score) made a fully on-goal checkpoint accrue 0.4 drift > k, false-alarming after ~20 on-goal prompts. The graded, identifier-aware CLASSIFICATION (the real win) is kept; only the magnitude fed to cusum reverts. - anchor.js (LOW): onGoalScore fuzzy channel is now a >=4-char PREFIX match, not raw substring, so 'port' no longer matches 'report' and hides drift; 'auth'->'authentication' recall preserved. - docs_check.js (LOW): headingSlug keeps Unicode letters (Cafe->cafe, matching GitHub); checkRoadmap guards the package.json read and parses versions prerelease-safe. - Reconcile Biome formatting (fixes the CI Lint & format failure). - Docs + tests updated to the corrected semantics. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019PXuKmJp92Gdo2FudNjSx2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Auditing the whitepaper and every module against the project's own rule — "DATA may be a table, but DECISIONS must be a formula — graded, inspectable, testable" — found the codebase already overwhelmingly mathematical (route, intent, lessons, ledger, reuse, context, atlas-impact, predictor, diagnose are all properly graded). Exactly two static-decision holdouts remained, plus a set of concrete docs problems. This PR closes both, and adds guards so the doc problems can't silently recur.
Static decisions → graded formulas
src/anchor.js(M4 goal-drift): on-goal-ness was a binary path-substring match (path.includes(keyword)) that fed a graded CUSUM — a graded wrapper over an ungraded core. NowonGoalScoreis a noisy-OR (1 − (1 − p)^hits, the same estimatorlessons.jsuses) over how many distinct goal concepts a file exhibits in its path and the identifiers it defines (via the atlas). So a file that implements the goal but never names it in its path is caught deterministically, not just by the opt-in LLM pass.driftScorebecomes a continuousmean(1 − score)that strictly generalizes the old off-goal fraction (identical on binary scores, graded otherwise), sharpening the drift detector's input.src/preflight.js(M2 completenesss(x)): the completeness score was a hand-tuned additive scorer with magic coefficients and discontinuous word-count step bonuses. Now it's a logistic over the same features (concreteness, named specifics, vagueness, a smoothtanhlength term), reusingpredictor.js'ssigmoid. Smooth (no step jumps), self-bounding (no ad-hoc clamp), every feature attributable, and a labeled bank could refine the weights viatrainLogistic. Calibrated so the paper's own examples land where documented (make the auth better≈ 0.23 → ask; the concrete verifyToken edit ≈ 0.63 → proceed).Docs problems fixed (found by a full-doc audit) + made self-enforcing
forge routeexample indocs/GUIDE.md(an impossibleFable 5 / Opus/ "premium tier" output) → the real routed verdict.#installanchors (ONBOARDING.md, substrate README — README has no Install heading) →#60-second-quickstart; a dead#use-it-in-a-scriptself-link resolved; stale ROADMAP marker →v0.11.0; a couple of front-page jargon/consistency touches.forge docs checkguards:checkLinksresolves every intra-repo Markdown anchor (#xandpath.md#x) against the target's real headings using GitHub-exact slugs (an em-dash yields--, never collapsed; code-fence link syntax is skipped);checkRoadmapfails when the ROADMAP "Now" marker trailspackage.json. These make the classes above CI failures, not recurring complaints.The research whitepaper itself was audited and found rigorous and internally consistent (its reported 62.1% / 30-30 gate / 21-21 routing numbers match
eval_results.jsonexactly); no corrections needed there.Checklist
npm testpasses (658 pass, 0 fail, 2 skipped)npm run checkpasses (Biome lint + format)onGoalScore,completenessScore/completenessFeatures,checkLinks,checkRoadmap; +34 tests)CHANGELOG.mdupdated under## [Unreleased]math.js/predictor.jsprimitives)Risk & rollback
Extra checks
npm run typecheckpassesnode src/cli.js docs checkpasses (all 9 dimensions, incl. newlinks,roadmap)scripts/bump.mjs check— version fields agree (0.11.0)🤖 Generated with Claude Code
https://claude.ai/code/session_019PXuKmJp92Gdo2FudNjSx2
Generated by Claude Code