Repository navigation
kb init: judge coverage by the most specific owner; never absorb a module with an ancestor prefix - #258
Merged
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 02f6dad0bd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+194
to
198
| counts = {o.owner: 0 for o in owners} | ||
| for path in files: | ||
| for owner in most_specific(path, owners): | ||
| counts[owner.owner] += 1 | ||
| return max((o for o in owners if counts[o.owner]), key=lambda o: counts[o.owner], default=None) |
There was a problem hiding this comment.
When a module is split among multiple most-specific owners, _owner returns only the owner covering the largest number of files. If that owner's page already bears rules, the loop at lines 145–146 skips the entire module even though its other owners may still be ruleless; for example, a module with two files owned by a rule-bearing owner and one file owned by a ruleless owner can never improve beyond 2/3 coverage, yet deepen proceeds to the shadow flip. Select a ruleless owner for the remaining files or skip only when every most-specific owner represented in the module bears rules.
Useful? React with 👍 / 👎.
Stage 1 of kb init (design kb-init v3): bootstrap a repository's knowledge base from its docs and seed pages, as one human-merged PR. - init_budget: hard per-stage ceiling by reservation. A generator call reserves threshold + one worst-case request (claude's --max-budget-usd only stops the next request); models without a price are refused before any call. Judge calls are accounted at a fixed cost. - init_support: InitRuntime (never KbRuntime, so never kb.db), the per-stage InitRecord, the upstream pin (bare mirror; the pin defaults to the mirror's HEAD, i.e. the upstream default branch, never an assumed main; safe export; PinnedObserver), model calls through the budget, claim/evidence and outside-knowledge path checks, the CI knowledge validators on the result, and InitPublisher (one deterministic commit on an exact base, pushed to a branch that must not exist, PR via the owner's gh; or a local dry run). Writes need ALLOW_PUSH=1, ALLOW_POST=1 and KB_INIT_GIT_AUTHOR (checked before any model call); private upstreams are always dry runs. INIT_PATHS allows knowledge/repos/_index.md, the shared list a new repository joins. - init_stages: run_stage + validate_change (the only check_changeset(bootstrap=True) caller). Skeleton: map call (entry page, architecture page, routes, general links), doc-invariant rules, adapted seed pages. Every rule is D5-filtered, evidenced and checked at the pin, then PLACED on a running tree before judging (a page that fills up sends the rule to a sibling page), so the advisory judge sees each rule where it is written; fail verdicts are stripped. Existing pages are only appended to; routes start from owners read deterministically from existing sub-directory entry pages (code paths they name that exist at the pin); findings on existing pages go to the PR checklist; a failed push is recorded as blocked. suggest_seeds ranks candidates without a model call. - InitConfig gains generator_call_usd and harness_overhead_bytes; build_doc_corpus takes optional globs (doc_files). - knowledge.init step, kb-init playbook, `kb init REPO --stage S [--dry-run] [--pin SHA]` and `kb init REPO --suggest-seeds`, dispatched before any ledger is opened. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
L1 reads any line starting with "## <Word>" as a rule ID (lifecycle.RULE_HEADING / ANY_RULE_HEADING), so a model-written "## Overview" or "## SERV-1 — x" in architecture.md or the entry page intro could collide with real rule IDs or masquerade as a rule. Every non-rule page the skeleton writes now goes through neutral_headings: an ATX heading outside code becomes a bold paragraph, any other line that starts with "#" is escaped, and such a line inside a fence is indented by one space. No line starts with "#", so the rule regexes (all anchored at column 0) can never match. The entry page's list heading is bold too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 2 of kb init (design v3 §7.1-7.2): route every code module. - init_stages: the skeleton's flow becomes a _Stage base shared by every stage (pin, record, rule screening and placement, the deterministic checks as _conclude, the publication); _Skeleton keeps only its own steps. New: the stage chain. Each earlier stage must be merged (record published and gh pr view MERGED, so main holds it), or, for a dry run of this stage, dry-run: its snapshot is overlaid on main as the base and handed to the knowledge validators with this stage's change. A stage with nothing to change is recorded "empty", not blocked. Later stages keep the earlier stages' pin unless --pin says otherwise. - init_modules: scan_modules_at_depth at the pin (language from the adapter's repo.language; none or unknown -> no scan, checklist). A module an owner partly reaches is absorbed: its prefix is appended to that owner's scope_prefixes. Every other module gets one bounded generator call (file names, signatures, leading docstrings, never bodies) and a map card: a prose page (type architecture) in an existing components/<group>/ (index append-only) or a new group with its own _index.md and route owner, linked upward. Sections are bold labels: the knowledge format reads any "## <Word>" heading as a rule ID. The route table is only appended to (routes_append_only). Budget or card cap -> the cards written so far still make a PR; the rest is listed. - InitRecord.coverage: the stage's coverage report, rendered as a before/after table in the PR body. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 3 of kb init (design v3 §7.3, §9 PR 3). - UpstreamPin.first_parent_changes: the PR window, read offline from the mirror: the files each of the last pr_window_count first-parent commits up to the pin changed, no older than pr_window_max_age_days before the PIN's commit time (a function of the pin, never of the clock). - init_deepen: modules by churn in the window, highest first. A module no route reaches, or whose owner already bears rules (the page itself, or a page in the same directory, has an active rule), is skipped. Every other module gets one generator call that reads its code at the pin (numbered, byte-bounded) and proposes rules with line-ranged evidence, which must lie inside the module. They go through the shared screening (D5, pinned evidence and claims, placement, advisory judge) and are appended to the owner's rule page, or rules.md beside it; existing rules are never edited. Stops at coverage_target (rule-bearing PR-weighted), when the budget runs out, or at the end of the list. The record reports routed and rule-bearing coverage before/after and the hot paths still without rules. - The shadow flip in the same PR: flip_to_shadow edits exactly the enabled / mode lines of the adapter's knowledge_lifecycle block (inserting enabled under the header if missing); comments and every other line are kept. _check_flip accepts only that manifest, check_flip_to_shadow, and parse_lifecycle on the head (on, shadow, same knowledge dir). A manifest without the block blocks before any model call. - init_modules.scan_modules is now a module function both stages use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The last kb init stage (design v3 §11). After the deepen PR merged (a dry run chains on the dry-run snapshots), it compares what the advisory judge said about every rule init wrote with what the owner merged, and proposes adapters/<adapter>/kb-calibration/cases/*.json plus knowledge_lifecycle.calibration_set: kb-calibration as its own PR. - Stage records now keep the judged rule text and its pinned evidence in each verdict, so rules the judge stripped or the owner deleted can be turned into cases. - Labels follow the design table: kept pass/unsure/unjudged -> good (source owner); stripped fail -> bad (source judge); deleted or rewritten pass -> bad with override; deleted or rewritten unsure/unjudged -> bad (source owner). A rewritten rule's case holds the text the judge saw. - Cases are self-contained in the load_cases shape: the page without the rule as base, the judged text re-applied through apply_operations as head, evidence re-read from the pinned mirror. A case that does not pass L1 on its own is left out and noted. Case IDs carry the stage: an ID one stage's judge stripped can be reassigned by a later stage. - Deterministic mutations of good rules (broken path claim, shifted evidence range, negated invariant, a sibling module's evidence) balance the set and top owner-sourced bad cases up to five (a broken path must really be absent at the pin, a moved range must show different text); with fewer than five bad cases from the owner or from mutations the stage is blocked, since the judge's own fails cannot calibrate it; so it is with no good case at all (kb calibrate never passes without one). - The manifest edit is one line (set_calibration_set), checked with check_lifecycle_flip(allowed=CALIBRATION_KEYS) and parse_lifecycle; existing cases are never overwritten. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found by the first afd-plugin pilot (skeleton, dry run): - The adapter's repos/ seed was a type: guide page; the skeleton only adapted type: rule pages, so it was skipped with no trace, which the design forbids (a seed is never skipped silently). Any content page a seed names is now adapted (indexes are navigation, not content). A seed naming no content page goes on the checklist, a page that transfers no rule goes in the notes, and a budget stop lists that seed and every one after it as unfinished. - The PR body said "Model spend (accounted): $0.00" while the record held $7.42: spent_usd was written in the stage's finally block, after the body was rendered. It is now written before rendering. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tzhouam
force-pushed
the
kb/init-pilot-fixes
branch
from
September 30, 2026 15:37
66d3231 to
81b8a48
Compare
…dule with an ancestor prefix The afd-plugin pilot (dry run) made the coverage metric meaningless. The skeleton wrote an aggregate owner "components" listing its components' prefixes; the modules stage absorbed the package root by appending afd_plugin/ to it, a catch-all that matches every file; deepen gave the hottest module to that owner (it reached as many files as the component owner and came first in routing order) and wrote 12 rules to components/rules.md; and pr_weighted_coverage counted a file rule-bearing when ANY matching owner had rules, so coverage jumped 10% -> 100% and deepen stopped with 8 hot modules untouched. - init_coverage.most_specific: the owners with the longest matching prefix, then the deepest page (a component's own page beats an aggregate entry page listing the same area); rule-bearing now uses only those. shadowing() lists directory prefixes that are strict ancestors of another owner's prefix. - init_modules: absorption picks the owner that owns most of the module's files most specifically, and cover_prefixes adds only prefixes for the module's own unrouted files: the module directory when no other module or OTHER owner's prefix lies under it, else the highest such directory per file, else the file. A module needing more than 20 prefixes gets a map card instead; card owners use the same cover. - init_deepen: a module's owner is its most specific owner; "already bears rules" and the uncovered-hot list use the same rule. - skeleton: a checklist note names every owner prefix that shadows other owners' prefixes, so the owner can narrow it. Replayed offline on the pilot's pin: the root absorbs 4 files instead of afd_plugin/, the hottest module goes to execution-platforms, and rules on the aggregate page now count for 4.5%, not 100%. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tzhouam
force-pushed
the
kb/init-owner-specificity
branch
from
September 30, 2026 15:37
02f6dad to
aa5523a
Compare
tzhouam
added a commit
that referenced
this pull request
Sep 30, 2026
…pped The afd-plugin pilot re-run on #258 wrote 36 rules on 3 owner pages, reached 74% rule-bearing coverage of an 85% target, and stopped with no note and nothing unfinished. The hottest files still without rules (v1/worker/attention_model_runner.py, ffn_model_runner.py, ...) sit in modules already visited, but their most specific owners (attention-runtime, ffn-runtime) differ from the module's majority owner, and the loop went module by module, so it never reached them. - Deepen now works on groups: the files of one module that share one most specific owner. Groups are ordered by churn; one bounded generator call per group sees only that group's files, and evidence must lie in them. A group whose owner already bears rules is skipped (counted). Caps, budget reservation, D5, pinned claims and the advisory judge are unchanged. - The stop reason is always recorded in the notes (and so the PR body): target reached, budget exhausted (the remaining groups stay in unfinished, labelled with their owner), or "every hot group visited; target not reached (X% < Y%)" with the five hottest files still without rules. Hot files no route reaches are listed too. The pass never ends below target silently. Replayed offline on the pilot (no model calls): from the pre-deepen state the loop makes 5 calls (execution-platforms, model-integration, connectors, attention-runtime, ffn-runtime) and reaches 81.8%; after the module-loop re-run, exactly the attention-runtime and ffn-runtime groups remain to visit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pilot evidence
The afd-plugin pilot (dry run, pin
d360690a590a) reported deepen coverage 10% → 100% from one module's 12 rules, and stopped with 8 hot modules untouched ("coverage target reached"). Cause, one defect in three places:components(pagecomponents/_index.md) listing its components' prefixes (afd_plugin/connectors/,model_executor/,compat/).afd_plugin/by appendingafd_plugin/to that owner — a catch-all matching every file.components(it reached as many files as the component owner and came first in routing order) and wrote the rules tocomponents/rules.md;pr_weighted_coveragecounted a file rule-bearing when any matching owner had rules.Fix
init_coverage.most_specific(path, owners): owners with the longest matching prefix, then the deepest page (a component's own page beats an aggregate entry page listing the same area); ties on both return all. Rule-bearing uses only these.routedstill means any owner.shadowing(owners)lists directory prefixes that are strict ancestors of another owner's prefix.init_modules: absorption picks the owner that owns most of the module's files most specifically;cover_prefixesadds prefixes only for the module's own unrouted files — the module directory when no other module and no other owner's prefix lies under it, else the highest such directory per file, else the file. A module needing more than 20 prefixes gets a map card instead (card owners use the same cover). A module with child modules (package root, repo root) is never routed by its own directory.init_deepen: a module's owner is its most specific owner; "already bears rules" anduncovered_hotuse the same rule.Offline replay on the pilot's pin (no model calls)
afd_plugin/{config,config_utils,envs,validation}.pyintoconnectors(was:afd_plugin/intocomponents).afd_plugin/v1/worker/absorbs 6 files +afd_plugin/v1/worker/npu/intoexecution-platforms(was: the whole directory, which swallowed the attention/ffn runner owners' files).afd_plugin/v1/worker/, 222 changes) goes toexecution-platforms; rules on the aggregatecomponentspage now count for 4.5% rule-bearing (pilot: 100%); rules onexecution-platforms→ 22.2%.connectorsbecause folded sub-directories (distributed/) make it the owner of most root-module files — deterministic but semantically debatable; the skeleton checklist already flagscomponents→afd_plugin/compat/shadowing.Tests
test/test_kb_init_owner_specificity.py(9): specificity + page-depth tie-break, catch-all rules not counting for component files, shadowing, root-with-children cover, directory blocked by another owner's prefix but allowed for the same owner, repo-root module, deepen owner choice, a miniature of the pilot shape (<100% after one module's rules), skeleton checklist note. Existing kb init suites unchanged and green. Commit gate approved (20260930T133823…-codeloop-i1-approve); local CI mirror green (pytest, doc links/citations, SPEC freshness --strict, knowledge linters).Stacked on #256.
🤖 Generated with Claude Code