Skip to content

kb init: deepen stage (code rules for hot modules) and the shadow flip - #252

Merged
tzhouam merged 4 commits into
mainfrom
kb/init-deepen
Sep 30, 2026
Merged

kb init: deepen stage (code rules for hot modules) and the shadow flip#252
tzhouam merged 4 commits into
mainfrom
kb/init-deepen

Conversation

@tzhouam

@tzhouam tzhouam commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Stage 3 of kb init (design v3 §7.3, §9 PR 3): code rules for hot modules, and the flip to shadow mode.

  • PR window (UpstreamPin.first_parent_changes), read offline from the mirror.
    • It covers the files changed by each of the last pr_window_count first-parent commits up to the pin.
    • Commits older than pr_window_max_age_days before the pin's commit time are skipped. So the window is fixed by the pin, not by the clock.
    • Paths are read with -z, so non-ASCII names aren't quoted. Expired commits are skipped rather than ending the scan, because first-parent commit times need not decrease.
    • On a merge- or squash-based default branch, each first-parent commit is one merged PR.
  • init_deepen. It takes modules in order of churn, highest first.
    • A module is skipped when no route reaches it, or when its route owner already bears rules. "Bears rules" means the owner page has an active rule, or, for an entry or prose owner, a page in the same directory does.
    • Every other module gets one generator call that reads its code at the pin (numbered and byte-bounded). It proposes rules whose line-ranged evidence must lie inside that module.
    • Those rules go through the shared screening: D5, pinned evidence and claims, placement, and the advisory judge. They're then appended to the owner's rule page, or to rules.md beside it. Existing rules are never edited, and a full page spills over to a sibling page.
    • The stage stops when rule-bearing PR-weighted coverage reaches coverage_target, when the budget runs out, or at the end of the list.
    • The report gives routed and rule-bearing coverage before and after, the hot paths still without rules (routed or not), and the unrouted ones.
  • Shadow flip, in the same PR.
    • flip_to_shadow is a textual edit of exactly the enabled and mode lines in the adapter's knowledge_lifecycle block. It inserts enabled under the block header if the line is missing, and keeps comments and every other line byte for byte.
    • _check_flip accepts only that manifest. It runs check_flip_to_shadow, then parse_lifecycle on the result, which must be on, in shadow mode, and serving the same knowledge directory.
    • A manifest without the block blocks the stage before any model call.

Deviations

  • PR window source. It comes from the upstream mirror's first-parent history, not from GitHubReader (spec "PR E"). That makes it pinned, offline and free of API rate limits; unauthenticated search plus per-PR file listing would take roughly 200 calls per run.
  • "Bears rules" is limited to the same directory. Otherwise the repository entry page would count as rule-bearing as soon as any component had a rule, which would inflate the metric.
  • Refactor: init_modules.scan_modules is now a module-level function that both stages use.

Tests

  • test/test_kb_init_deepen.py (12 tests):
    • hot module rules, the evidence-outside-module drop, a new rule page with an index link, coverage before/after, and the exact two-line flip
    • stops at the coverage target and still flips
    • budget exhaustion keeps the rules already written and reports routed-but-ruleless hot paths
    • a pre-existing rule page is only appended to
    • a missing lifecycle block blocks before any model call
    • flip_to_shadow variants; the flip check refuses any other edit
    • an empty rule-page owner bears nothing whatever its siblings hold
    • the window is bounded by count and by age from the pin, handles non-ASCII names and out-of-order times
  • Full imc-check.sh: rc=0.
  • Commit gate (gpt-6.1-sol-high): approved.

Part 7/9 of kb init (design v3, sol-approved), stacked on #248 (via #251).

🤖 Generated with Claude Code

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 30, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-09-30T12:11:27.718449Z da346b5 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@tzhouam tzhouam mentioned this pull request Sep 30, 2026
Merged
tzhouam added a commit that referenced this pull request Sep 30, 2026
@tzhouam @claude
Let /imkbinit run every stage now that all four exist
955178b 
modules (#251), deepen (#252) and harvest-calibration (#254) landed, so
the skill no longer treats them as unimplemented: it runs the next
stage and says what each one opens. The rules are unchanged: only
MERGED PRs count as done when publishing, and dry-run records chain in
dry-run mode. It notes that harvest refuses fewer than five
owner-or-synthetic bad cases, and that auto_merge still needs
kb calibrate and the shadow period (the skill never flips it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tzhouam tzhouam mentioned this pull request Sep 30, 2026
Merged
tzhouam and others added 3 commits September 30, 2026 23:36
@tzhouam @claude
kb init: skeleton stage with its own runtime, budget and publisher
f0d822f 
Stage 1 of kb init (design kb-init v3): bootstrap a repository's
knowledge base from its docs and seed pages, as one human-merged PR.

- init_budget: hard per-stage ceiling by reservation. A generator call
  reserves threshold + one worst-case request (claude's
  --max-budget-usd only stops the next request); models without a price
  are refused before any call. Judge calls are accounted at a fixed cost.
- init_support: InitRuntime (never KbRuntime, so never kb.db), the
  per-stage InitRecord, the upstream pin (bare mirror; the pin defaults
  to the mirror's HEAD, i.e. the upstream default branch, never an
  assumed main; safe export; PinnedObserver), model calls through the
  budget, claim/evidence and outside-knowledge path checks, the CI
  knowledge validators on the result, and InitPublisher (one
  deterministic commit on an exact base, pushed to a branch that must
  not exist, PR via the owner's gh; or a local dry run). Writes need
  ALLOW_PUSH=1, ALLOW_POST=1 and KB_INIT_GIT_AUTHOR (checked before any
  model call); private upstreams are always dry runs. INIT_PATHS allows
  knowledge/repos/_index.md, the shared list a new repository joins.
- init_stages: run_stage + validate_change (the only
  check_changeset(bootstrap=True) caller). Skeleton: map call (entry
  page, architecture page, routes, general links), doc-invariant rules,
  adapted seed pages. Every rule is D5-filtered, evidenced and checked at
  the pin, then PLACED on a running tree before judging (a page that
  fills up sends the rule to a sibling page), so the advisory judge sees
  each rule where it is written; fail verdicts are stripped. Existing
  pages are only appended to; routes start from owners read
  deterministically from existing sub-directory entry pages (code paths
  they name that exist at the pin); findings on existing pages go to the
  PR checklist; a failed push is recorded as blocked. suggest_seeds
  ranks candidates without a model call.
- InitConfig gains generator_call_usd and harness_overhead_bytes;
  build_doc_corpus takes optional globs (doc_files).
- knowledge.init step, kb-init playbook, `kb init REPO --stage S
  [--dry-run] [--pin SHA]` and `kb init REPO --suggest-seeds`, dispatched
  before any ledger is opened.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tzhouam @claude
kb init: never let generated prose carry a rule heading
7e98f94 
L1 reads any line starting with "## <Word>" as a rule ID
(lifecycle.RULE_HEADING / ANY_RULE_HEADING), so a model-written
"## Overview" or "## SERV-1 — x" in architecture.md or the entry page
intro could collide with real rule IDs or masquerade as a rule. Every
non-rule page the skeleton writes now goes through neutral_headings: an
ATX heading outside code becomes a bold paragraph, any other line that
starts with "#" is escaped, and such a line inside a fence is indented
by one space. No line starts with "#", so the rule regexes (all anchored
at column 0) can never match. The entry page's list heading is bold too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tzhouam @claude
kb init: modules stage (module map cards and routes)
f24e5d1 
Stage 2 of kb init (design v3 §7.1-7.2): route every code module.

- init_stages: the skeleton's flow becomes a _Stage base shared by every
  stage (pin, record, rule screening and placement, the deterministic
  checks as _conclude, the publication); _Skeleton keeps only its own
  steps. New: the stage chain. Each earlier stage must be merged (record
  published and gh pr view MERGED, so main holds it), or, for a dry run
  of this stage, dry-run: its snapshot is overlaid on main as the base and
  handed to the knowledge validators with this stage's change. A stage
  with nothing to change is recorded "empty", not blocked. Later stages
  keep the earlier stages' pin unless --pin says otherwise.
- init_modules: scan_modules_at_depth at the pin (language from the
  adapter's repo.language; none or unknown -> no scan, checklist). A
  module an owner partly reaches is absorbed: its prefix is appended to
  that owner's scope_prefixes. Every other module gets one bounded
  generator call (file names, signatures, leading docstrings, never
  bodies) and a map card: a prose page (type architecture) in an existing
  components/<group>/ (index append-only) or a new group with its own
  _index.md and route owner, linked upward. Sections are bold labels: the
  knowledge format reads any "## <Word>" heading as a rule ID. The route
  table is only appended to (routes_append_only). Budget or card cap ->
  the cards written so far still make a PR; the rest is listed.
- InitRecord.coverage: the stage's coverage report, rendered as a
  before/after table in the PR body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tzhouam @claude
kb init: deepen stage (code rules for hot modules) and the shadow flip
1c6ceea 
Stage 3 of kb init (design v3 §7.3, §9 PR 3).

- UpstreamPin.first_parent_changes: the PR window, read offline from the
  mirror: the files each of the last pr_window_count first-parent commits
  up to the pin changed, no older than pr_window_max_age_days before the
  PIN's commit time (a function of the pin, never of the clock).
- init_deepen: modules by churn in the window, highest first. A module no
  route reaches, or whose owner already bears rules (the page itself, or a
  page in the same directory, has an active rule), is skipped. Every other
  module gets one generator call that reads its code at the pin (numbered,
  byte-bounded) and proposes rules with line-ranged evidence, which must lie
  inside the module. They go through the shared screening (D5, pinned
  evidence and claims, placement, advisory judge) and are appended to the
  owner's rule page, or rules.md beside it; existing rules are never
  edited. Stops at coverage_target (rule-bearing PR-weighted), when the
  budget runs out, or at the end of the list. The record reports routed
  and rule-bearing coverage before/after and the hot paths still without
  rules.
- The shadow flip in the same PR: flip_to_shadow edits exactly the
  enabled / mode lines of the adapter's knowledge_lifecycle block
  (inserting enabled under the header if missing); comments and every
  other line are kept. _check_flip accepts only that manifest,
  check_flip_to_shadow, and parse_lifecycle on the head (on, shadow, same
  knowledge dir). A manifest without the block blocks before any model
  call.
- init_modules.scan_modules is now a module function both stages use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tzhouam
tzhouam changed the base branch from kb/init-modules to main September 30, 2026 15:37
tzhouam added a commit that referenced this pull request Sep 30, 2026
@tzhouam @claude
Let /imkbinit run every stage now that all four exist
9418ac0 
modules (#251), deepen (#252) and harvest-calibration (#254) landed, so
the skill no longer treats them as unimplemented: it runs the next
stage and says what each one opens. The rules are unchanged: only
MERGED PRs count as done when publishing, and dry-run records chain in
dry-run mode. It notes that harvest refuses fewer than five
owner-or-synthetic bad cases, and that auto_merge still needs
kb calibrate and the shadow period (the skill never flips it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tzhouam
tzhouam merged commit 59cb218 into main Sep 30, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@tzhouam