Skip to content

kb init: modules stage (module map cards and routes) - #251

Merged
tzhouam merged 3 commits into
mainfrom
kb/init-modules
Sep 30, 2026
Merged

tzhouam merged 3 commits into
mainfrom
kb/init-modules

Conversation

@tzhouam

@tzhouam tzhouam commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Stage 2 of kb init (design v3 §7.1–§7.2): every code module gets a route.

  • Shared stage flow. init_stages._Skeleton is split into a _Stage base, which every stage uses, plus _Skeleton. The base handles the pin, the record, rule screening and placement, the deterministic checks (_conclude) and publishing. It's behaviour-preserving: all skeleton tests pass unchanged except one, which now expects deepen rather than modules to be unimplemented.
  • Stage chain (design §9).
    • Every earlier stage must be merged: its record is published and gh pr view says MERGED, so main already holds it.
    • Alternatively, when this stage is a dry run, an earlier stage may be a dry run too. Its snapshot is then laid over main as this stage's base, and handed to the knowledge validators along with this stage's change.
    • A stage with nothing to change is recorded as empty, which doesn't block the next stage.
    • Later stages keep the earlier stages' pin unless --pin says otherwise. The chain state is part of the inputs digest.
  • init_modules.
    • Runs scan_modules_at_depth at the pin. The language comes from the adapter's repo.language; if none is declared or it's unknown, nothing is scanned and a checklist item is added.
    • Modules are processed deepest first. A module some owner already partly reaches is absorbed: its prefix is appended to that owner's scope_prefixes.
    • Every other module gets one bounded generator call and a map card. The call sees file names, declarations and leading docstrings, never a function body.
    • A map card is a prose page (type: architecture) listing purpose, entry points, key files (only files that exist in the module), docs to read (only files in the doc set) and routes.
    • Cards go into an existing components/<group>/ (append-only index, append-only owner), or into a new group with its own _index.md and route owner, linked from the page above it.
    • _routes.yaml is only appended to; routes_append_only guards this.
    • When the budget or the card cap runs out, the cards written so far still make a PR, and the remaining modules are listed as unfinished.
  • InitRecord.coverage: the stage's coverage report, rendered as a before/after table in the PR body.

Deviations and findings

  • Card sections are bold labels, not ## headings. The knowledge format reads any ## <Word> … heading as a rule ID, so English card headings collided (duplicate_rule_id). The skeleton stage's architecture.md has the same hazard when the model writes English ## headings. PR kb init: skeleton stage (docs map, doc invariants, seeds) with its own runtime, budget and publisher #248 should strip or convert those; I noted it for the owner and did not fix it here.
  • Fail-closed declarations for brace languages. For JavaScript, Go and Rust, declaration() cuts each signature at the earliest { / => / ; / lone =, without lexing strings, comments or regex literals. A body can't start earlier, so nothing after it is sent; a default value may be cut short. Python uses a bracket-, string- and comment-aware scan up to the header's :. The commit gate found three JS lexing holes in a scanner version first, which is why the simpler rule won.
  • Map cards are prose pages. Their backticked paths are filtered to files at the pin, but they aren't pinned claims (only rules are).

Tests

  • test/test_kb_init_modules.py (11 tests):
    • a dry-run chain from a skeleton dry run: absorption, a new group with its own index, upward links and coverage
    • needs the skeleton; publishing needs the skeleton merged (OPEN blocks, MERGED builds on main)
    • budget exhaustion mid-way still gives a valid PR with an unfinished list
    • joining an existing group append-only
    • no language means an empty stage
    • routes_append_only, declaration, _leading_doc, an empty owners: table
    • card prompts carry declarations only
  • Full imc-check.sh: rc=0 (full pytest, doc links and citations, SPEC freshness --strict, knowledge linters).
  • Commit gate (gpt-6.1-sol-high): approved on the final amend.

Part 6/9 of kb init (design v3, sol-approved), stacked on #248.

🤖 Generated with Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eebcef89bc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +316 to +320
if used + size > MAX_CARD_BYTES:
entry = {"path": rel}
size = len(rel) + 12
listed.append(entry)
used += size

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Enforce the byte cap after falling back to paths

When a module's detailed entries reach MAX_CARD_BYTES, each remaining file is still appended as a path-only entry, and used can continue growing without limit until the separate 200-file cap. A module with many long paths can therefore produce a prompt substantially larger than the advertised 40 KB bound, potentially exceeding the model context or increasing cost and blocking the entire stage. Stop adding entries when even the path-only form would exceed the remaining byte budget.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-30T11:55:30.947463Z eebcef8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

tzhouam added a commit that referenced this pull request Sep 30, 2026
modules (#251), deepen (#252) and harvest-calibration (#254) landed, so
the skill no longer treats them as unimplemented: it runs the next
stage and says what each one opens. The rules are unchanged: only
MERGED PRs count as done when publishing, and dry-run records chain in
dry-run mode. It notes that harvest refuses fewer than five
owner-or-synthetic bad cases, and that auto_merge still needs
kb calibrate and the shadow period (the skill never flips it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tzhouam and others added 2 commits September 30, 2026 23:36
Stage 1 of kb init (design kb-init v3): bootstrap a repository's
knowledge base from its docs and seed pages, as one human-merged PR.

- init_budget: hard per-stage ceiling by reservation. A generator call
  reserves threshold + one worst-case request (claude's
  --max-budget-usd only stops the next request); models without a price
  are refused before any call. Judge calls are accounted at a fixed cost.
- init_support: InitRuntime (never KbRuntime, so never kb.db), the
  per-stage InitRecord, the upstream pin (bare mirror; the pin defaults
  to the mirror's HEAD, i.e. the upstream default branch, never an
  assumed main; safe export; PinnedObserver), model calls through the
  budget, claim/evidence and outside-knowledge path checks, the CI
  knowledge validators on the result, and InitPublisher (one
  deterministic commit on an exact base, pushed to a branch that must
  not exist, PR via the owner's gh; or a local dry run). Writes need
  ALLOW_PUSH=1, ALLOW_POST=1 and KB_INIT_GIT_AUTHOR (checked before any
  model call); private upstreams are always dry runs. INIT_PATHS allows
  knowledge/repos/_index.md, the shared list a new repository joins.
- init_stages: run_stage + validate_change (the only
  check_changeset(bootstrap=True) caller). Skeleton: map call (entry
  page, architecture page, routes, general links), doc-invariant rules,
  adapted seed pages. Every rule is D5-filtered, evidenced and checked at
  the pin, then PLACED on a running tree before judging (a page that
  fills up sends the rule to a sibling page), so the advisory judge sees
  each rule where it is written; fail verdicts are stripped. Existing
  pages are only appended to; routes start from owners read
  deterministically from existing sub-directory entry pages (code paths
  they name that exist at the pin); findings on existing pages go to the
  PR checklist; a failed push is recorded as blocked. suggest_seeds
  ranks candidates without a model call.
- InitConfig gains generator_call_usd and harness_overhead_bytes;
  build_doc_corpus takes optional globs (doc_files).
- knowledge.init step, kb-init playbook, `kb init REPO --stage S
  [--dry-run] [--pin SHA]` and `kb init REPO --suggest-seeds`, dispatched
  before any ledger is opened.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
L1 reads any line starting with "## <Word>" as a rule ID
(lifecycle.RULE_HEADING / ANY_RULE_HEADING), so a model-written
"## Overview" or "## SERV-1 — x" in architecture.md or the entry page
intro could collide with real rule IDs or masquerade as a rule. Every
non-rule page the skeleton writes now goes through neutral_headings: an
ATX heading outside code becomes a bold paragraph, any other line that
starts with "#" is escaped, and such a line inside a fence is indented
by one space. No line starts with "#", so the rule regexes (all anchored
at column 0) can never match. The entry page's list heading is bold too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 2 of kb init (design v3 §7.1-7.2): route every code module.

- init_stages: the skeleton's flow becomes a _Stage base shared by every
  stage (pin, record, rule screening and placement, the deterministic
  checks as _conclude, the publication); _Skeleton keeps only its own
  steps. New: the stage chain. Each earlier stage must be merged (record
  published and gh pr view MERGED, so main holds it), or, for a dry run
  of this stage, dry-run: its snapshot is overlaid on main as the base and
  handed to the knowledge validators with this stage's change. A stage
  with nothing to change is recorded "empty", not blocked. Later stages
  keep the earlier stages' pin unless --pin says otherwise.
- init_modules: scan_modules_at_depth at the pin (language from the
  adapter's repo.language; none or unknown -> no scan, checklist). A
  module an owner partly reaches is absorbed: its prefix is appended to
  that owner's scope_prefixes. Every other module gets one bounded
  generator call (file names, signatures, leading docstrings, never
  bodies) and a map card: a prose page (type architecture) in an existing
  components/<group>/ (index append-only) or a new group with its own
  _index.md and route owner, linked upward. Sections are bold labels: the
  knowledge format reads any "## <Word>" heading as a rule ID. The route
  table is only appended to (routes_append_only). Budget or card cap ->
  the cards written so far still make a PR; the rest is listed.
- InitRecord.coverage: the stage's coverage report, rendered as a
  before/after table in the PR body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tzhouam
tzhouam changed the base branch from kb/init-skeleton to main September 30, 2026 15:37
tzhouam added a commit that referenced this pull request Sep 30, 2026
modules (#251), deepen (#252) and harvest-calibration (#254) landed, so
the skill no longer treats them as unimplemented: it runs the next
stage and says what each one opens. The rules are unchanged: only
MERGED PRs count as done when publishing, and dry-run records chain in
dry-run mode. It notes that harvest refuses fewer than five
owner-or-synthetic bad cases, and that auto_merge still needs
kb calibrate and the shadow period (the skill never flips it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tzhouam
tzhouam merged commit 2efcef4 into main Sep 30, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant