Skip to content
Merged
11 changes: 9 additions & 2 deletions doc/architecture/SPEC/engine/steps/knowledge.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# engine/steps/knowledge.py —— 规范

<!-- verified-against: 2026-09-28 -->
<!-- verified-against: 2026-09-30 -->

`知识服务的 knowledge.* 步骤 · refactor-status: new`

Expand All @@ -12,10 +12,17 @@ auto_merge 仓库写 outbox 的 `open_pr` 项,shadow 只记录)。逻辑全
## 不变量
- 只处理 `params.repo` 指定的一个仓库;未知或未启用的仓库 → BLOCKED。
- 步骤之间只经 `state_updates` 传递纯数据(`kb_repo`、`kb_changeset`、`kb_changeset_status`)。
- 从不直接写 GitHub;发布只经签名 outbox 由 GPU 盒发布器执行。
- 从不直接写 GitHub;发布只经签名 outbox 由 GPU 盒发布器执行(唯一例外:`knowledge.init`,见下)。

## 测试
`test_kb_intake_gate.py`。

## 2026-09-28 合并、巡检、激活步骤
新增 `knowledge.advance_merges`、`knowledge.sweep`、`knowledge.activate`;`kb run` 路径自行持有租约,调度器路径复用其租约。

## 2026-09-30 `knowledge.init`(kb init)
参数 `repo`、`stage`、`dry_run`(缺省 true)、`pin`。用自己的 `InitRuntime`(`kb_service/init_support.py`),**不**经
`_runtime`/`_lifecycle`:服务运行时会打开 `kb.db`,且 `_lifecycle` 拒绝未启用的仓库,而为未启用的仓库建库正是 init 的目的。
调 `init_stages.run_stage`;`InitError`/`NotImplementedError` 与状态 `blocked` 的记录都返回 BLOCKED。`state_updates`:
`kb_init_stage`、`kb_init_status`、`kb_init_pr`。它是本模块里唯一直接写 GitHub 的步骤:经仓库主人的 `gh` 开一个由人合并的
PR,且只在 `ALLOW_PUSH=1` 与 `ALLOW_POST=1` 同时成立、非 dry run、上游非私有时。测试:`test_kb_init_skeleton.py`。
148 changes: 146 additions & 2 deletions doc/architecture/SPEC/kb_service.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions doc/architecture/SPEC/knowledge_service/l1.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
- 共享的 `knowledge/repos/_index.md` 可以被**修改**(登记新仓库),新增/删除仍拒绝。
其余规则(尾注、frontmatter、`sources`、白名单其余部分、保护规则)不变;默认 `False`
时行为与原来逐字节一致。
- `link_targets(directory, text)`:公开的只读帮助函数,返回 `text`(位于 `directory` 的页面)按 CommonMark 渲染后实际链接到的知识库相对路径集合(与 `check_index_links` 同一解析器);kb init 用它确认新索引导语不再链接目录清单已列出的页面。
- `check_index_links(base, head)`:init 侧的索引链接检查(索引改成表格或跨目录链接时
L1 只把它当 prose 块)。**链接 = 渲染器实际生成的链接**:用 markdown-it-py 的 CommonMark 解析器
(开启 GitHub 管道表格)取所有 `link_open` 的 href(图片及其 alt 文本里的不算);先按 URL 拆分(编码的 `#`、`:` 属于文件名),
Expand Down
6 changes: 5 additions & 1 deletion doc/architecture/SPEC/playbooks/PLAYBOOKS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# playbooks/*.yaml —— 规范

<!-- verified-against: 2026-09-29 -->
<!-- verified-against: 2026-09-30 -->

`9 个文件 · 声明式编排数据 · refactor-status: ok`

Expand Down Expand Up @@ -57,3 +57,7 @@ success, steps[]`。

## kb-merge / kb-sweep / kb-activate(candidate)
知识服务的合并推进、发版巡检与快照激活;candidate 状态,由 `kb serve` 调度或 `kb run` 手动运行。

## kb-init(candidate)
为一个仓库建知识库:单步 `knowledge.init`,每次运行一个阶段(skeleton → modules → deepen + 生命周期开关 →
harvest-calibration),每阶段一个由人合并的 PR。只由 `kb init REPO --stage S` 运行,不经 `kb serve`,不写服务账本。
7 changes: 5 additions & 2 deletions doc/architecture/SPEC/profiles/establish.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

## 公开契约
`fact_id`、`build_doc_corpus`、`is_redundant`、`extract_directives`、
`scan_modules`、`scan_modules_at_depth`、`normalize_root`、`ROOT_MODULE`、`HUMAN_DOC_NAMES`。
`scan_modules`、`scan_modules_at_depth`、`normalize_root`、`ROOT_MODULE`、`HUMAN_DOC_NAMES`、`doc_files`。

## 不变量
- `is_redundant`(对 README+docs 做 6 词 shingle)会丢弃任何**仓库自己的文档已经写过**
Expand All @@ -19,6 +19,9 @@
- `scan_modules_at_depth`(kb init 的模块定义):源根以下至多 `depth` 层目录为模块,更深的文件归最深的那层祖先;
非空行少于 `min_loc` 的模块由深到浅折入父目录模块,直到稳定;源根永不折叠(`normalize_root` 先把 `.`、`./pkg`、`/pkg/` 等写法规范化,仓库根为 `""` 且包含其他根,保证折叠必然终止);嵌套源根各自拥有自己的文件;
每一层都跳过隐藏目录与非代码目录;`exclude` 按仓库相对路径做 fnmatch;未知语言返回 `{}`;结果确定、有序。
- `build_doc_corpus(..., globs=None)`:不给 `globs` 时语料与原来完全相同(README* + docs/**/*.md);给出时读
`doc_files(repo, globs)` —— 按 glob 顺序、再按路径排序,每个文件只读一次,只收普通文件,解析后逃出仓库的
符号链接一律跳过。kb init 用它按 adapter 的 `doc_globs` 建 D5 语料,归一化与 briefing 路径是同一份。

## 边界 —— 不属于这里
纯确定性 helper —— 不含 LLM、不写 store、不含 step 逻辑。
Expand All @@ -27,7 +30,7 @@
仅 stdlib。

## 测试
`test_profile_steps.py`(冗余过滤、模块扫描、指令抽取);`test_kb_init_coverage.py`(按深度的模块扫描)。
`test_profile_steps.py`(冗余过滤、模块扫描、指令抽取);`test_kb_init_coverage.py`(按深度的模块扫描);`test_kb_init_skeleton.py`(`doc_files` 与带 `globs` 的语料)。

## 重构备注
纯函数 —— 易测、易复用。冗余过滤器是承载 ETH 研究结论的关键防线;**保持它确定性**。
Expand Down
21 changes: 21 additions & 0 deletions playbooks/kb-init.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# kb init for ONE repository: one stage per run (skeleton, then modules, then
# deepen + lifecycle flip, then harvest-calibration), each opened as a
# human-merged PR (design kb-init v3). Runs outside the knowledge service: it
# never writes the service ledger. Candidate.
name: kb-init
version: 1
status: candidate
provenance:
created_by: human
source: kb init design v3 (bootstrap a repository's knowledge base)
task_kinds: [knowledge_maintenance]
repos: []
requires: []
params:
repo: "knowledge repository name"
stage: "skeleton | modules | deepen | harvest-calibration"
dry_run: "true (default): write the tree and PR body locally; false needs ALLOW_PUSH=1 and ALLOW_POST=1"
pin: "upstream commit to pin (default: the default branch head)"
steps:
- {id: init, step: knowledge.init}
success: the stage's PR is open (or its dry run is written), or the record says why it is blocked
37 changes: 37 additions & 0 deletions src/infermatrix_copilot/engine/steps/knowledge.py
Original file line number Diff line number Diff line change
Expand Up @@ -149,3 +149,40 @@ async def activate_snapshot(ctx: StepContext) -> StepResult:
return StepResult(False, FailureKind.ESCALATE, f"activation refused: {exc}")
return StepResult(True, summary=f"active snapshot {sha}",
outputs={"state_updates": {"kb_active_snapshot": sha, "kb_snapshot_path": str(snapshot)}})


def _init_runtime(ctx: StepContext):
"""kb init's own runtime: never the service's (that opens kb.db), and it
serves disabled repositories — bootstrapping one is the point."""
from pathlib import Path

from ...kb_service.cli import DEFAULT_STATE_DIR
from ...kb_service.init_support import InitRuntime

return InitRuntime.from_env(ctx.settings, state_dir=Path(_STATE_DIR or DEFAULT_STATE_DIR))


@step("knowledge.init", kind="agent", risk="knowledge",
description="kb init: bootstrap one repository's knowledge base, one human-merged stage at a time")
async def init_stage(ctx: StepContext) -> StepResult:
from ...kb_service.init_stages import run_stage
from ...kb_service.init_support import InitError

repo = str(ctx.params.get("repo") or ctx.state.get("kb_repo") or "")
stage = str(ctx.params.get("stage") or "")
dry_run = str(ctx.params.get("dry_run", "true")).lower() not in ("0", "false", "no")
pin = str(ctx.params.get("pin") or "") or None
try:
rt = _init_runtime(ctx)
lifecycle = rt.registry.get(repo)
if lifecycle is None:
return StepResult(False, FailureKind.BLOCKED, f"no adapter declares knowledge repo {repo!r}")
record = run_stage(rt, lifecycle, stage, dry_run=dry_run, pin=pin)
except (InitError, NotImplementedError) as exc:
return StepResult(False, FailureKind.BLOCKED, str(exc))
updates = {"kb_init_stage": stage, "kb_init_status": record.status, "kb_init_pr": dict(record.pr)}
if record.status == "blocked":
return StepResult(False, FailureKind.BLOCKED, "; ".join(record.problems)[:2000],
outputs={"state_updates": updates})
return StepResult(True, summary=f"kb init {stage} for {repo}: {record.status}",
outputs={"state_updates": updates})
45 changes: 45 additions & 0 deletions src/infermatrix_copilot/kb_service/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@
kb traces [--kind K] [--changeset ID] [--rule ID] [--limit N] query trace/1 records
kb replay --record ID --model PROVIDER:MODEL[:EFFORT] re-ask a recorded call
kb export --out FILE [--role judge|generator] dataset (calibration-safe)
kb init REPO --stage STAGE [--dry-run] [--pin SHA]
bootstrap a repository's knowledge base, one
human-merged stage at a time (skeleton, modules,
deepen, harvest-calibration); never touches kb.db
kb init REPO --suggest-seeds rank existing knowledge pages worth seeding from (no model call)
kb publish (--remote HOST:/STATE_DIR | --local DIR) [--once]
the publisher (GPU box, owner's gh login): perform
the signed outbox items; writes need ALLOW_POST=1
Expand Down Expand Up @@ -103,6 +108,36 @@ def _refresh(state_dir: Path, ledger, registry) -> None:
_outbox(state_dir, ledger).transition(lambda: None)


def _init_command(args, state_dir: Path) -> int:
from ..config import Settings

if args.suggest_seeds == bool(args.stage):
print("kb init: pass exactly one of --stage or --suggest-seeds", file=sys.stderr)
return 2
if args.suggest_seeds:
from .init_stages import suggest_seeds
from .init_support import InitError, InitRuntime

try:
rt = InitRuntime.from_env(Settings(), state_dir=state_dir)
lifecycle = rt.registry.get(args.repo)
if lifecycle is None:
print(f"no adapter declares knowledge repo {args.repo!r}", file=sys.stderr)
return 2
for path, score, shared in suggest_seeds(rt, lifecycle):
print(f"{score:6.3f} {path} ({', '.join(shared[:8])})")
except InitError as exc:
print(str(exc), file=sys.stderr)
return 1
return 0
from .runner import run_playbook

params = {"stage": args.stage, "dry_run": "true" if args.dry_run else "false", "pin": args.pin or ""}
outcome, run_dir = run_playbook(Settings(), "kb-init", args.repo, state_dir=state_dir, params=params)
print(f"kb init {args.repo} {args.stage}: {outcome.status} ({run_dir})")
return 0 if outcome.status == "done" else 1


def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="infermatrix-copilot kb")
parser.add_argument("--state-dir", help="default: $KB_STATE_DIR or ~/.infermatrix-copilot/kb")
Expand Down Expand Up @@ -142,6 +177,13 @@ def main(argv: list[str] | None = None) -> int:
export = sub.add_parser("export")
export.add_argument("--out", required=True)
export.add_argument("--role", default="judge", choices=("judge", "generator"))
init = sub.add_parser("init")
init.add_argument("repo")
init.add_argument("--stage", choices=("skeleton", "modules", "deepen", "harvest-calibration"))
init.add_argument("--dry-run", action="store_true",
help="write the tree and PR body under the state directory instead of opening a PR")
init.add_argument("--pin", help="upstream commit to pin (default: the default branch head)")
init.add_argument("--suggest-seeds", action="store_true")
publish = sub.add_parser("publish")
where = publish.add_mutually_exclusive_group(required=True)
where.add_argument("--remote", help="host:/absolute/path of the service state directory (over ssh)")
Expand All @@ -163,6 +205,9 @@ def main(argv: list[str] | None = None) -> int:
return _publish(args)
if args.command in {"traces", "replay", "export"}:
return _traces_command(args, _state_dir(args.state_dir))
if args.command == "init":
# before any ledger is opened: kb init never touches the service's kb.db
return _init_command(args, _state_dir(args.state_dir))

state_dir = _state_dir(args.state_dir)
ledger = _ledger(state_dir)
Expand Down
11 changes: 11 additions & 0 deletions src/infermatrix_copilot/kb_service/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
_INIT_KEYS = {
"seeds", "doc_globs", "source_roots", "exclude", "module_depth", "min_module_loc",
"pr_window", "coverage_target", "budget_usd", "judge_call_usd",
"generator_call_usd", "harness_overhead_bytes",
}
# README* + docs/**/*.md is what profiles/establish.build_doc_corpus reads;
# the contribution and agent guides carry most cross-doc invariants.
Expand Down Expand Up @@ -68,6 +69,10 @@ class InitConfig:
coverage_target: float = 0.85
budget_usd: float = 30.0
judge_call_usd: float = 0.50
# per generator call: the spend threshold handed to the transport, and the
# bytes its own harness adds to the prompt (for the one-request overshoot bound)
generator_call_usd: float = 2.0
harness_overhead_bytes: int = 200_000


@dataclass(frozen=True)
Expand Down Expand Up @@ -176,6 +181,9 @@ def parse_init(section: object, where: str, *, repo: str, manifest: dict) -> Ini
judge_call = _number(data.get("judge_call_usd", 0.50), f"{where}.judge_call_usd")
if judge_call < 0:
raise LifecycleConfigError(f"{where}.judge_call_usd must be >= 0")
generator_call = _number(data.get("generator_call_usd", 2.0), f"{where}.generator_call_usd")
if generator_call <= 0:
raise LifecycleConfigError(f"{where}.generator_call_usd must be > 0")
return InitConfig(
seeds=seeds,
doc_globs=(_strings(data["doc_globs"], f"{where}.doc_globs") if "doc_globs" in data
Expand All @@ -190,6 +198,9 @@ def parse_init(section: object, where: str, *, repo: str, manifest: dict) -> Ini
coverage_target=coverage,
budget_usd=budget,
judge_call_usd=judge_call,
generator_call_usd=generator_call,
harness_overhead_bytes=_int(data.get("harness_overhead_bytes", 200_000),
f"{where}.harness_overhead_bytes", minimum=0),
)


Expand Down
Loading
Loading