Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions doc/architecture/SPEC/knowledge_service.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ library only.
## 2026-09-28 T1 索引修复
L1 的索引检查允许为目录中已存在但未列入索引的页面补链接(巡检 T1 修复),新页面仍必须被链接。

## 2026-09-30 kb init 的 L1 bootstrap 模式
`l1.check_changeset(..., bootstrap=)` 只供 `kb init`(`kb_service/init_stages.py`):允许在新目录新建
`_index.md`、修改共享的 `repos/_index.md`;`l1.check_index_links(base, head)` 检查 init 改动的索引链接。
细节见 [l1](knowledge_service/l1.md)。服务门禁与发布器从不传 `bootstrap`。

## 2026-09-28 知识验证器(v8:发布器本地门禁)
`gate_verifier.verify_change(ctx, pr=, head_sha=, pre=, post=, final=, effective_base=, verdict=)` 是发布器本地门禁
(`kb_service.local_gate`)在合并前与合并后运行的检查;v7 的仓库端 `kb-gate` 工作流、`.github/kb-gate/` 验证包、
Expand Down
31 changes: 28 additions & 3 deletions doc/architecture/SPEC/knowledge_service/l1.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# knowledge_service/l1.py —— 规范

<!-- verified-against: 2026-09-29 -->
<!-- verified-against: 2026-09-30 -->

`LOC ~420 · 知识质量门的确定性一层(L1) · refactor-status: stable`
`LOC ~600 · 知识质量门的确定性一层(L1) · refactor-status: stable`

## 职责
- `check_tree(files)`:尾注合法、supersede 双向一致、tombstone ID 未被复用、
Expand All @@ -24,11 +24,36 @@
同目录 `_index.md` 出现。
- purge 时序:tombstone 的 `purged_at` 不得等于该规则的 `retired_at`;传入可信的
`release` 时 `purged_at` 必须等于它。
- `check_changeset(..., bootstrap=)`:仅供 `kb init`(唯一允许的调用方是
`kb_service/init_stages.py`,测试扫描源码钉住)。只放宽两条:
- 新目录(base 中该目录下没有任何文件)可以**新建** `_index.md`,产出一个 prose 块;新索引必须
链接本目录的每个页面和每个子目录的 `_index.md`,已存在的父目录索引必须链接它
(父索引也是本次新建时由父索引自己报告)。在已有目录新建索引、删除索引仍然拒绝。
- 共享的 `knowledge/repos/_index.md` 可以被**修改**(登记新仓库),新增/删除仍拒绝。
其余规则(尾注、frontmatter、`sources`、白名单其余部分、保护规则)不变;默认 `False`
时行为与原来逐字节一致。
- `check_index_links(base, head)`:init 侧的索引链接检查(索引改成表格或跨目录链接时
L1 只把它当 prose 块)。**链接 = 渲染器实际生成的链接**:用 markdown-it-py 的 CommonMark 解析器
(开启 GitHub 管道表格)取所有 `link_open` 的 href(图片及其 alt 文本里的不算);先按 URL 拆分(编码的 `#`、`:` 属于文件名),
再只对路径做百分号解码;引用定义、标题、转义、代码段/代码块、
HTML 块/注释都由解析器按渲染器的方式处理,不手写匹配。URL、纯锚点不算目标。
对每个新增或改动的 `_index.md`:
- 本次新增的链接必须解析到 head 中的文件或目录(知识树根目录本身,如 `../..`,算已存在的目录)、不得逃出知识树(`index_link_broken` /
`index_link_escapes`,绝对路径保留各自的 `/…` 身份且一律算逃逸;base 已有的链接不重新判定——
有些指向 `doc/`)。
- 保留:base 的链接(按解析后的路径)必须仍在 head 的链接里,否则 `index_link_dropped`——把链接挪进
代码、注释或 HTML 块 = 丢链接。
- 覆盖:head 中每个新页面、新子目录索引都必须被父索引链接(`index_unlinked`,知识树根没有索引);
bootstrap 新建索引的 `index_missing` 用同一套链接。
在真实知识树的 142 个索引上逐一追加一行、逐一新增一个被链接的页面:0 误报、0 漏判。

## 不变量
纯函数;标准库 + PyYAML + `lifecycle`/`ops`,验证包原样携带。段比较忽略尾部空行
纯函数;标准库 + PyYAML + `lifecycle`/`ops`(索引链接检查另用 markdown-it-py,延迟导入,
`pyproject` 显式声明),验证包原样携带。段比较忽略尾部空行
(追加段落时前一段多出的分隔空行不算改动)。

## 测试
`test_kb_init_l1.py`(bootstrap 新目录索引、父子链接、已有目录仍拒绝、删除拒绝、`repos/_index.md`
只允许修改、链接检查各类 issue、仅 `init_stages.py` 可传 `bootstrap=True`);
`test_knowledge_ops_l1.py`(伪造的 sources/frontmatter/删除 active 规则/重复 ID、白名单与
模式、prose 分块、外部引用、真实知识树上执行各类操作后两个既有校验器均通过)。
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ dependencies = [
"anthropic>=0.34",
"openai>=1.50",
"rich>=13.0",
"markdown-it-py>=3",
]

[project.optional-dependencies]
Expand Down
159 changes: 152 additions & 7 deletions src/infermatrix_copilot/knowledge_service/l1.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,28 @@
pages, tombstones. Recomputed here; a mismatch is an issue,
never a block a verdict could cover.

Standard library + PyYAML + ``lifecycle``/``ops`` only.
``check_changeset`` with ``bootstrap`` set is for ``kb init`` only (the one
allowed caller is ``kb_service/init_stages.py``; a test pins that). It relaxes
exactly two rules so a repository's knowledge can be created: an ``_index.md``
may be created in a directory that had no files before, and the shared
``repos/_index.md`` may be edited to list a new repository.
``check_index_links(base, head)`` is the init-side link check for the index
edits that bootstrap produces.

Standard library + PyYAML + ``lifecycle``/``ops``; ``check_index_links`` (and the
bootstrap index check) also uses markdown-it-py, imported lazily.
"""

from __future__ import annotations

import hashlib
import json
import posixpath
import re
from dataclasses import dataclass
from pathlib import PurePosixPath
from typing import Iterable, Mapping
from urllib.parse import unquote, urlsplit

import yaml

Expand All @@ -48,6 +59,8 @@
_DENY = re.compile(r"knowledge/(?:skills|tools|\.claude)/|knowledge/[^/]+$")
_REGULAR_MODE = "100644"
_INDEX_LINE = re.compile(r"- \[[^\]\n]+\]\((?P<file>[^)/\s]+\.md)\)")
# bootstrap only: the shared list of repositories, edited to link a new one
_REPOS_INDEX = "knowledge/repos/_index.md"


@dataclass(frozen=True)
Expand Down Expand Up @@ -188,11 +201,13 @@ def _check_routes(path: str, text: str, files: Mapping[str, str]) -> list[Issue]

# -- change-set level ----------------------------------------------------------

def _whitelist_issues(changes: Iterable[Change]) -> list[Issue]:
def _whitelist_issues(changes: Iterable[Change], bootstrap: bool = False) -> list[Issue]:
issues = []
for change in changes:
path = change.path
if not path.startswith(KNOWLEDGE_PREFIX) or _DENY.match(path) \
if bootstrap and path == _REPOS_INDEX and change.status == "M":
pass # a new repository is listed in the shared index
elif not path.startswith(KNOWLEDGE_PREFIX) or _DENY.match(path) \
or not any(p.fullmatch(path) for p in _WHITELIST):
issues.append(Issue("path_not_whitelisted", path, "only knowledge rule/guide pages, _routes.yaml and _tombstones.yaml auto-merge"))
continue
Expand Down Expand Up @@ -327,7 +342,11 @@ def _classify_page(path: str, base_text: str | None, head_text: str | None,

def _index_mechanical(path: str, base_text: str | None, head_text: str | None,
new_pages: set[str], issues: list[Issue], blocks: list[Block],
head: Mapping[str, str] | None = None) -> None:
head: Mapping[str, str] | None = None, *,
base: Mapping[str, str] | None = None, bootstrap: bool = False) -> None:
if bootstrap and base_text is None and head_text is not None and head is not None:
_index_created(path, base or {}, head, issues, blocks)
return
if base_text is None or head_text is None:
issues.append(Issue("index_added_or_deleted", path, "index files are never created or deleted by a change set"))
return
Expand Down Expand Up @@ -361,23 +380,148 @@ def _index_mechanical(path: str, base_text: str | None, head_text: str | None,
f"index lines must cover new pages {expected_new} and only link unlisted pages; got {files}"))


_MARKDOWN = None


def _links(text: str) -> list[str]:
"""The link destinations a CommonMark renderer (with GitHub's pipe tables)
makes of ``text``, as written (still percent-encoded). The parser settles
what a hand-written matcher cannot: reference definitions, titles, escapes,
code spans and blocks, HTML blocks and comments. Images are not links, and
neither is anything in an image's alt text."""
global _MARKDOWN
if _MARKDOWN is None:
from markdown_it import MarkdownIt # lazy: only index checks need it

_MARKDOWN = MarkdownIt("commonmark").enable("table")
out: list[str] = []

def walk(tokens) -> None:
for token in tokens:
if token.type == "link_open":
out.append(str(token.attrGet("href") or ""))
if token.children and token.type != "image":
walk(token.children)

walk(_MARKDOWN.parse(text))
return out


def _link_targets(directory: str, text: str) -> dict[str, str]:
"""Relative links of an index: {destination without anchor: resolved
knowledge-relative path}. URLs and anchor-only links are not targets; a
link that leaves the knowledge tree resolves to a path starting ``..``,
and an absolute one keeps its own normalised ``/...`` path (both escape)."""
out: dict[str, str] = {}
for href in _links(text):
parts = urlsplit(href) # split first: an encoded "#" or ":" is part of the name
if parts.scheme or parts.netloc or not parts.path:
continue
target = unquote(parts.path)
out[target] = posixpath.normpath(target) if target.startswith("/") else \
posixpath.normpath(posixpath.join(directory, target))
return out


def _dir_entries(head: Mapping[str, str], directory: str) -> list[str]:
"""Pages directly in ``directory`` and the indexes of its child directories."""
prefix = f"{directory}/" if directory else ""
out = []
for p in sorted(head):
if not p.startswith(prefix) or not p.endswith(".md"):
continue
rest = p[len(prefix):].split("/")
if (len(rest) == 1 and rest[0] != INDEX_NAME) or (len(rest) == 2 and rest[1] == INDEX_NAME):
out.append(p)
return out


def _index_created(path: str, base: Mapping[str, str], head: Mapping[str, str],
issues: list[Issue], blocks: list[Block]) -> None:
"""Bootstrap: an index created for a directory that is new in this change set."""
directory = posixpath.dirname(path)
if any(p.startswith(f"{directory}/") for p in base):
issues.append(Issue("index_added_or_deleted", path,
"an index may be created only for a new directory"))
return
linked = set(_link_targets(directory, head[path]).values())
for entry in _dir_entries(head, directory):
if entry not in linked:
issues.append(Issue("index_missing", path, f"{entry} is not linked"))
if directory:
parent = posixpath.join(posixpath.dirname(directory), INDEX_NAME)
# a parent index created in the same change set reports its own children
if parent in base and parent in head and path not in set(
_link_targets(posixpath.dirname(parent), head[parent]).values()):
issues.append(Issue("index_missing", parent, f"the new {path} is not linked"))
blocks.append(Block("prose", path, "", "prose", _sha(head[path])))


def check_index_links(base: Mapping[str, str], head: Mapping[str, str]) -> list[Issue]:
"""Links of every index added or changed from ``base`` to ``head``: each
link the change adds resolves inside the knowledge tree to a file (or a
directory) of ``head``; a changed index keeps every link it had; every
page or child index new in ``head`` is linked from its parent index.
Links are what a CommonMark renderer makes (``_links``): text in code, a
comment or an HTML block is not a link, so moving a link there drops it.
Links an index already had are not re-judged (some point into doc/)."""
issues: list[Issue] = []
head_dirs = {posixpath.dirname(p) for p in head}
for path in sorted(p for p in head if posixpath.basename(p) == INDEX_NAME):
if base.get(path) == head[path]:
continue
directory = posixpath.dirname(path)
before = _link_targets(directory, base.get(path) or "")
after = _link_targets(directory, head[path])
kept = set(after.values())
for written, resolved in sorted(before.items()):
if resolved not in kept:
issues.append(Issue("index_link_dropped", path, written))
had = set(before.values())
for written, resolved in sorted(after.items()):
if resolved in had:
continue
if resolved == ".." or resolved.startswith(("../", "/")):
issues.append(Issue("index_link_escapes", path, written))
elif resolved in (".", ""):
continue # the knowledge root itself: a directory every tree has
elif resolved not in head and not any(d == resolved or d.startswith(resolved + "/")
for d in head_dirs):
issues.append(Issue("index_link_broken", path, written))
for page in sorted(p for p in head if p not in base and p.endswith(".md")):
directory = posixpath.dirname(page)
if posixpath.basename(page) == INDEX_NAME:
if not directory:
continue
directory = posixpath.dirname(directory)
parent = posixpath.join(directory, INDEX_NAME)
if parent not in head:
if directory: # the knowledge root has no index
issues.append(Issue("index_unlinked", page, f"{parent} does not exist"))
continue
if page not in set(_link_targets(directory, head[parent]).values()):
issues.append(Issue("index_unlinked", page, f"{parent} does not link it"))
return issues


def check_changeset(
base: Mapping[str, str],
head: Mapping[str, str],
changes: Iterable[Change],
*,
external_texts: Mapping[str, str] | None = None,
release: str = "",
bootstrap: bool = False,
) -> ChangesetResult:
"""Gate one change set. ``base``/``head`` map knowledge-relative paths
(``repos/...``) to text; ``changes`` are repository-relative git changes.
``external_texts`` (repo path -> text, outside ``knowledge/``) is scanned
for references to rules this change retires or purges. ``release`` is the
trusted current release (from the signed verdict); when given, purges must
carry it as ``purged_at``. A purge is always rejected in the release that
retired the rule."""
retired the rule. ``bootstrap`` is for ``kb init`` only (module docstring)."""
changes = list(changes)
issues = _whitelist_issues(changes)
issues = _whitelist_issues(changes, bootstrap)
blocks: list[Block] = []
retired: list[str] = []
purged: list[tuple[str, str]] = [] # (rule id, page)
Expand All @@ -400,7 +544,8 @@ def check_changeset(
for path in sorted(knowledge_paths):
name = PurePosixPath(path).name
if name == INDEX_NAME:
_index_mechanical(path, base.get(path), head.get(path), new_pages, issues, blocks, head)
_index_mechanical(path, base.get(path), head.get(path), new_pages, issues, blocks, head,
base=base, bootstrap=bootstrap)
elif name == TOMBSTONES_NAME:
tombstone_paths.append(path)
elif name == ROUTES_NAME:
Expand Down
Loading
Loading