Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ jobs:
if not caps.supports_knowledge_curation:
sys.exit("SDK capabilities omit knowledge curation")
if not (
caps.knowledge_api_version == KNOWLEDGE_API_VERSION == "1.0.0"
caps.knowledge_api_version == KNOWLEDGE_API_VERSION == "1.1.0"
):
sys.exit("installed SDK knowledge API version mismatch")
if KnowledgeCurator.proposal_schema(max_rules=3)["properties"][
Expand Down
3 changes: 2 additions & 1 deletion doc/RFC-knowledge-intake.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
## Motivation

The knowledge tree has a written intake contract — PR-learning produces only
executable rules in the nearest owner's `rules.md`, raw material is deleted,
executable rules on the nearest owner's rule pages (its `rules.md` entry page
or a `type: rule` `rules-<topic>.md` topic page), raw material is deleted,
two validators plus the release audit gate every batch — but no automated
feeder. Learnings evaporate at exactly the two moments they are freshest:

Expand Down
2 changes: 1 addition & 1 deletion doc/architecture/SPEC/contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

## 公开契约(`__all__`)
`SDK_API_VERSION` / `DIRECT_API_VERSION` / `STRICT_API_VERSION` /
`QUALITY_API_VERSION` / `KNOWLEDGE_API_VERSION`(当前均为 `"1.0.0"`,且都从 `sdk.v1.models`
`QUALITY_API_VERSION` / `KNOWLEDGE_API_VERSION`(当前前四者为 `"1.0.0"`,Knowledge 为 `"1.1.0"`,且都从 `sdk.v1.models`
取唯一值)。`capabilities(max_strict_workers=1,
supports_file_locking=True) -> dict` 委托 SDK typed handshake 再投影为兼容
dict;它包含 distribution/SDK/Direct/Strict/Quality/Knowledge 版本、resource
Expand Down
15 changes: 10 additions & 5 deletions doc/architecture/SPEC/sdk.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# sdk/ —— 规范

<!-- verified-against: 2026-08-31 -->
<!-- verified-against: 2026-09-08 -->

`Python SDK v1 · 跨仓库唯一 typed 边界 · refactor-status: ok`

Expand Down Expand Up @@ -51,8 +51,12 @@
idempotency key 的重试返回 `created=False` 且绝不再次入队。review depth 只经
policy allowlisted `params` 传入。
- **知识规则由 provider 唯一定义**:catalog 只暴露当前仓库 owner 与 general 的
`rules.md` document ID;prompt 把事件放进唯一 `<untrusted_data>` fence,proposal
shape、rule ID、heading、source citation、目标页和重复 ID 都由 SDK 机械校验。
owner rule page document ID —— 每个 owner 的 `rules.md` 入口页及其
`type: rule` 的 `rules-<topic>.md` 专题页(`catalog_entries()` 附每页
`free_bytes`/`free_lines`,与 `check_knowledge_tree.py` 的拆分门对齐);prompt 把
事件放进唯一 `<untrusted_data>` fence,proposal shape、rule ID、heading、source
citation、目标页、重复 ID 与页面剩余容量都由 SDK 机械校验——超出目标页剩余容量的
proposal 以 `page full` 在 validator 之前被拒绝,宿主据此改路由而不是整批回滚。
`proposal_id` 同时绑定 batch、输入下标、repository、section、sources 与目标页
SHA,宿主不能在 model call 后静默改写已接纳 proposal。
- **知识 apply 是 append-only transaction**:只追加完整 rule section 并更新唯一
Expand All @@ -65,8 +69,9 @@
push、开 PR 或 schedule。ReviewBot 必须向 `KnowledgeCurator` 传 dedicated work
checkout,并继续拥有重试、artifact 与 fork publication;SDK 也绝不写 packaged
knowledge tree。
- SDK、Direct、Strict、Quality、Knowledge API 版本常量均为 `1.0.0`,distribution 为
`0.2.0`;`Capabilities.knowledge_api_version` 与
- SDK、Direct、Strict、Quality API 版本常量均为 `1.0.0`,Knowledge 为 `1.1.0`
(1.1 向后兼容地新增 `catalog_entries()` / `KnowledgeCatalogEntry` 与容量预检),
distribution 为 `0.2.0`;`Capabilities.knowledge_api_version` 与
`supports_knowledge_curation` 组成 ReviewBot 的 paired-release 握手,避免只按
wheel 名称误接缺失知识 API 的 artifact。后者只有完整 apply 所需的 process file
lock 在当前平台可用时才为 true,否则 fail closed;旧 `contract.py` 仅做
Expand Down
3 changes: 2 additions & 1 deletion doc/knowledge/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,8 @@ python knowledge/tools/check_wiki_lint.py
- 新增、移动、重命名、拆分或删除 Markdown 时,必须在同一修改中更新索引和所有链接。
- PR/review 学习只能落可执行规则:完整 diff、评论、thread、commit 和回放输出只在
整理时进入 Git 忽略的临时目录;批次结束必须删除。正式树只更新最近 component 或
model owner 的 `rules.md` 及必要索引,不新增 PR case/history/result/incident 页面。
model owner 的 rule page(`rules.md` 入口页,或该 owner 已拆出的 `type: rule`
`rules-<topic>.md` 专题页)及必要索引,不新增 PR case/history/result/incident 页面。
- 非 PR 学习的既有 guide/architecture 继续按原 owner 维护;不得把不同模型、组件、
benchmark 和 review 内容压成一篇 catch-all 页面。
- 评测 cases、hidden labels、predictions、judgments 和 run reports 属于 `eval/`,
Expand Down
2 changes: 2 additions & 0 deletions src/infermatrix_copilot/sdk/v1/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
IncompatibleVersionError,
InvalidRequestError,
KnowledgeApplyResult,
KnowledgeCatalogEntry,
KnowledgeCurationError,
KnowledgeEvidenceBatch,
KnowledgeEvidenceEvent,
Expand Down Expand Up @@ -59,6 +60,7 @@
"IncompatibleVersionError",
"InvalidRequestError",
"KnowledgeApplyResult",
"KnowledgeCatalogEntry",
"KnowledgeCurationError",
"KnowledgeCurator",
"KnowledgeEvidenceBatch",
Expand Down
145 changes: 130 additions & 15 deletions src/infermatrix_copilot/sdk/v1/knowledge.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
from .models import (
InvalidRequestError,
KnowledgeApplyResult,
KnowledgeCatalogEntry,
KnowledgeCurationError,
KnowledgeEvidenceBatch,
KnowledgeProposalRejection,
Expand All @@ -48,6 +49,15 @@
"knowledge/tools/check_knowledge_tree.py",
"knowledge/tools/check_wiki_lint.py",
)
# Owner rule pages: the entry page plus its split-out topic pages, which the
# tree already treats as rule pages (frontmatter ``type: rule``).
_RULE_PAGE_NAME: Final = re.compile(r"rules(?:-[a-z0-9][a-z0-9-]*)?\.md")
_PAGE_TYPE: Final = re.compile(r"^type:\s*(?P<type>[A-Za-z]+)\s*$", re.MULTILINE)
# Mirrors knowledge/tools/check_knowledge_tree.py's split gate: a page fails
# at >= 32 KiB or >= 500 non-empty lines, so that is the capacity a proposal
# may consume before validation would reject the whole batch.
_PAGE_MAX_BYTES: Final = 32 * 1024
_PAGE_MAX_LINES: Final = 500
_MAX_SECTION_CHARS = 16 * 1024
_MAX_SUMMARY_CHARS = 3000
_MAX_ATTRIBUTE_CHARS = 3000
Expand Down Expand Up @@ -204,35 +214,92 @@ def _document_path(self, document_id: str) -> Path:
)
return path

def _is_rule_page(self, path: Path) -> bool:
"""A rule page is named ``rules.md`` or ``rules-<topic>.md`` AND
declares ``type: rule``; a topic page of another type (a guide that
happens to share the prefix) is never a proposal target."""
if not _RULE_PAGE_NAME.fullmatch(path.name):
return False
try:
head = path.read_text(encoding="utf-8")[:4096]
except OSError as exc:
raise KnowledgeCurationError(
f"knowledge catalog page is unreadable: {path.name}"
) from exc
match = _FRONTMATTER.match(head)
if match is None:
return False
page_type = _PAGE_TYPE.search(match.group("body"))
return page_type is not None and page_type.group("type") == "rule"

def catalog(
self, repository: RepositoryRef | None = None
) -> tuple[str, ...]:
"""Return sorted, relative IDs for allowed ``rules.md`` owner pages."""
"""Return sorted, relative IDs for allowed owner rule pages: each
owner's ``rules.md`` entry page and its ``rules-<topic>.md`` topic
pages (``type: rule``)."""
return tuple(entry.document_id for entry in self.catalog_entries(repository))

def catalog_entries(
self, repository: RepositoryRef | None = None
) -> tuple[KnowledgeCatalogEntry, ...]:
"""The catalog with each page's remaining capacity."""
prefixes = ("knowledge/general/", "knowledge/repos/")
if repository is not None:
slug = self._repo_slug(repository)
prefixes = ("knowledge/general/", f"knowledge/repos/{slug}/")

documents: list[str] = []
for path in sorted(self._knowledge.rglob("rules.md")):
entries: list[KnowledgeCatalogEntry] = []
for path in sorted(self._knowledge.rglob("rules*.md")):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Support topic pages' existing heading levels

When vllm-omni proposals target a newly exposed topic page that stores rules as level-three headings—for example, knowledge/repos/vllm-omni/components/diffusion/rules-platform-kernels.md:15 contains ### DIFF-1t—this expansion is incompatible with the existing _HEADING and _rule_exists logic, which recognize only ##. A proposal matching the page's style is rejected, while a ## DIFF-1t proposal is accepted as new and can be appended beside the existing rule because neither fixed validator checks duplicate rule IDs. Recognize both heading levels or exclude incompatible pages before cataloging them; otherwise automated intake can reject valid proposals or persist duplicate conclusions.

AGENTS.md reference: AGENTS.md:L11-L11

Useful? React with 👍 / 👎.

if path.is_symlink():
raise KnowledgeCurationError(
"knowledge catalog contains a symlinked rules.md"
"knowledge catalog contains a symlinked rules page"
)
if not self._is_rule_page(path):
continue
resolved = path.resolve()
try:
document_id = resolved.relative_to(self._workspace).as_posix()
except ValueError as exc:
raise KnowledgeCurationError(
"knowledge catalog contains an escaping rules.md"
"knowledge catalog contains an escaping rules page"
) from exc
if any(document_id.startswith(prefix) for prefix in prefixes):
documents.append(document_id)
if len(documents) > self.max_catalog_pages:
if not any(document_id.startswith(prefix) for prefix in prefixes):
continue
data = resolved.read_bytes()
entries.append(KnowledgeCatalogEntry(
document_id=document_id,
size_bytes=len(data),
free_bytes=max(0, _PAGE_MAX_BYTES - 1 - len(data)),
free_lines=max(
0, _PAGE_MAX_LINES - 1 - self._non_empty_lines(data)
),
))
if len(entries) > self.max_catalog_pages:
raise KnowledgeCurationError(
"knowledge rules catalog exceeds the configured page bound"
)
return tuple(documents)
return tuple(entries)

@staticmethod
def _non_empty_lines(data: bytes) -> int:
return sum(
1 for line in data.decode("utf-8", "replace").splitlines()
if line.strip()
)

@staticmethod
def _appended_size(section: str, page_data: bytes) -> tuple[int, int]:
"""Bytes and non-empty lines ``apply`` adds for one section: what
``_updated_page`` renders, including the newline it first inserts
when the page does not end with one (the ``updated:`` bump is a
same-length replacement)."""
rendered = "\n" + section.rstrip() + "\n"
normalization = 0 if page_data.endswith((b"\n", b"\r")) else 1
return (
len(rendered.encode("utf-8")) + normalization,
sum(1 for line in rendered.splitlines() if line.strip()),
)

def _validate_batch(self, batch: KnowledgeEvidenceBatch) -> None:
if not batch.batch_id.strip() or len(batch.batch_id) > 128:
Expand Down Expand Up @@ -305,21 +372,35 @@ def _prompt_events(self, batch: KnowledgeEvidenceBatch) -> list[dict[str, Any]]:
def build_prompt(self, batch: KnowledgeEvidenceBatch) -> str:
"""Build the catalog-constrained prompt with evidence fenced as data."""
self._validate_batch(batch)
catalog = self.catalog(batch.repository)
if not catalog:
entries = self.catalog_entries(batch.repository)
if not entries:
raise KnowledgeCurationError(
"knowledge workspace has no rules.md page for this repository"
"knowledge workspace has no rule page for this repository"
)
catalog = [
{
"page": entry.document_id,
"free_bytes": entry.free_bytes,
"free_lines": entry.free_lines,
}
for entry in entries
]
schema = self.proposal_schema(max_rules=batch.max_rules)
return (
"You distill repository-maintenance learnings into a governed "
"knowledge tree. The catalog lists every rules.md page you may "
"target.\n\n"
"knowledge tree. The catalog lists every owner rule page you may "
"target: an owner's `rules.md` entry page and its `rules-<topic>.md` "
"topic pages, each with the room it has left.\n\n"
"Contract (violations are rejected mechanically):\n"
"- Propose only executable rules that change what a reviewer or "
"debugger does next time; no case narration or raw event pages.\n"
"- Route each rule to the nearest owner page in the catalog. Read "
"that page first and match its language, heading, and bullet style.\n"
"- Respect capacity: a page's free_bytes / free_lines are what new "
"sections may add before the page must split. A proposal that "
"needs more than its page has left is rejected, so prefer the "
"owner's topic page that fits, or shorten; never target a page "
"outside the catalog.\n"
"- Each proposal is one complete `## <rule_id> — <title>` section. "
"Never modify or restate an existing section.\n"
f"- Return at most {batch.max_rules} rules. An empty list is correct "
Expand Down Expand Up @@ -384,13 +465,21 @@ def validate_proposals(
if not isinstance(raw_rules, list):
raise InvalidRequestError("proposal output rules must be an array")

catalog = set(self.catalog(batch.repository))
entries = {
entry.document_id: entry
for entry in self.catalog_entries(batch.repository)
}
catalog = set(entries)
allowed_sources = {
event.source_reference.strip() for event in batch.events
}
accepted: list[KnowledgeRuleProposal] = []
rejected: list[KnowledgeProposalRejection] = []
seen: set[tuple[str, str]] = set()
# Room consumed on each page by proposals accepted earlier in this
# same batch, so two rules routed to one nearly full page do not
# both pass here and then fail together at the validator.
consumed: dict[str, tuple[int, int]] = defaultdict(lambda: (0, 0))
for index, item in enumerate(raw_rules):
reason = ""
if index >= batch.max_rules:
Expand Down Expand Up @@ -459,6 +548,32 @@ def validate_proposals(
)
if self._rule_exists(page_text, rule_id):
reason = "rule_id already exists in the target page"
if not reason:
used_bytes, used_lines = consumed[page]
# The normalization newline is paid once per page:
# after the first accepted section the page ends
# with one.
add_bytes, add_lines = self._appended_size(
section,
self._document_path(page).read_bytes()
if (used_bytes, used_lines) == (0, 0) else b"\n",
)
entry = entries[page]
if (
used_bytes + add_bytes > entry.free_bytes
or used_lines + add_lines > entry.free_lines
):
reason = (
"page full: section needs "
f"{add_bytes} bytes / {add_lines} lines but "
f"{entry.free_bytes - used_bytes} bytes / "
f"{entry.free_lines - used_lines} lines remain "
"before the page must split"
)
else:
consumed[page] = (
used_bytes + add_bytes, used_lines + add_lines
)

if reason:
rejected.append(KnowledgeProposalRejection(index, reason))
Expand Down
16 changes: 15 additions & 1 deletion src/infermatrix_copilot/sdk/v1/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
DIRECT_API_VERSION = "1.0.0"
STRICT_API_VERSION = "1.0.0"
QUALITY_API_VERSION = "1.0.0"
KNOWLEDGE_API_VERSION = "1.0.0"
KNOWLEDGE_API_VERSION = "1.1.0"


class _Serializable:
Expand Down Expand Up @@ -84,6 +84,20 @@ class KnowledgeRoute(_Serializable):
read_required: bool


@dataclass(frozen=True)
class KnowledgeCatalogEntry(_Serializable):
"""One owner rule page a proposal may target, with its remaining room.

``free_bytes`` / ``free_lines`` are what a new section may add before the
page crosses the tree validator's split gate; a proposal that needs more
is rejected before validation so the host can reroute it."""

document_id: str
size_bytes: int
free_bytes: int
free_lines: int


@dataclass(frozen=True)
class KnowledgeEvidenceEvent(_Serializable):
"""One host-collected, untrusted fact bundle offered for distillation."""
Expand Down
2 changes: 1 addition & 1 deletion test/test_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ def test_capabilities_reports_the_real_worker_count():
assert caps["supports_expected_head"] is True
assert caps["supports_structured_result"] is True
assert caps["supports_knowledge_curation"] is True
assert contract.KNOWLEDGE_API_VERSION == "1.0.0"
assert contract.KNOWLEDGE_API_VERSION == "1.1.0"


def test_capabilities_reports_missing_file_locking():
Expand Down
Loading
Loading