Skip to content

sdk(knowledge): carry a bounded diff excerpt per evidence event - #137

Merged
tzhouam merged 1 commit into
mainfrom
fix/knowledge-diff-evidence
Sep 8, 2026
Merged

tzhouam merged 1 commit into
mainfrom
fix/knowledge-diff-evidence

Conversation

@tzhouam

@tzhouam tzhouam commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #136 (base branch fix/knowledge-catalog-capacity); merge that first.

The distillation prompt showed the model only a merged PR's title, up to 3000 characters of body, and changed paths, never the change itself. Across the 68 merged PRs processed so far that yielded one rule.

Change

  • KnowledgeEvidenceEvent gains an optional diff_excerpt: the host's selection of the change's most rule-bearing hunks (the reviewbot will pass non-test, non-doc hunks first).
  • The SDK bounds it in UTF-8 bytes, markers included: 8 KiB per event as a marked prefix cut that never splits a multi-byte character, and 160 KiB per batch spent in event order, with room reserved so every later event still gets a charged "budget exhausted" marker rather than a silent drop. Twenty events stay one model call.
  • The excerpt is fenced with the rest of the evidence (<untrusted_data> escaping applies) and the contract text tells the model to read it for the contract the change fixed or introduced.
  • Defaults are empty and lossless; to_dict() carries the field; non-string values are rejected. No knowledge API version change beyond the 1.1.0 in sdk(knowledge): widen the catalog to owner rule pages with capacity #136.

Validation

  • Full suite passes except the pre-existing environment failure test_phase1_cluster.py::test_drift_guard_unimportable_checked_class_fails_guard (vllm import, fails identically on clean origin/main).
  • New tests: fencing, per-event and per-batch byte bounds with charged markers, multi-byte excerpts, defaults, and type rejection.
  • check_doc_links.py, check_doc_citations.py, check_spec_freshness.py --strict pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Js5CsiwSYjcAYvqti1Lp1Y

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-08T07:18:08.939198Z 1b2458a PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1b2458a5f6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +394 to +395
if spent + len(data) + reserve > _MAX_BATCH_DIFF_BYTES:
data = exhausted

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve first-come ordering after exhausting the batch budget

When more than 20 nonempty excerpts are allowed, replacing an excerpt with the exhaustion marker does not latch the exhausted state, so a smaller later excerpt can still be admitted. For example, after nineteen 8 KiB excerpts, an eighth-kibibyte twentieth excerpt is marked omitted while a one-byte twenty-first excerpt is included. This contradicts the documented first-come behavior and makes the marker falsely claim the budget is exhausted; once an event cannot be admitted, subsequent nonempty events should remain omitted (or the current event should consume the remaining prefix budget).

Useful? React with 👍 / 👎.

@tzhouam
tzhouam force-pushed the fix/knowledge-catalog-capacity branch from 96b1b8f to 2d9f8d0 Compare September 8, 2026 07:41
The distillation prompt showed the model only a merged PR's title, body
and changed paths, never the change itself; one rule per fifty merged
PRs was the result.

KnowledgeEvidenceEvent gains an optional diff_excerpt, the host's
selection of the change's most rule-bearing hunks. The SDK bounds it to
8 KiB per event (a marked prefix cut, never a silent drop) and 160 KiB
per batch, spent in event order so the host sets priority by ordering,
fences it with the rest of the evidence, and tells the model to read it
for the contract the change fixed or introduced. Defaults are empty and
lossless, so existing hosts are unaffected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Js5CsiwSYjcAYvqti1Lp1Y
@tzhouam
tzhouam force-pushed the fix/knowledge-diff-evidence branch from 1b2458a to 02d2c45 Compare September 8, 2026 07:41
@tzhouam
tzhouam changed the base branch from fix/knowledge-catalog-capacity to main September 8, 2026 08:01
@tzhouam
tzhouam merged commit 75d1a48 into main Sep 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant