Skip to content

Run the release audit in each sweep and draft the adapter-baseline companion - #222

Merged
tzhouam merged 1 commit into
mainfrom
kb/sweep-release-audit
Sep 28, 2026
Merged

tzhouam merged 1 commit into
mainfrom
kb/sweep-release-audit

Conversation

@tzhouam

@tzhouam tzhouam commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator

Knowledge-service redesign follow-up (design §9.1). Each release sweep now runs the repo's release audit, and baseline drift becomes an adapter-baseline companion PR. Sweeps accepted audit hints before, but nothing ran the auditor.

  • Audit: sweep_audit.audit_sweep runs the adapter's release.auditor in generated-baseline mode for the sweep's upstream SHA pair (fallback re-checks with no upstream movement are skipped).
    • It reads one fetched revision of the knowledge repo, materialized via git archive, and the sweep then runs on that same revision.
    • Knowledge-document issues become per-page hints.
    • A failing audit goes to people once per sweep; the sweep runs without hints.
  • Baseline companion: reconciliation findings become one draft companion per sweep.
    • The plugin names its baseline file via a new baseline_file field (vllm-omni: release_baseline.yaml).
    • Only the upstream: and inventories: blocks are replaced; other content and comments are kept, and block boundaries are line-aware.
    • The companion never blocks the sweep: drafting and recovery failures are isolated, traced and sent to people once.
  • The sweep summary reports the reconciliation count.

Commit hook approved; full suite and SPEC freshness pass locally.

🤖 Generated with Claude Code

…mpanion

Sweeps already accepted audit hints, but nothing ran the repository's
release auditor for them. The design also has the service draft the
adapter-baseline update as a companion PR that runs in parallel with the
sweep and never blocks it.

- sweep_audit.audit_sweep runs the adapter's release.auditor plugin
  (generated-baseline mode) for the sweep's upstream SHA pair before the
  sweep; a fallback re-check with no upstream movement is skipped. It reads
  one fetched revision of the knowledge repository, materialized with git
  archive (never a checkout or snapshot that may lag main), and the sweep
  then runs on that same revision. Knowledge
  document issues become per-page hints for the sweep generator. An audit
  that fails goes to people once per sweep, and the sweep runs without hints.
- Reconciliation findings (the committed adapter baseline lags the audited
  upstream) become one companion PR per sweep: the plugin names its baseline
  file (new baseline_file field, relative to the adapter; vllm-omni:
  release_baseline.yaml). Only the upstream: and inventories: blocks are
  replaced with the generated values, so everything else, comments included,
  is kept. A block runs to the next top-level key: a column-0 comment inside
  it does not end it, and comments just before the next key stay with that
  key. It is published as a draft through open_companion_pr and goes to
  people. The sweep never waits for it: the companion uses the sweep's
  revision (no extra fetch), and any failure drafting it is traced and sent
  to people once per sweep while the sweep still runs. The scheduler's
  recovery pass for staged companions is isolated per change set too, so one
  that keeps failing to publish never blocks later ticks.
- The sweep summary reports the number of reconciliation findings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-28T17:53:26.510931Z e9dd619 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@tzhouam
tzhouam merged commit fb61e8f into main Sep 28, 2026
6 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e9dd619948

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 279 to +281
def run_sweep(rt, lifecycle, owner: str, sweep: dict, upstream: UpstreamRepo,
audit_hints: dict[str, list[dict]] | None = None) -> dict:
audit_hints: dict[str, list[dict]] | None = None, reconciliation: int | None = None,
base_sha: str | None = None) -> dict:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Run the audit from the playbook sweep path

The new audit inputs remain optional here, so the documented playbooks/kb-sweep.yaml path still reaches engine/steps/knowledge.py::sweep, which calls run_sweep(...) without running audit_sweep or staging the baseline companion. Consequently, invoking kb run rather than the scheduler silently performs a release sweep with no audit hints, no reconciliation report, and no baseline PR; move the audit orchestration into the shared sweep path or update every entry point.

Useful? React with 👍 / 👎.

committed = rt.knowledge.show(main, path)
if committed is None:
return None
updated = baseline_text(committed, result.generated_baseline)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Handle reconciliation that baseline blocks cannot fix

When the audit reports any reconciliation, this always creates a PR that changes only upstream and inventories. The vLLM-Omni auditor also classifies unmatched_path, suspicious_path_route, stale_pin, and other routing/policy findings as reconciliation, none of which this edit resolves; in particular, a changed unmatched path can disappear from future audits once the sweep baseline advances. These findings need separate remediation or escalation rather than being treated as fixed by the baseline companion.

Useful? React with 👍 / 👎.

Comment on lines +51 to +55
hints: dict[str, list[dict]] = {}
for issue in result.issues:
page = str(issue.get("document") or "").removeprefix("knowledge/")
if page:
hints.setdefault(page, []).append(issue)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Escalate audit issues with no sweepable page

Audit issues are only converted to hints, but some valid enforced issues cannot be consumed by the sweep: owner_document_missing may omit document entirely, or name a document that is absent from the fetched tree. The former is dropped here and the latter is keyed to a path that rule_pages(base, ...) never iterates, so the sweep completes and advances its baseline without fixing or notifying people about the failed audit condition. Unmappable or nonexistent-page issues should be explicitly escalated.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant