Skip to content

Publish a summary issue after each release sweep - #219

Merged
tzhouam merged 1 commit into
mainfrom
kb/sweep-report-issue
Sep 28, 2026
Merged

tzhouam merged 1 commit into
mainfrom
kb/sweep-report-issue

Conversation

@tzhouam

@tzhouam tzhouam commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator

Knowledge-service redesign follow-up: a summary after each release sweep (findings, fixes, people's queue, cost), as the design asks.

  • Run identity: each sweep instance gets a run ID (repo, tag, start time), persisted before any work so resumed attempts share it.
  • Durable queueing: a settled sweep queues its report before clearing its progress. The scheduler flushes queued reports every tick and keeps any that fail.
  • Contents: the summary is written to <state_dir>/reports/. It covers:
    • T1 findings and index fixes;
    • pages kept or failed, aggregated across all attempts;
    • change sets by status and operations by kind;
    • the repo's human queue;
    • model calls, tokens and time for that run ID only.
  • Publication: only auto_merge repos with a public upstream also get one issue, via a new outbox kind open_issue. The publisher reuses an existing issue with the same title and retries without labels if one is missing. open_issue is retried like pause/close: never acked as failed, kept until it succeeds or expires after 7 days.
  • IDs: fallback sweeps, which re-check the same tag, get their own report ID and never overwrite or dedupe into an earlier report.

Commit hook approved; full suite and SPEC freshness pass locally (rebased on #218).

🤖 Generated with Claude Code

The design asks for a summary issue after each release sweep: findings,
fixes, people's queue, cost. Sweeps ran, but nothing reported them.

- Every sweep instance gets a run id (repository, tag, start time),
  persisted before any work so each resumed attempt traces under it. A
  settled sweep queues its report durably BEFORE its progress is cleared; the
  scheduler flushes queued reports every tick and keeps any that fail, so a
  failure after the sweep settles cannot lose the report. Model-call cost is
  summed for that run id only, so a fallback sweep never reports the
  release sweep's calls.
- report.publish_summary writes
  <state_dir>/reports/sweep-<repo>-<tag>.md: T1 findings, index fixes,
  pages kept or failed, change sets by status, operations by kind, the
  repository's human queue, and model calls with tokens and time (from the
  trace store). Kept and failed pages are aggregated across every resumed
  attempt through the persisted sweep progress, not just the final call.
- Each report has a durable id, stable across the sweep's resumed attempts:
  a release sweep is its tag, and a periodic fallback sweep (which re-checks
  the same tag) adds its start date. Fallback reports therefore never
  overwrite, or get deduplicated into, an earlier one.
- A repository in auto_merge with a public upstream also gets one issue in
  the knowledge repository, through a new outbox item kind open_issue (valid
  7 days). Shadow and private repositories only record the file.
- The publisher's open_issue reuses an existing issue with the same title
  (one issue per report), and retries without labels if a label is missing,
  so a report is never lost. open_issue is retried like pause and close:
  a transient GitHub failure is never acked as failed, so the item stays in
  the outbox until it succeeds (or expires after 7 days).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-28T16:54:13.985551Z 4b09a1e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4b09a1e953

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +104 to +106
path = rt.state_dir / "reports" / f"sweep-{lifecycle.repo}-{report_id(rt, report)}.md"
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(f"# {title}\n\n{body}", encoding="utf-8")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Encode the release tag before building the report path

For repositories whose valid release tags contain / (for example, release/v1.2), report_id() returns that tag verbatim, so this constructs reports/sweep-demo-release/v1.2.md. Only the top-level reports directory is created, causing write_text() to raise FileNotFoundError; every subsequent scheduler tick retries and fails before creating either the local report or issue. Sanitize or encode the report ID before using it as a filename.

Useful? React with 👍 / 👎.

Comment on lines +31 to +34
publish_summary(rt, lifecycle, pending[0])
done.append(str(pending[0].get("run_id")))
pending = pending[1:]
rt.ledger.set_cursor(lifecycle.repo, PENDING, json.dumps(pending, sort_keys=True, default=str))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep the report pending until its issue is acknowledged

When an open_issue item expires or its generation becomes stale because the repository is paused or rolled back before the publisher handles it, check_item() refuses it without an ack. These lines nevertheless remove the report from the pending queue immediately after merely writing that item, and unlike change-set actions nothing reissues stale or expired report items, so the promised summary issue is permanently lost. Track the item through acknowledgement or retain enough pending state to reissue it.

Useful? React with 👍 / 👎.

Comment on lines +403 to +407
# the report is queued durably BEFORE the progress that produced it is
# cleared; the scheduler flushes it (and retries until it succeeds)
from .report import queue_report

queue_report(rt.ledger, lifecycle.repo, {**report, "run_id": sweep_run_id(lifecycle.repo, sweep, progress)})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Flush reports from the manual sweep path

A sweep can run through the existing kb-sweep playbook's knowledge.sweep step, but that path calls run_sweep() and never calls flush_reports(); only Scheduler._repo_tick() does. Consequently, a successful manual kb run leaves this queued report in the ledger and creates neither the documented local Markdown file nor an issue unless a scheduler daemon happens to run later. Publish or flush through a shared completion path used by both scheduler and manual sweeps.

Useful? React with 👍 / 👎.

@tzhouam
tzhouam merged commit e3bc60d into main Sep 28, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant