Skip to content

Add release evidence issue label drift checks#240

Merged
punk6529 merged 3 commits into
mainfrom
codex/release-evidence-label-drift
Jun 13, 2026
Merged

Add release evidence issue label drift checks#240
punk6529 merged 3 commits into
mainfrom
codex/release-evidence-label-drift

Conversation

@punk6529

@punk6529 punk6529 commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a deterministic no-network release evidence issue label checker for committed applied_labels posture.
  • Add optional --live-json audit mode for exported gh issue list --json number,title,labels snapshots, including remediation commands for missing labels.
  • Wire the deterministic checker into Makefile, CI, shell/PowerShell check wrappers, release-readiness gates, docs, roadmap state, and regenerated release manifest/checksums.

Validation

  • python scripts/test_release_evidence_issue_labels.py
  • python scripts/check_release_evidence_issue_labels.py
  • python scripts/test_release_evidence_issue_body_sync.py
  • python scripts/generate_release_evidence_issue_body_sync.py --check
  • python scripts/check_release_readiness.py
  • python scripts/test_release_readiness.py
  • python scripts/test_release_manifest.py
  • python scripts/generate_release_manifest.py --check
  • python scripts/test_release_checksums.py
  • python scripts/generate_release_checksums.py --check
  • python scripts/check_changelog.py
  • bash -n scripts/check.sh
  • PowerShell parser check for scripts/check.ps1
  • rg -n "^#|^##|^###" docs/tooling.md docs/public-beta-evidence.md docs/release-readiness.md release-artifacts/README.md ops/ROADMAP.md ops/AUTONOMOUS_RUN.md
  • git diff --check
  • Live GitHub label snapshot audit with python scripts/check_release_evidence_issue_labels.py --live-json tmp/release-evidence-issue-labels.json

Closes #239

Summary by CodeRabbit

  • New Features

    • Added deterministic label-drift checks for release evidence issues and an optional live snapshot audit mode to detect label inconsistencies.
  • Chores

    • CI and local check targets updated to run the new label validation as part of release readiness.
  • Documentation

    • Updated release, tooling, and evidence docs and release artifacts to document and reflect label validation and audit workflows.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 49a6e824-56c8-4237-b150-60dd461a4f23

📥 Commits

Reviewing files that changed from the base of the PR and between 5a33b9f and e7e29f2.

📒 Files selected for processing (2)
  • ops/AUTONOMOUS_RUN.md
  • ops/ROADMAP.md
✅ Files skipped from review due to trivial changes (2)
  • ops/AUTONOMOUS_RUN.md
  • ops/ROADMAP.md

📝 Walkthrough

Walkthrough

This PR implements deterministic release evidence tracker issue label validation with optional live GitHub snapshot audit mode. It adds a new checker script, comprehensive tests, CI/local check integration, documentation updates, and refreshes related release artifact metadata and operational state.

Changes

Release Evidence Label Drift Checking

Layer / File(s) Summary
Label validation checker implementation
scripts/check_release_evidence_issue_labels.py
Core checker loads issue-links and backlog JSON, validates that applied labels are non-empty, unique, and a subset of suggested labels. Optional snapshot mode verifies each linked GitHub issue includes all expected labels and generates gh issue edit --add-label remediation commands for any missing labels.
Label validation test suite
scripts/test_release_evidence_issue_labels.py
Tests cover committed label validation, snapshot audit mode with live JSON, duplicate label rejection, applied labels not in suggested set rejection, missing issues in snapshot, missing snapshot labels with remediation text, title mismatches, and CLI behavior with explicit snapshot paths and drift detection without traceback.
CI and local check pipeline wiring
.github/workflows/ci.yml, scripts/check.sh, scripts/check.ps1, Makefile, scripts/check_release_readiness.py
Label checker and test script integrated into CI workflow job steps, shell/PowerShell check scripts, and Makefile dependency graph with release-evidence-issue-labels-check target; Makefile reorders label-check to precede body-sync steps; release-readiness REQUIRED_COMMANDS updated.
User-facing documentation and operational guidance
docs/public-beta-evidence.md, docs/release-readiness.md, docs/tooling.md, release-artifacts/README.md, CHANGELOG.md
Public-beta-evidence procedure expands steps 13–15 with clearer label check command presentation and gh issue list snapshot export workflow; release-readiness and tooling docs document --live-json snapshot audit mode; release-artifacts README clarifies deterministic label validation; CHANGELOG entry added.
Operational state, roadmap, and artifact checksums
ops/AUTONOMOUS_RUN.md, ops/ROADMAP.md, release-artifacts/latest/*
Autonomous-run worklog records Queue Item 114 (PR #240) completion with branch/issue linkage and validation command summary; roadmap metadata refreshed with new verification timestamp and test matrix entry for label drift checks; manifest/checksum artifacts updated to reflect documentation changes.

Sequence Diagram

sequenceDiagram
  participant CLI
  participant IssueLinksJSON
  participant BacklogJSON
  participant SnapshotJSON
  CLI->>IssueLinksJSON: load and validate issue links
  CLI->>BacklogJSON: load and validate backlog
  CLI->>SnapshotJSON: optionally load --live-json snapshot
  CLI->>CLI: compute missing labels and title mismatches
  CLI->>CLI: assemble gh issue edit --add-label remediation commands
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

  • #239: Harden release evidence tracker label drift checks — This PR implements the deterministic checker, optional live snapshot audit, tests, CI wiring, and docs requested by issue #239.

Possibly related PRs

Poem

🐰 I hop through JSON, labels in tow,
I check what's written and what won't show,
If duplicates sneak or labels fall shy,
I'll print a fix command — give it a try!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Add release evidence issue label drift checks' accurately and specifically describes the main change: adding new deterministic checks for release evidence tracker label drift.
Linked Issues check ✅ Passed The pull request comprehensively implements all scope objectives from #239: deterministic label validation, optional --live-json audit mode, tests covering happy path and error cases, integration into CI/local gates, and documentation updates.
Out of Scope Changes check ✅ Passed All changes are scope-aligned: two new validation scripts, integration into build/CI workflows, documentation updates, and manifest regeneration. No unrelated changes to issue closure or evidence claims.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/release-evidence-label-drift

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
ops/AUTONOMOUS_RUN.md (1)

177-224: ⚡ Quick win

Record the PR #240 transition in the decision log.

The current worklog says PR #240 is open, but the durable decision log still stops at PR #238. Please append the matching state-transition entry so resume/compaction has one authoritative history.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ops/AUTONOMOUS_RUN.md` around lines 177 - 224, Append a durable
state-transition entry to the decision log that mirrors the format used for
prior entries and records Queue Item 114 / PR `#240`: include "PR candidate:
Harden release evidence tracker label drift checks (Queue Item 114)", Status: PR
`#240` open, PR: https://github.com/6529-Collections/6529Stream/pull/240, Branch:
codex/release-evidence-label-drift, Branch started from PR `#238` squash merge
commit 3c738f51c8fa2cf623fda1f3d1fe5284db946d99, and include the prior queue
transition notes (Queue Item 113 merged in PR `#238` as squash commit
3c738f51c8fa2cf623fda1f3d1fe5284db946d99, PR `#238` final implementation head
d56f7d2405fd0c52fdf2320a0f167253aceb4bf7) plus the Completed local validation
checklist from the diff so the durable log and worklog are consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ops/ROADMAP.md`:
- Around line 95-100: Update the provenance lines in the verification metadata
table: replace references to the prior baseline PR (`#238`), its CI run id
(`27468501813`), head commit (`d56f7d2405fd0c52fdf2320a0f167253aceb4bf7`) and
merged baseline commit (`3c738f51c8fa2cf623fda1f3d1fe5284db946d99`) with the
current label-drift PR/run/commit that actually performed the label-drift audit;
specifically edit the "Last verified", "OS tested" and "CI run" rows so the PR
number, CI run number and commit SHAs reflect the label-drift change set (use
the actual current PR number, run id, and commit SHA when applying the change).

---

Nitpick comments:
In `@ops/AUTONOMOUS_RUN.md`:
- Around line 177-224: Append a durable state-transition entry to the decision
log that mirrors the format used for prior entries and records Queue Item 114 /
PR `#240`: include "PR candidate: Harden release evidence tracker label drift
checks (Queue Item 114)", Status: PR `#240` open, PR:
https://github.com/6529-Collections/6529Stream/pull/240, Branch:
codex/release-evidence-label-drift, Branch started from PR `#238` squash merge
commit 3c738f51c8fa2cf623fda1f3d1fe5284db946d99, and include the prior queue
transition notes (Queue Item 113 merged in PR `#238` as squash commit
3c738f51c8fa2cf623fda1f3d1fe5284db946d99, PR `#238` final implementation head
d56f7d2405fd0c52fdf2320a0f167253aceb4bf7) plus the Completed local validation
checklist from the diff so the durable log and worklog are consistent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e3b20730-1460-4f46-b7aa-4014b038551e

📥 Commits

Reviewing files that changed from the base of the PR and between 3c738f5 and 5a33b9f.

📒 Files selected for processing (17)
  • .github/workflows/ci.yml
  • CHANGELOG.md
  • Makefile
  • docs/public-beta-evidence.md
  • docs/release-readiness.md
  • docs/tooling.md
  • ops/AUTONOMOUS_RUN.md
  • ops/ROADMAP.md
  • release-artifacts/README.md
  • release-artifacts/latest/SHA256SUMS
  • release-artifacts/latest/release-checksums.json
  • release-artifacts/latest/release-manifest.json
  • scripts/check.ps1
  • scripts/check.sh
  • scripts/check_release_evidence_issue_labels.py
  • scripts/check_release_readiness.py
  • scripts/test_release_evidence_issue_labels.py

Comment thread ops/ROADMAP.md Outdated
@punk6529 punk6529 merged commit a9e8bcc into main Jun 13, 2026
2 checks passed
punk6529 added a commit that referenced this pull request Jun 13, 2026
Record PR #240 merge evidence and refresh durable roadmap state.

Closes #241.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Harden release evidence tracker label drift checks

1 participant