Skip to content

Guard release evidence tracker closure state#248

Merged
punk6529 merged 3 commits into
mainfrom
codex/release-evidence-closure-readiness
Jun 13, 2026
Merged

Guard release evidence tracker closure state#248
punk6529 merged 3 commits into
mainfrom
codex/release-evidence-closure-readiness

Conversation

@punk6529

@punk6529 punk6529 commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add scripts/check_release_evidence_issue_closure.py and focused tests for deterministic release evidence tracker closure/readiness validation.
  • Wire the checker into Makefile, shell/PowerShell local gates, GitHub Actions, release-readiness checks, docs, roadmap/run-state traceability, and release artifact metadata.
  • Document optional live GitHub audits from gh issue list --state all --json number,title,state, including premature-closure remediation and UTF-8 BOM snapshot support.

Closes #246.

Validation

  • python scripts/test_release_evidence_issue_closure.py
  • python scripts/check_release_evidence_issue_closure.py
  • python -m py_compile scripts/check_release_evidence_issue_closure.py scripts/test_release_evidence_issue_closure.py
  • python scripts/test_release_evidence_issue_bodies.py
  • python scripts/check_release_evidence_issue_bodies.py
  • python scripts/test_release_readiness.py
  • python scripts/check_release_readiness.py
  • python scripts/generate_release_manifest.py --check
  • python scripts/generate_release_checksums.py --check
  • python scripts/test_changelog_check.py
  • python scripts/check_changelog.py
  • bash -n scripts/check.sh
  • PowerShell parser syntax check for scripts\check.ps1
  • rg -n "^#|^##|^###" ops\ROADMAP.md ops\AUTONOMOUS_RUN.md docs\release-readiness.md docs\public-beta-evidence.md docs\tooling.md release-artifacts\README.md
  • git diff --check
  • Optional live audit: exported gh issue list --repo 6529-Collections/6529Stream --state all --limit 100 --json number,title,state to tmp\release-evidence-issue-closure.json, then ran python scripts/check_release_evidence_issue_closure.py --live-json tmp\release-evidence-issue-closure.json

Summary by CodeRabbit

  • New Features

    • Added deterministic release-evidence issue closure/readiness checks to prevent premature tracker issue closure.
  • Tests

    • Added comprehensive unit and CLI tests covering closure policies, snapshot audit mode, and error/recovery paths.
  • Documentation

    • Updated release procedures, readiness dashboard, tooling, and release-artifact docs to include closure validation and audit steps.
  • Chores

    • Extended CI, local check targets, and release verification pipelines to run the new closure readiness validations.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0d69b3d3-ff8c-4c91-bb86-f3a9bb7c745a

📥 Commits

Reviewing files that changed from the base of the PR and between 7a905e8 and 9968e0f.

📒 Files selected for processing (10)
  • docs/public-beta-evidence.md
  • docs/release-readiness.md
  • docs/tooling.md
  • ops/AUTONOMOUS_RUN.md
  • release-artifacts/README.md
  • release-artifacts/latest/SHA256SUMS
  • release-artifacts/latest/release-checksums.json
  • release-artifacts/latest/release-manifest.json
  • scripts/check_release_evidence_issue_closure.py
  • scripts/test_release_evidence_issue_closure.py
✅ Files skipped from review due to trivial changes (7)
  • release-artifacts/latest/SHA256SUMS
  • release-artifacts/latest/release-checksums.json
  • release-artifacts/README.md
  • docs/release-readiness.md
  • release-artifacts/latest/release-manifest.json
  • docs/public-beta-evidence.md
  • docs/tooling.md
🚧 Files skipped from review as they are similar to previous changes (3)
  • scripts/test_release_evidence_issue_closure.py
  • ops/AUTONOMOUS_RUN.md
  • scripts/check_release_evidence_issue_closure.py

📝 Walkthrough

Walkthrough

Adds a deterministic closure/readiness checker and tests that load and cross-validate committed release-evidence artifacts, optionally audit a live GitHub issue snapshot, and wire the check into CI/Makefile/local scripts, release-readiness enforcement, and documentation.

Changes

Release Evidence Closure Readiness Validator

Layer / File(s) Summary
Core closure validator implementation
scripts/check_release_evidence_issue_closure.py
Loads issue links, backlog, body-sync, packet index, and public-beta evidence; validates shapes, indexes rows by phase:requirement_id, computes per-requirement closure_allowed, and optionally audits a live GitHub snapshot to fail when issues are closed while evidence is non-complete.
Comprehensive test suite
scripts/test_release_evidence_issue_closure.py
Unit and CLI tests that exercise validate_snapshot_closure and CLI --live-json modes, cover happy paths (open blockers, closed when allowed) and failures (premature closure, missing/malformed inputs, duplicate issues), and assert exit codes and stderr diagnostics.
Build/CI orchestration
Makefile, .github/workflows/ci.yml, scripts/check.sh, scripts/check.ps1
Adds release-evidence-issue-closure-check Make target and .PHONY; includes new scripts in CI py_compile and public-beta evidence steps (with dedicated logs); updates shell/PowerShell check runners to execute test and checker sequentially.
Release-readiness & committed checks
scripts/check_release_readiness.py, scripts/test_release_readiness.py, release-artifacts/README.md
Adds release evidence issue closure readiness to required phrases and appends the test/check commands to REQUIRED_COMMANDS; documents closure-check steps in release-artifacts README and committed --check sequences.
Procedural docs & release metadata
docs/public-beta-evidence.md, docs/tooling.md, docs/release-readiness.md, CHANGELOG.md, release-artifacts/latest/*, ops/AUTONOMOUS_RUN.md, ops/ROADMAP.md
Documents the checker and remediation workflow (including --live-json usage), updates docs step numbering, refreshes manifest/checksum entries for changed docs, and records ops/ROADMAP state and Gate G exit criteria referencing issue #246.

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related issues

Possibly Related PRs

Poem

🐰 A rabbit hops through issues with care,

Ensuring evidence waits before we dare,
Scripts and snapshots guard each closing gate,
Reopen commands keep closure deterministic and straight.
Hop on, small checks, keep releases fair!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Guard release evidence tracker closure state' clearly summarizes the main change: adding deterministic validation for release evidence tracker issue closure/readiness state.
Linked Issues check ✅ Passed All coding requirements from issue #246 are met: deterministic closure checker added [check_release_evidence_issue_closure.py], comprehensive tests included [test_release_evidence_issue_closure.py], optional live audit support with remediation guidance implemented, integration into Make/CI/local/docs completed.
Out of Scope Changes check ✅ Passed All changes are directly in-scope: new closure checker and tests, integration into build/CI/docs workflows, manifest/checksum updates reflecting documentation changes, and operational state updates—all aligned with issue #246 objectives.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/release-evidence-closure-readiness

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/public-beta-evidence.md`:
- Around line 86-91: Update the closure-check documentation to mention the
backlog JSON input alongside the other inputs: in docs/public-beta-evidence.md
(lines 86-91) add "release-evidence-issue-backlog.json" to the list of things
the release evidence issue closure checker loads; in docs/release-readiness.md
(lines 41-44) add the backlog JSON to the dashboard note so the dashboard is
described as loading the tracker map, body-sync artifact, packet index, evidence
manifest, and release-evidence-issue-backlog.json; in docs/tooling.md (lines
280-293) add the backlog JSON to the live-audit instructions so the live-audit
step explicitly lists release-evidence-issue-backlog.json as an input; and in
release-artifacts/README.md (lines 185-190) add
release-evidence-issue-backlog.json to the closure-check overview list of
inputs.
- Around line 86-91: The doc text must explicitly list the backlog artifact that
the checker reads: update the sentence describing inputs to include "the
release-evidence-issue-backlog.json backlog artifact" in addition to the
committed tracker map, body-sync artifact, packet index, and public-beta
evidence manifest; reference the checker script name
check_release_evidence_issue_closure.py in the change so readers know this file
loads that backlog JSON before validating closure state.

In `@scripts/check_release_evidence_issue_closure.py`:
- Around line 181-182: The call to body_checker.expected_issue_rows in main()
can raise the body-checker's own exception type and bypass this script's error
normalization; wrap the call in a try/except that catches all exceptions raised
by body_checker.expected_issue_rows and re-raise them as the script's checker
error type (e.g., raise CheckerError(...) from e) so main() and the existing
error-handling path receive a normalized CheckerError; update the block around
body_checker.expected_issue_rows(...) in main() accordingly to perform this
catch-and-wrap.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1cdc1c94-9d2d-4119-aaff-13ebc74349cd

📥 Commits

Reviewing files that changed from the base of the PR and between 17209f7 and 7a905e8.

📒 Files selected for processing (18)
  • .github/workflows/ci.yml
  • CHANGELOG.md
  • Makefile
  • docs/public-beta-evidence.md
  • docs/release-readiness.md
  • docs/tooling.md
  • ops/AUTONOMOUS_RUN.md
  • ops/ROADMAP.md
  • release-artifacts/README.md
  • release-artifacts/latest/SHA256SUMS
  • release-artifacts/latest/release-checksums.json
  • release-artifacts/latest/release-manifest.json
  • scripts/check.ps1
  • scripts/check.sh
  • scripts/check_release_evidence_issue_closure.py
  • scripts/check_release_readiness.py
  • scripts/test_release_evidence_issue_closure.py
  • scripts/test_release_readiness.py

Comment thread docs/public-beta-evidence.md Outdated
Comment thread scripts/check_release_evidence_issue_closure.py Outdated

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Follow-up commit 9968e0f addresses the open review threads:

  • wraps body-sync validation errors into the closure checker error type and adds a CLI regression for malformed body-sync JSON without traceback output;
  • documents release-evidence-issue-backlog.json as an explicit closure-check input in the public beta evidence docs, release readiness dashboard, tooling docs, and release artifacts README;
  • refreshes the release manifest/checksum artifacts and durable run state.

Local validation passed:

  • python scripts/test_release_evidence_issue_closure.py
  • python scripts/check_release_evidence_issue_closure.py
  • python scripts/test_release_readiness.py
  • python scripts/check_release_readiness.py
  • python scripts/generate_release_manifest.py --check
  • python scripts/generate_release_checksums.py --check
  • heading scan
  • git diff --check

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown

@punk6529 Thanks for the detailed follow-up summary. Triggering a review of commit 9968e0f now.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@punk6529 punk6529 merged commit 1c25823 into main Jun 13, 2026
2 checks passed
punk6529 added a commit that referenced this pull request Jun 13, 2026
Record PR #248 merge evidence and update autonomous run state.

Closes #249.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Guard release evidence tracker closure and readiness state

1 participant