Add release evidence issue backlog artifact#213
Merged
Conversation
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
Contributor
Author
|
@coderabbitai review |
📝 WalkthroughWalkthroughThis PR adds a deterministic release-evidence issue-backlog generator and tests, wires generator/check into Makefile and CI, records new JSON/Markdown artifacts in the release manifest and checksums, updates release-readiness/tooling/docs, and commits the generated Markdown backlog artifact. ChangesRelease Evidence Issue Backlog Feature
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related issues
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/tooling.md (1)
22-84:⚠️ Potential issue | 🟡 MinorAdd release evidence issue backlog scripts to docs/tooling.md command sequences
docs/tooling.md’s “make check” command list (lines 22-84) omitspython scripts/test_release_evidence_issue_backlog.pyandpython scripts/generate_release_evidence_issue_backlog.py --check, even though the Makefile’schecktarget depends onrelease-evidence-issue-backlog-check.docs/tooling.md’s “Release Artifacts” regeneration commands (lines 291-333) also omit the issue backlog steps (addpython scripts/generate_release_evidence_issue_backlog.pyto the refresh list, andpython scripts/test_release_evidence_issue_backlog.py+python scripts/generate_release_evidence_issue_backlog.py --checkto the--checklist).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/tooling.md` around lines 22 - 84, Summary: docs/tooling.md omits the release evidence issue backlog scripts from the "make check" and "Release Artifacts" command lists; add them. Fix: in the "make check" command sequence add the two commands python scripts/test_release_evidence_issue_backlog.py and python scripts/generate_release_evidence_issue_backlog.py --check (place them with the other python test/generate checks), and in the "Release Artifacts" regeneration commands add python scripts/generate_release_evidence_issue_backlog.py to the refresh list and ensure python scripts/test_release_evidence_issue_backlog.py plus python scripts/generate_release_evidence_issue_backlog.py --check are present in the --check list so the docs match the Makefile dependency.
🧹 Nitpick comments (1)
scripts/test_release_evidence_issue_backlog.py (1)
287-351: ⚡ Quick winAdd a regression test for missing nested row keys to enforce typed failures.
Current negative-path coverage is strong, but it does not verify behavior when required nested keys (for example
template.path) are missing. Add a case assertingReleaseEvidenceIssueBacklogErrorso malformed packet rows don’t regress to uncaughtKeyError.Example test to add
+ def test_rejects_missing_nested_required_fields(self) -> None: + """Missing nested row keys fail with typed backlog error.""" + with tempfile.TemporaryDirectory() as temp_dir: + root = Path(temp_dir) + bad_row = packet_row(generator.PUBLIC_BETA_PHASE, "external_audit_report") + del bad_row["template"]["path"] + write_json(root / generator.DEFAULT_PACKET_INDEX, packet(bad_row)) + + with self.assertRaises(generator.ReleaseEvidenceIssueBacklogError): + generator.build_backlog( + root, + generator.DEFAULT_PACKET_INDEX, + generator.DEFAULT_JSON_OUTPUT, + generator.DEFAULT_MARKDOWN_OUTPUT, + )
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/generate_release_evidence_issue_backlog.py`:
- Around line 154-160: The code currently directly indexes nested packet fields
in completion_gate, issue_body, and build_backlog, which can raise uncaught
KeyError; update each function to validate required top-level and nested fields
with require_string/require_bool/require_dict (and require_dict for nested maps)
before any direct indexing (e.g., replace usages like packet["foo"]["bar"] or
row["baz"] with validated values from require_* calls), and ensure any
missing/invalid fields propagate the library's controlled error type so main
continues to handle ReleaseEvidenceIssueBacklogError rather than letting raw
KeyError escape.
---
Outside diff comments:
In `@docs/tooling.md`:
- Around line 22-84: Summary: docs/tooling.md omits the release evidence issue
backlog scripts from the "make check" and "Release Artifacts" command lists; add
them. Fix: in the "make check" command sequence add the two commands python
scripts/test_release_evidence_issue_backlog.py and python
scripts/generate_release_evidence_issue_backlog.py --check (place them with the
other python test/generate checks), and in the "Release Artifacts" regeneration
commands add python scripts/generate_release_evidence_issue_backlog.py to the
refresh list and ensure python scripts/test_release_evidence_issue_backlog.py
plus python scripts/generate_release_evidence_issue_backlog.py --check are
present in the --check list so the docs match the Makefile dependency.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ad7ca4ec-91dd-474a-aafb-23761b14489f
📒 Files selected for processing (24)
.github/workflows/ci.ymlCHANGELOG.mdMakefiledocs/public-beta-evidence.mddocs/release-readiness.mddocs/tooling.mdops/AUTONOMOUS_RUN.mdops/ROADMAP.mdrelease-artifacts/README.mdrelease-artifacts/latest/SHA256SUMSrelease-artifacts/latest/release-checksums.jsonrelease-artifacts/latest/release-evidence-issue-backlog.jsonrelease-artifacts/latest/release-evidence-issue-backlog.mdrelease-artifacts/latest/release-manifest.jsonscripts/check.ps1scripts/check.shscripts/check_release_readiness.pyscripts/generate_release_artifacts.pyscripts/generate_release_evidence_issue_backlog.pyscripts/generate_release_manifest.pyscripts/test_release_artifacts.pyscripts/test_release_evidence_issue_backlog.pyscripts/test_release_manifest.pyscripts/test_release_readiness.py
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ops/AUTONOMOUS_RUN.md`:
- Around line 174-176: Choose one canonical state (either "pre-push, local ready
to push" or "pushed, CI passed on head") and make the Status snapshot
consistent: if you choose pre-push, change the Status line to something like "PR
`#213` open; local fixes validated and ready to push (commit 9f20c46...);
CI/CodeRabbit pending" and leave the Remote review block as-is; if you choose
the pushed state, change the Status line to "PR `#213` open; pushed; CI passed on
head 9f20c46..." and update the Remote review block to remove or mark
CI/CodeRabbit as passed. Ensure references to PR `#213`, commit
9f20c468eb2f8cce5a42d22080773945fc0cfe46 and the "Remote review" block are
updated accordingly so the two sections do not contradict each other.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: e08cbeac-0871-4445-a8dc-98bcf4b841e8
📒 Files selected for processing (7)
docs/tooling.mdops/AUTONOMOUS_RUN.mdrelease-artifacts/latest/SHA256SUMSrelease-artifacts/latest/release-checksums.jsonrelease-artifacts/latest/release-manifest.jsonscripts/generate_release_evidence_issue_backlog.pyscripts/test_release_evidence_issue_backlog.py
✅ Files skipped from review due to trivial changes (1)
- docs/tooling.md
🚧 Files skipped from review as they are similar to previous changes (3)
- release-artifacts/latest/SHA256SUMS
- scripts/generate_release_evidence_issue_backlog.py
- release-artifacts/latest/release-manifest.json
Comment on lines
+174
to
+176
| Status: PR #213 open; CI passed on head | ||
| `9f20c468eb2f8cce5a42d22080773945fc0cfe46`; CodeRabbit review fix validated | ||
| locally and ready to push. |
There was a problem hiding this comment.
Normalize the PR #213 status snapshot.
This reads as both “CI already passed on 9f20...” and “ready to push,” while the earlier Remote review block still says CI/CodeRabbit are pending. Please make this section consistently describe either the pre-push local state or the pushed CI-passed head so the durable log doesn’t drift.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ops/AUTONOMOUS_RUN.md` around lines 174 - 176, Choose one canonical state
(either "pre-push, local ready to push" or "pushed, CI passed on head") and make
the Status snapshot consistent: if you choose pre-push, change the Status line
to something like "PR `#213` open; local fixes validated and ready to push (commit
9f20c46...); CI/CodeRabbit pending" and leave the Remote review block as-is; if
you choose the pushed state, change the Status line to "PR `#213` open; pushed; CI
passed on head 9f20c46..." and update the Remote review block to remove or mark
CI/CodeRabbit as passed. Ensure references to PR `#213`, commit
9f20c468eb2f8cce5a42d22080773945fc0cfe46 and the "Remote review" block are
updated accordingly so the two sections do not contradict each other.
This was referenced Jun 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
release-artifacts/latest/release-evidence-issue-backlog.jsonand.mdfrom the release evidence packet index, with one no-secret issue-ready entry per incomplete public-beta or production-release evidence requirement.Closes #212.
Validation
python -m py_compile scripts/generate_release_evidence_issue_backlog.py scripts/test_release_evidence_issue_backlog.py scripts/generate_release_manifest.py scripts/test_release_manifest.py scripts/check_release_readiness.py scripts/test_release_artifacts.pypython scripts/test_release_evidence_issue_backlog.pypython scripts/generate_release_evidence_issue_backlog.py --checkpython scripts/test_release_manifest.pypython scripts/generate_release_manifest.py --checkpython scripts/test_release_checksums.pypython scripts/generate_release_checksums.py --checkpython scripts/test_release_artifacts.pypython scripts/generate_release_artifacts.py --checkpython scripts/test_release_readiness.pypython scripts/check_release_readiness.pypython scripts/test_changelog_check.pypython scripts/check_changelog.pybash -n scripts/check.shscripts/check.ps1rg -n "^#|^##|^###" ops/ROADMAP.md ops/AUTONOMOUS_RUN.md docs/release-readiness.md docs/public-beta-evidence.md release-artifacts/README.md docs/tooling.mdgit diff --check(passes with the known Windows LF-to-CRLF warning forscripts/check.ps1)make check(passes with existing Solidity compiler and Foundry trace warning noise only)Summary by CodeRabbit
New Features
Documentation
Chores