Skip to content

Add release evidence packet index#209

Merged
punk6529 merged 3 commits into
mainfrom
codex/release-evidence-packet-index
Jun 13, 2026
Merged

Add release evidence packet index#209
punk6529 merged 3 commits into
mainfrom
codex/release-evidence-packet-index

Conversation

@punk6529

@punk6529 punk6529 commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a deterministic no-secret release evidence packet index generator and focused tests.
  • Commit JSON and Markdown packet outputs mapping every public-beta and production-release evidence row to blocker report rows, checked templates, retained-artifact expectations, validation commands, and current readiness posture.
  • Wire packet checks into Make, Unix/PowerShell wrappers, CI, release manifest/checksum coverage, release readiness, public-beta evidence docs, release artifacts docs, tooling docs, roadmap, changelog, and autonomous run state.

Closes #207.

Readiness posture

This does not change Solidity, deployment behavior, evidence contents, or public-beta/production readiness claims. Template-only evidence remains preparation material and cannot complete a requirement.

Validation

  • python -m py_compile scripts/generate_release_evidence_packet_index.py scripts/test_release_evidence_packet_index.py scripts/generate_release_manifest.py scripts/test_release_manifest.py scripts/generate_release_artifacts.py scripts/test_release_artifacts.py scripts/check_release_readiness.py
  • python scripts/test_release_evidence_packet_index.py
  • python scripts/generate_release_evidence_packet_index.py --check
  • python scripts/test_public_beta_evidence.py
  • python scripts/check_public_beta_evidence.py
  • python scripts/test_public_beta_blocker_report.py
  • python scripts/generate_public_beta_blocker_report.py --check
  • python scripts/test_production_release_blocker_report.py
  • python scripts/generate_production_release_blocker_report.py --check
  • python scripts/test_non_local_release_evidence.py
  • python scripts/check_non_local_release_evidence.py
  • python scripts/test_release_artifacts.py
  • python scripts/generate_release_artifacts.py --check
  • python scripts/test_release_manifest.py
  • python scripts/generate_release_manifest.py --check
  • python scripts/test_release_checksums.py
  • python scripts/generate_release_checksums.py --check
  • python scripts/test_release_readiness.py
  • python scripts/check_release_readiness.py
  • python scripts/test_changelog_check.py
  • python scripts/check_changelog.py
  • bash -n scripts/check.sh
  • PowerShell parser check for scripts/check.ps1
  • rg -n "^#|^##|^###" ops/ROADMAP.md ops/AUTONOMOUS_RUN.md docs/public-beta-evidence.md docs/release-readiness.md docs/tooling.md release-artifacts/README.md
  • git diff --check passed with existing CRLF-normalization warning for scripts/check.ps1
  • make check passed with existing warning noise only

Summary by CodeRabbit

  • New Features

    • Added a generated "release evidence packet index" in JSON and Markdown that maps release requirements to templates, retained-artifact expectations, validation commands, and readiness posture.
  • Tests

    • Added a comprehensive test suite validating generation, determinism, check-mode drift detection, and error cases for the packet index.
  • Documentation

    • Updated release, tooling, and runbook docs with commands and guidance to generate and verify the packet index.
  • Chores

    • Integrated packet index generation and verification into CI, build targets, and release artifact manifests.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 16f054b9-bc7c-44bd-957e-93b14ff821f4

📥 Commits

Reviewing files that changed from the base of the PR and between 8e2d9af and e571a2a.

📒 Files selected for processing (8)
  • .github/workflows/ci.yml
  • docs/public-beta-evidence.md
  • ops/AUTONOMOUS_RUN.md
  • ops/ROADMAP.md
  • release-artifacts/latest/SHA256SUMS
  • release-artifacts/latest/release-checksums.json
  • release-artifacts/latest/release-manifest.json
  • scripts/test_release_evidence_packet_index.py
✅ Files skipped from review due to trivial changes (2)
  • docs/public-beta-evidence.md
  • release-artifacts/latest/SHA256SUMS
🚧 Files skipped from review as they are similar to previous changes (6)
  • .github/workflows/ci.yml
  • release-artifacts/latest/release-checksums.json
  • ops/ROADMAP.md
  • release-artifacts/latest/release-manifest.json
  • ops/AUTONOMOUS_RUN.md
  • scripts/test_release_evidence_packet_index.py

📝 Walkthrough

Walkthrough

Adds a deterministic release evidence packet index generator CLI, tests, and committed JSON/Markdown artifacts; wires generation/checks into Makefile, CI, local check scripts, release manifest/checksum records, release-readiness checks, and documentation.

Changes

Release Evidence Packet Index

Layer / File(s) Summary
Core Generator Implementation
scripts/generate_release_evidence_packet_index.py
CLI with IO/path helpers, requirement canonicalization, template selection/validation, blocker-report consistency checks, packet row assembly, secret scan, deterministic JSON/Markdown rendering, write/check functions, and main dispatch.
Generator test suite
scripts/test_release_evidence_packet_index.py
End-to-end tests and fixtures: verify coverage, per-row template/commands/policy, determinism, secret detection, missing-template/template-only/blocker-reference failures, and check-mode drift detection.
Committed Packet Artifacts
release-artifacts/latest/release-evidence-packet-index.json, ... .md
Committed generated JSON (schema/metadata/status/rows/validation commands) and human-readable Markdown index with status table and command list.
Manifest & Checksum Integration
scripts/generate_release_manifest.py, release-artifacts/latest/release-manifest.json, release-artifacts/latest/SHA256SUMS, release-artifacts/latest/release-checksums.json
Adds release_evidence_packet_index entries to the manifest and updates checksum records for the new artifacts and changed docs.
Build System & CI Wiring
Makefile, .github/workflows/ci.yml, scripts/check.sh, scripts/check.ps1
New Makefile targets (release-evidence-packet-index / -check), make release-manifest depend on the check, include generator/test in CI py_compile and evidence-check phases, and add local script invocations.
Test Fixtures & Readiness Checks
scripts/test_release_artifacts.py, scripts/test_release_manifest.py, scripts/test_release_readiness.py, scripts/check_release_readiness.py, scripts/generate_release_artifacts.py
Seed packet index fixtures for tests, validate manifest inclusion/digests, require "release evidence packet index" phrase/commands/links in readiness checker, and exclude packet files from generated-file drift checks.
Documentation
docs/public-beta-evidence.md, docs/release-readiness.md, docs/tooling.md, release-artifacts/README.md
Document generator and --check commands, describe packet index mapping (blocker rows → templates/retained-artifacts/validation commands/posture), and add regeneration/check instructions.
Ops & Roadmap
ops/AUTONOMOUS_RUN.md, ops/ROADMAP.md
Advance durable run state and worklog for Queue Item 107; add packet-index artifacts and commands to Gate G required evidence and verification metadata.
Changelog
CHANGELOG.md
Adds Unreleased "Added" bullet describing the generated release evidence packet index and associated checks.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • 6529-Collections/6529Stream#106: Related to scripts/generate_release_artifacts.py downstream exclusions and manifest drift logic that this PR extends for packet-index outputs.
  • 6529-Collections/6529Stream#205: Adds the production blocker report artifacts that the generator consumes and validates when building packet rows.
  • 6529-Collections/6529Stream#193: Introduced public-beta blocker report artifacts that the packet-index generator references for requirement linkage.

Poem

🐰 I hop through manifest, template, and row,
I stitch each blocker where requirements go,
JSON hums, Markdown sings the map so bright,
check-mode finds drift and keeps the index right —
carrot-powered checksums guard the night.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Add release evidence packet index' clearly and concisely summarizes the main change in the changeset - the addition of a deterministic, no-secret release evidence packet index generator with supporting tests and integration across the build system, documentation, and CI pipeline.
Linked Issues check ✅ Passed The PR fully addresses issue #207 requirements: it adds a deterministic release evidence packet index generator that maps every public-beta and production-release requirement to blocker rows, templates, retained-artifact expectations, validation commands, and completion policies; includes focused tests with --check drift mode; wires the packet into local/CI checks, release manifest/checksum coverage, and documentation; and maintains blocked status without adding real evidence or secrets.
Out of Scope Changes check ✅ Passed All changes are tightly scoped to issue #207: the PR adds the packet index generator/tests, updates build targets and CI workflows, regenerates deterministic packet artifacts, updates checksums, integrates packet checks into documentation and release-readiness validation, and updates metadata/roadmap - with no unrelated modifications to Solidity, deployment behavior, or evidence contents.
Docstring Coverage ✅ Passed Docstring coverage is 87.76% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/release-evidence-packet-index

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
ops/ROADMAP.md (1)

373-399: ⚡ Quick win

Tighten the #207 summary to match the packet contract.

The new roadmap note stops at templates, retained-artifact expectations, validation commands, and readiness posture. Please also mention the packet fields required by the issue brief: phase, requirement ID, current status, owner/reviewer posture, blocker-report reference, and whether template-only evidence can ever satisfy a row. That keeps the roadmap aligned with the generator contract.

Proposed wording
- Issue `#207` adds a no-secret release evidence packet index/checker that maps blocker rows to templates, retained-artifact expectations, validation commands, and readiness posture without changing evidence contents or readiness claims.
+ Issue `#207` adds a no-secret release evidence packet index/checker that records phase, requirement ID, current status, owner/reviewer posture, blocker-report reference, template path, retained-artifact expectation, validation commands, and whether template-only evidence can ever complete a row, without changing evidence contents or readiness claims.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ops/ROADMAP.md` around lines 373 - 399, Update the `#207` roadmap summary to
explicitly list the release evidence packet fields required by the issue
contract: include "phase", "requirement ID", "current status", "owner/reviewer
posture", "blocker-report reference", and "template-only evidence
satisfiability" (whether template-only evidence can ever satisfy a row), in
addition to the already-mentioned mappings to templates, retained-artifact
expectations, validation commands, and readiness posture; edit the existing `#207`
sentence in the ROADMAP note so it enumerates these packet fields and their
presence in the generated release-artifacts/latest/release-evidence-packet-index
(JSON/MD) output.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/ci.yml:
- Around line 90-91: The py_compile ordering is inconsistent: swap the two
entries so the test script precedes its generator. In the CI py_compile list,
move scripts/test_release_evidence_packet_index.py to come before
scripts/generate_release_evidence_packet_index.py so the file order matches the
established convention (test_… before generate_…).

In `@docs/public-beta-evidence.md`:
- Around line 181-183: Update step 11 so the command both regenerates and
verifies the packet index by invoking the script with the --check flag (or show
two commands: run python scripts/generate_release_evidence_packet_index.py to
regenerate, then python scripts/generate_release_evidence_packet_index.py
--check to validate); specifically modify the line that currently shows python
scripts/generate_release_evidence_packet_index.py to include the --check
invocation (or add the separate --check command immediately after) so the
documentation reflects the drift-check workflow and ensures validation is not
skipped.

In `@scripts/test_release_evidence_packet_index.py`:
- Line 260: The test currently asserts a hard-coded row count and references a
hard-coded template filename; update the assertions to derive values from the
requirement lists instead: replace the literal 17 used in the assert on
packet["rows"] with len(checker.PUBLIC_BETA_REQUIREMENTS) or
len(checker.PRODUCTION_REQUIREMENTS) as appropriate for the test case, and
replace the hard-coded "external-audit-report-template.json" template filename
with a value computed from the same requirement set or a helper that maps the
requirement set to the expected template name; change the assertions and any
setup that builds packet["rows"] to compute expected_cardinality =
len(checker.PUBLIC_BETA_REQUIREMENTS) (or PRODUCTION_REQUIREMENTS) and
assertEqual(len(packet["rows"]), expected_cardinality), and similarly compute
the expected template filename from the requirement constants instead of using
the literal string.

---

Nitpick comments:
In `@ops/ROADMAP.md`:
- Around line 373-399: Update the `#207` roadmap summary to explicitly list the
release evidence packet fields required by the issue contract: include "phase",
"requirement ID", "current status", "owner/reviewer posture", "blocker-report
reference", and "template-only evidence satisfiability" (whether template-only
evidence can ever satisfy a row), in addition to the already-mentioned mappings
to templates, retained-artifact expectations, validation commands, and readiness
posture; edit the existing `#207` sentence in the ROADMAP note so it enumerates
these packet fields and their presence in the generated
release-artifacts/latest/release-evidence-packet-index (JSON/MD) output.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2f01ffa5-ca87-4e5c-9aac-543356fa0784

📥 Commits

Reviewing files that changed from the base of the PR and between 43a9596 and 8e2d9af.

📒 Files selected for processing (24)
  • .github/workflows/ci.yml
  • CHANGELOG.md
  • Makefile
  • docs/public-beta-evidence.md
  • docs/release-readiness.md
  • docs/tooling.md
  • ops/AUTONOMOUS_RUN.md
  • ops/ROADMAP.md
  • release-artifacts/README.md
  • release-artifacts/latest/SHA256SUMS
  • release-artifacts/latest/release-checksums.json
  • release-artifacts/latest/release-evidence-packet-index.json
  • release-artifacts/latest/release-evidence-packet-index.md
  • release-artifacts/latest/release-manifest.json
  • scripts/check.ps1
  • scripts/check.sh
  • scripts/check_release_readiness.py
  • scripts/generate_release_artifacts.py
  • scripts/generate_release_evidence_packet_index.py
  • scripts/generate_release_manifest.py
  • scripts/test_release_artifacts.py
  • scripts/test_release_evidence_packet_index.py
  • scripts/test_release_manifest.py
  • scripts/test_release_readiness.py

Comment thread .github/workflows/ci.yml Outdated
Comment thread docs/public-beta-evidence.md
Comment thread scripts/test_release_evidence_packet_index.py Outdated
@punk6529 punk6529 merged commit dec3450 into main Jun 13, 2026
2 checks passed
punk6529 added a commit that referenced this pull request Jun 13, 2026
Reconcile release evidence packet index merge state

Mark PR #209 and Queue Item 107 as merged in the durable autonomous run state, refresh roadmap verification metadata, and preserve the blocked public-beta/production readiness posture.

Closes #210.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add release evidence packet index and checker

1 participant