Skip to content

Add release evidence live audit report bundle#270

Merged
punk6529 merged 3 commits into
mainfrom
codex/release-evidence-live-audit-report-bundle
Jun 13, 2026
Merged

Add release evidence live audit report bundle#270
punk6529 merged 3 commits into
mainfrom
codex/release-evidence-live-audit-report-bundle

Conversation

@punk6529

@punk6529 punk6529 commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add optional JSON/Markdown report output to the release evidence live issue snapshot audit orchestrator.
  • Record selected profiles, snapshot paths, snapshot SHA-256 digests, command provenance, checker outcomes, repo target, generated-at value, and blocked-readiness warning.
  • Document the retained no-secret report workflow across tooling, release-readiness, public-beta evidence, release artifacts, roadmap traceability, and durable autonomous run state.
  • Refresh release manifest/checksum artifacts after the docs and roadmap changes.

Readiness posture

This remains an operator-only live audit enhancement. CI continues to use mocked/no-network coverage, and this PR does not mark public-beta or production-release retained evidence complete.

Closes #269

Validation

  • python scripts/test_release_evidence_issue_snapshot_audit.py
  • python scripts/test_release_readiness.py
  • python scripts/check_release_readiness.py
  • python scripts/audit_release_evidence_issue_snapshots.py --help
  • python -m py_compile scripts\audit_release_evidence_issue_snapshots.py scripts\test_release_evidence_issue_snapshot_audit.py scripts\check_release_readiness.py scripts\test_release_readiness.py
  • python scripts/generate_release_manifest.py --check
  • python scripts/generate_release_checksums.py --check
  • python scripts/check_changelog.py
  • rg -n "release evidence live audit report bundle|release-evidence-live-audit-report|Queue Item 128|Queue Item 129|PR #268|27478649600|bb36ddb|ce9f2ea|#267|#269|Last verified|CI run" ops/ROADMAP.md ops/AUTONOMOUS_RUN.md docs/tooling.md docs/public-beta-evidence.md docs/release-readiness.md release-artifacts/README.md CHANGELOG.md
  • rg -n "^#|^##|^###" ops/ROADMAP.md ops/AUTONOMOUS_RUN.md docs/tooling.md docs/public-beta-evidence.md docs/release-readiness.md release-artifacts/README.md
  • git diff --check
  • powershell -ExecutionPolicy Bypass -File scripts\check.ps1

Summary by CodeRabbit

  • New Features

    • Generate deterministic, no-secret release evidence audit reports (JSON + Markdown) with snapshot digests, command provenance, and stable formatting.
  • Documentation

    • Updated guides, roadmap, and READMEs to explain report generation, retention behavior, CLI flags, example commands, and validation expectations.
  • Tests

    • Added tests for report-mode scenarios, metadata stability, digest validation, and failure handling.
  • Chores

    • Updated release manifest and checksum records to reflect new documentation and retained report artifacts.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bf8a1780-fd5f-49bf-a389-cc00d6403029

📥 Commits

Reviewing files that changed from the base of the PR and between 178b8b2 and 3a71a61.

📒 Files selected for processing (5)
  • docs/tooling.md
  • release-artifacts/README.md
  • release-artifacts/latest/SHA256SUMS
  • release-artifacts/latest/release-checksums.json
  • release-artifacts/latest/release-manifest.json
✅ Files skipped from review due to trivial changes (5)
  • release-artifacts/latest/SHA256SUMS
  • release-artifacts/latest/release-checksums.json
  • release-artifacts/README.md
  • docs/tooling.md
  • release-artifacts/latest/release-manifest.json

📝 Walkthrough

Walkthrough

Adds a retained no‑secret release evidence live audit report bundle: deterministic JSON/Markdown reports with snapshot paths and SHA‑256 digests, command provenance, and readiness warnings; wired into the audit CLI, validated by tests, enforced in readiness checks, and documented with manifest/checksum updates.

Changes

Release Evidence Live Audit Report Bundle

Layer / File(s) Summary
Report schema, hashing, and serialization
scripts/audit_release_evidence_issue_snapshots.py
Introduces REPORT_SCHEMA_VERSION, fixed notices, sha256_file() helper, and deterministic report builders (build_report, write_report_json, markdown_cell, markdown_report, write_report_markdown).
Orchestrator CLI and collection
scripts/audit_release_evidence_issue_snapshots.py
Adds --report-json, --report-md, --generated-at args; updates audit_profile(..., collect_report=True) and main() to aggregate per-profile results and write selected reports.
Deterministic report and failure tests
scripts/test_release_evidence_issue_snapshot_audit.py
Adds deterministic snapshot helpers and subprocess mocking to materialize snapshot files; tests validate JSON/Markdown determinism, schema/version/repo/timestamp fields, profile ordering, SHA‑256 digests, and failure modes that must prevent report writes.
Readiness checks and fixtures
scripts/check_release_readiness.py, scripts/test_release_readiness.py, docs/release-readiness.md
Adds "release evidence live audit report bundle" to required readiness phrases and requires audit_release_evidence_issue_snapshots.py --report-json tmp/release-evidence-live-audit-report.json --report-md tmp/release-evidence-live-audit-report.md in the release-readiness doc; updates test fixture wording.
Docs, changelog, and manifest updates
CHANGELOG.md, docs/public-beta-evidence.md, docs/tooling.md, release-artifacts/README.md, release-artifacts/latest/*
Documents the retained no-secret JSON/Markdown report bundle and its contents (profiles, snapshot paths, SHA‑256 digests, command provenance, checker outcomes, readiness warning). Updates release artifact checksums and release-manifest.json entries for the modified docs.
Autonomous workflow & roadmap tracking
ops/AUTONOMOUS_RUN.md, ops/ROADMAP.md
Records Queue Item 128 reconciliation (PR #268), advances Queue Item 129 to active with PR/issue linkage (#269), updates verification metadata and Test Matrix entry for the live audit report bundle.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

  • #265: The orchestrator and report generation implemented here align with the orchestrator objectives described in issue #265 (live audit orchestration and report flags).

Possibly related PRs

Poem

🐰 A tidy audit bundle, no secrets to hide,
SHA‑256 trails keeping truth by my side,
JSON and Markdown, stamped neat in a row,
Commands and snapshots all ready to show.
The roadmap hops onward—reports in a bundle, let's go!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically summarizes the main change: adding a release evidence live audit report bundle feature.
Linked Issues check ✅ Passed PR successfully delivers all coding objectives from issue #269: deterministic JSON/Markdown reporting [scripts/audit_release_evidence_issue_snapshots.py], digest calculation via sha256_file(), profile ordering/selection, failure handling, test coverage [scripts/test_release_evidence_issue_snapshot_audit.py], readiness validation [scripts/check_release_readiness.py], and comprehensive documentation across tooling/release-readiness/public-beta-evidence/release-artifacts.
Out of Scope Changes check ✅ Passed All changes align with issue #269 scope: audit report bundle implementation, deterministic output, digest/checksum updates, documentation, and roadmap/autonomous run state tracking. No unrelated modifications detected.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/release-evidence-live-audit-report-bundle

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/tooling.md (1)

183-188: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Document the report bundle's repo target everywhere it is described.

The retained live-audit report contract in the PR objectives includes the repository target, but these descriptions currently enumerate the other retained fields and omit that one. Please add it so the operator docs stay aligned with the generated report schema.

  • docs/tooling.md#L183-L188: mention the repo target in the high-level report-bundle summary.
  • docs/tooling.md#L259-L270: include the repo target alongside the snapshot paths, digests, provenance, and checker results.
  • release-artifacts/README.md#L183-L188: mirror the same field in the release-artifacts summary.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/tooling.md` around lines 183 - 188, The report-bundle descriptions omit
the "repo target" field—add it to each location so docs match the retained
live-audit report contract: in docs/tooling.md (lines 183-188) update the
high-level report-bundle summary to mention the repository target alongside the
regenerated manifest and the architecture/threat-model validation; in
docs/tooling.md (lines 259-270) add the repo target to the detailed list that
currently enumerates snapshot paths, digests, provenance, and checker results so
it appears as an explicit retained field; in release-artifacts/README.md (lines
183-188) mirror the same addition in the release-artifacts summary so the repo
target is listed together with the other retained fields.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ops/AUTONOMOUS_RUN.md`:
- Around line 35-39: Update the durable state metadata for Queue Item 129 so it
reflects the actual open PR instead of "TBD": change the "Active PR" field from
TBD to the opened pull request URL (use the repo's PR `#270` link) and ensure any
other metadata rows for Queue Item 129 (the "Active PR" entry shown earlier and
the duplicate block later around lines referenced in the review) are updated to
the same PR URL so the autonomous run state consistently shows PR `#270` rather
than TBD.

---

Outside diff comments:
In `@docs/tooling.md`:
- Around line 183-188: The report-bundle descriptions omit the "repo target"
field—add it to each location so docs match the retained live-audit report
contract: in docs/tooling.md (lines 183-188) update the high-level report-bundle
summary to mention the repository target alongside the regenerated manifest and
the architecture/threat-model validation; in docs/tooling.md (lines 259-270) add
the repo target to the detailed list that currently enumerates snapshot paths,
digests, provenance, and checker results so it appears as an explicit retained
field; in release-artifacts/README.md (lines 183-188) mirror the same addition
in the release-artifacts summary so the repo target is listed together with the
other retained fields.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 84889f3f-b5db-4461-873f-2cab60ec1489

📥 Commits

Reviewing files that changed from the base of the PR and between bb36ddb and 55709ab.

📒 Files selected for processing (14)
  • CHANGELOG.md
  • docs/public-beta-evidence.md
  • docs/release-readiness.md
  • docs/tooling.md
  • ops/AUTONOMOUS_RUN.md
  • ops/ROADMAP.md
  • release-artifacts/README.md
  • release-artifacts/latest/SHA256SUMS
  • release-artifacts/latest/release-checksums.json
  • release-artifacts/latest/release-manifest.json
  • scripts/audit_release_evidence_issue_snapshots.py
  • scripts/check_release_readiness.py
  • scripts/test_release_evidence_issue_snapshot_audit.py
  • scripts/test_release_readiness.py

Comment thread ops/AUTONOMOUS_RUN.md

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@punk6529 punk6529 merged commit 774d564 into main Jun 13, 2026
2 checks passed
punk6529 added a commit that referenced this pull request Jun 13, 2026
Record PR #270 merge evidence and refresh roadmap metadata.

Select the live audit report schema/checker as the next no-secret queue item.

Closes #271
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add release evidence live audit report bundle

1 participant