Skip to content

Add metadata fixture safety checks#112

Merged
punk6529 merged 3 commits into
mainfrom
codex/metadata-render-sandbox-checks
Jun 11, 2026
Merged

Add metadata fixture safety checks#112
punk6529 merged 3 commits into
mainfrom
codex/metadata-render-sandbox-checks

Conversation

@punk6529

@punk6529 punk6529 commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a dependency-free Python metadata fixture safety checker that validates committed golden fixtures for strict JSON/data-URI decoding, metadata JSON shape, current URI scheme policy, and final animation HTML script-boundary structure.
  • Add hostile fixture self-tests for bad URI schemes, malformed data URIs, unexpected HTML tags, raw script-boundary breakouts, and non-HTTPS external scripts.
  • Wire the checker into make check, Unix/Windows check wrappers, and CI, then update metadata docs, roadmap/run-state traceability, changelog, release manifest, and checksum bundle.

Continues #51. This intentionally keeps production bytecode unchanged and leaves production URI enforcement, invalid UTF-8 policy, semantic/structured attributes, and full browser execution sandboxing for later P1-META-006 slices.

Local validation

  • python scripts\test_metadata_fixtures.py
  • python scripts\check_metadata_fixtures.py
  • python -m py_compile scripts\check_metadata_fixtures.py scripts\test_metadata_fixtures.py
  • bash -n scripts/check.sh scripts/bootstrap-ec2.sh
  • PowerShell parser validation for scripts\check.ps1 and scripts\bootstrap-windows.ps1
  • python scripts\test_changelog_check.py
  • python scripts\check_changelog.py
  • git diff --check
  • make release-checksums
  • make check
  • powershell -ExecutionPolicy Bypass -File scripts\check.ps1

Production bytecode is unchanged; the size gate still reports StreamCore at 24,461 runtime bytes with 115 bytes of EIP-170 headroom.

Summary by CodeRabbit

  • New Features

    • Added metadata fixture safety validation for JSON/data-URI decoding, URI-scheme enforcement, and animation HTML wrapper/script boundary checks.
  • Chores

    • Integrated metadata fixture checks into local verification, Make/check targets, and CI/release verification flows.
  • Tests

    • Added automated tests exercising fixture validation and failure cases for unsafe schemes, malformed data URIs, missing images, and script-boundary violations.
  • Documentation

    • Updated docs and roadmap/status/changelog to describe the new fixture-level safety gates and remaining work.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8924a380-331f-4c33-9c00-27402b9891c2

📥 Commits

Reviewing files that changed from the base of the PR and between 3dbdc5c and 74e6be8.

📒 Files selected for processing (3)
  • ops/AUTONOMOUS_RUN.md
  • scripts/check_metadata_fixtures.py
  • scripts/test_metadata_fixtures.py
✅ Files skipped from review due to trivial changes (1)
  • ops/AUTONOMOUS_RUN.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • scripts/check_metadata_fixtures.py
  • scripts/test_metadata_fixtures.py

📝 Walkthrough

Walkthrough

This PR adds a Python metadata fixture checker and unittest harness enforcing data-URI/JSON/URI-scheme and animation-HTML wrapper/script-boundary rules, wires them into Makefile/check scripts and CI (including syntax checks and a dedicated fixture-safety step), and updates docs, changelog, release checksums, and ops state.

Changes

Metadata Fixture Validation Gate

Layer / File(s) Summary
Metadata fixture checker implementation
scripts/check_metadata_fixtures.py
Core validator with data-URI decoding, strict UTF‑8 JSON parsing, required metadata keys/state/schema/version checks, external URI allowlist and format checks, HTML wrapper parsing with exact two-script boundary constraints, and CLI plumbing.
Metadata fixture test harness
scripts/test_metadata_fixtures.py
Unittest suite that dynamically loads the checker, generates fixture sets, verifies repository fixtures pass, and asserts failure modes for unsafe schemes, malformed base64, empty image, unexpected HTML tags, script-boundary breakout sequences, and non-HTTPS external scripts.
Build/CI/check script integration
Makefile, scripts/check.sh, scripts/check.ps1, .github/workflows/ci.yml
New metadata-fixtures-check Makefile target; check scripts invoke both test and checker; CI adds the scripts to py_compile step and a dedicated Metadata fixture safety job that captures logs to ci-logs/.
Documentation, changelog, roadmap, blockers
docs/metadata.md, docs/status.md, test/README.md, docs/known-blockers.md, ops/ROADMAP.md, CHANGELOG.md
Docs updated to describe fixture-level JSON/data-URI/HTML boundary checks and Gate integration; changelog and roadmap updated; known blockers note remaining production-URI, UTF-8, and browser sandbox work.
Release artifacts, checksums, and operational state
release-artifacts/latest/*, ops/AUTONOMOUS_RUN.md
Regenerated checksums and manifest entries for changed docs; AUTONOMOUS_RUN updated to reflect PR/queue/decision log state for metadata work.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • 6529-Collections/6529Stream#83: Related schema/v1 on-chain tokenURI and fixture changes that the Python validators enforce (metadata_schema_version/metadata_state and base64 JSON decoding).
  • 6529-Collections/6529Stream#81: Also works with the same committed golden fixtures; this PR adds Python fixture checks complementary to Solidity fixture checks added earlier.
  • 6529-Collections/6529Stream#4: Prior CI/check flow changes (forge build/test wiring) that this PR extends by adding metadata fixture test/check steps.

Poem

🐰 I hop through fixtures, base64 in paw,

I check each URI and every JSON law,
Two scripts I count, no breakout in sight,
Metadata safe, the gates shine bright!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding metadata fixture safety checks. It directly corresponds to the PR's primary objective and is specific enough to convey the purpose without being overly verbose.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/metadata-render-sandbox-checks

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
scripts/check_metadata_fixtures.py (1)

236-237: Add symmetric structural validation for off-chain final tokenURI suffix/pattern

validate_fixture_set validates offchain_pending with safe schemes and enforces it ends with /{STATE_PENDING} (/pending), but offchain_final only gets the safe-schemes check—no corresponding endswith/shape constraint. If the “final” tokenURI is expected to follow a specific pattern (e.g., /{STATE_FINAL} or /final), apply the same structural validation to offchain_final as well. [optional improvement]

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/check_metadata_fixtures.py` around lines 236 - 237,
validate_fixture_set currently enforces that offchain_pending ends with
f"/{STATE_PENDING}" but does not apply a symmetric structural check to
offchain_final; add a corresponding validation in validate_fixture_set to verify
offchain_final.endswith(f"/{STATE_FINAL}") (or the appropriate STATE_FINAL
constant) and raise MetadataFixtureError with a clear message like "off-chain
final tokenURI must end with /final" when it fails, using the offchain_final
identifier and STATE_FINAL constant to locate and implement the check.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/check_metadata_fixtures.py`:
- Around line 106-108: validate_external_uri currently returns early for uri ==
"" causing metadata entries with {"image": ""} to pass; change the validation so
that when uri == "" and label == "image" it raises an error (or returns a
failing validation) instead of returning silently. Update the
validate_external_uri function (and its callers like validate_metadata_json) to
enforce non-empty image URIs while keeping existing off-chain tokenURI logic
unchanged (do not add any `/final` suffix checks).

---

Nitpick comments:
In `@scripts/check_metadata_fixtures.py`:
- Around line 236-237: validate_fixture_set currently enforces that
offchain_pending ends with f"/{STATE_PENDING}" but does not apply a symmetric
structural check to offchain_final; add a corresponding validation in
validate_fixture_set to verify offchain_final.endswith(f"/{STATE_FINAL}") (or
the appropriate STATE_FINAL constant) and raise MetadataFixtureError with a
clear message like "off-chain final tokenURI must end with /final" when it
fails, using the offchain_final identifier and STATE_FINAL constant to locate
and implement the check.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 35e98dff-a715-4deb-9bee-dfedd04f744d

📥 Commits

Reviewing files that changed from the base of the PR and between f844457 and 3dbdc5c.

📒 Files selected for processing (16)
  • .github/workflows/ci.yml
  • CHANGELOG.md
  • Makefile
  • docs/known-blockers.md
  • docs/metadata.md
  • docs/status.md
  • ops/AUTONOMOUS_RUN.md
  • ops/ROADMAP.md
  • release-artifacts/latest/SHA256SUMS
  • release-artifacts/latest/release-checksums.json
  • release-artifacts/latest/release-manifest.json
  • scripts/check.ps1
  • scripts/check.sh
  • scripts/check_metadata_fixtures.py
  • scripts/test_metadata_fixtures.py
  • test/README.md

Comment thread scripts/check_metadata_fixtures.py

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Latest head 74e6be8a0b922f92f76d765925de072ebf0e2559 addresses the empty metadata image finding with a validator guard and regression test. The optional off-chain final /final suffix suggestion was intentionally skipped because the committed final fixture is content-addressed (ipfs://base/10000000000) rather than state-suffixed.

Local validation after the fix:

  • python scripts\test_metadata_fixtures.py
  • python scripts\check_metadata_fixtures.py
  • python -m py_compile scripts\check_metadata_fixtures.py scripts\test_metadata_fixtures.py
  • make check
  • powershell -ExecutionPolicy Bypass -File scripts\check.ps1
  • git diff --check

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

@punk6529, I'll kick off a review of the latest head 74e6be8a0b922f92f76d765925de072ebf0e2559. Understood on the /final suffix skip — content-addressed IPFS paths (ipfs://base/10000000000) are inherently immutable, so a state suffix would be redundant there. I'll review the empty-image validator guard and regression test as part of this pass.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@punk6529 punk6529 merged commit 419fb1d into main Jun 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant