Skip to content

guard non-rust evidence boundaries#1056

Merged
TheGreenCedar merged 14 commits into
dev/codestory-nextfrom
codex/922-corpus-boundary
Jul 13, 2026
Merged

guard non-rust evidence boundaries#1056
TheGreenCedar merged 14 commits into
dev/codestory-nextfrom
codex/922-corpus-boundary

Conversation

@TheGreenCedar

@TheGreenCedar TheGreenCedar commented Jul 13, 2026

Copy link
Copy Markdown
Owner

Context

Closes #1054
Refs #958
Refs #922
Refs #899

CodeStory's release evaluator previously imported the packet benchmark and its checked evaluation/query corpus transitively. That made release-control code depend on proof-only data and left non-Rust product and release surfaces outside the corpus-boundary guard.

Outcome

Release-control code now uses a corpus-neutral provenance module. The generalization guard protects the repository-controlled non-Rust product and release boundary, including shipped plugin guidance, setup/install tooling, runtime configuration, and release automation.

The exact integrated head is 58e75f3ed0089eb7e57dfe83b8e0574570ab5bfb. It merges current dev/codestory-next at cea5aa3975641280d9c42ee8e6d29cc1e88d2697 while preserving the accepted corpus-guard fix at 0cd9b9ae3a84c74cdb56190bfca31660c7b7bfb7.

What changed

  • Extracted repository/cache provenance validation into the corpus-neutral scripts/codestory-evidence-provenance.mjs module. The benchmark driver keeps its public exports, while the release evaluator imports the neutral module directly.
  • Added an executable non-Rust inventory to the generalization lint. The current scan covers 73 protected files against 1,433 corpus-derived patterns; new supported files under protected roots enter automatically, while corpus/proof harnesses and tests/fixtures remain explicit exclusions.
  • Made missing protected files and classified harnesses fail closed. Direct, adjacent/split, and explicit harness-module imports are rejected across native separators, case variants, shell/workflow continuations, PowerShell continuations, and comment-separated JavaScript.
  • Bound the narrow release-policy exceptions to an exact root-relative file, literal, normalized use, occurrence offset, and single occurrence. Duplicate approved-looking lines, hostile direct uses, and later split occurrences fail closed.
  • Updated workflow triggers, architecture/playbook guidance, the changelog, and the mandatory repo-scale telemetry row.

How to review

  1. Review scripts/codestory-evidence-provenance.mjs and its two callers to confirm the extraction preserved provenance behavior without importing the benchmark corpus.
  2. Review the protected/default and explicit-harness inventories in scripts/lint-retrieval-generalization.mjs.
  3. Follow direct occurrences through regexOccurrences and compact occurrences through compactMarkerOccurrences; the allowance must attach only to the exact approved use.
  4. Inspect linter_binds_policy_allowances_to_the_exact_approved_use and the portable hostile fixture table.

Verification

Exact integrated head 58e75f3e:

  • cargo fmt --all -- --check — passed
  • cargo test --locked -p codestory-runtime --test retrieval_generalization_guard — 17 passed
  • node scripts/lint-retrieval-generalization.mjs — passed; 73 protected non-Rust files and 1,433 patterns
  • node .github/scripts/check-workflow-policy.mjs — passed
  • node .github/scripts/route-ci-proof.mjs --self-test — passed
  • git diff --check origin/dev/codestory-next...HEAD — passed
  • Exact-head re-review of the occurrence-bound allowance — accepted with no P1/P2 findings

Earlier implementation proof, before the focused review fixes:

  • cargo check --workspace --locked, cargo test --workspace --locked, all-target/all-feature clippy with warnings denied, and the release CLI build — passed at 0131b97
  • node --test scripts/tests/*.test.mjs — 125 passed
  • node --test plugins/codestory/tests/plugin-static.test.mjs — 71 passed, 1 platform skip
  • Release-detector unit harness — 6 passed with Python 3.14
  • Mandatory repo-scale telemetry at 9c97808a+1054wt — full-sidecar pass, 148,577 nodes, 123,886 edges, 308 files, 0 errors

The stats lane used explicit CPU allowance and emitted the expected living-baseline warning: retrieval indexing took 66.14 seconds versus the 19.23-second warning threshold. This row is telemetry, not baseline-promotion evidence.

Risk and follow-up

The guard is intentionally static. It does not claim to prove arbitrary runtime-generated strings, environment/generated bytes, or external-process internals; existing attestation and runtime checks remain responsible for those boundaries. This PR does not change release activation, baseline selection, or timing thresholds under #958.

@TheGreenCedar TheGreenCedar force-pushed the codex/922-corpus-boundary branch from 2fbe266 to 0cf43ca Compare July 13, 2026 15:02
@TheGreenCedar

Copy link
Copy Markdown
Owner Author

Review remediation pushed

The final corpus-boundary review gaps are fixed and pushed at cf8c3801c86ab3a83eb91a476711820bd862883b on codex/922-corpus-boundary.

What changed:

  • JavaScript and TypeScript comments are masked before import and adjacent-literal analysis, preventing comment-only false positives while retaining comment-separated split dependency detection.
  • Continued dependency scanning now covers shell, PowerShell, and workflow YAML and checks both evaluation-corpus and harness identities without rejecting existing full-token proof commands.
  • retrieval-sidecar-smoke now runs the focused retrieval_generalization_guard integration suite for script-only guard changes.
  • The existing guard table contains focused regressions for the four reviewed cases.

Verification:

  • git diff --check
  • node --check scripts/lint-retrieval-generalization.mjs
  • node scripts/lint-retrieval-generalization.mjs
  • Analyzer and release-evidence Node tests: 86 passed
  • Workflow policy check passed
  • Documentation link check passed
  • cargo fmt --all -- --check
  • cargo test -p codestory-runtime --test retrieval_generalization_guard --locked: 16 passed

This update is pushed for review. No readiness or merge action was taken.

@TheGreenCedar

Copy link
Copy Markdown
Owner Author

Review checkpoint — paused safely

PR #1056 remains draft at published head cf8c3801c86ab3a83eb91a476711820bd862883b on codex/922-corpus-boundary (reviewed against base 174b77ea20093150cd1de3640d885327e3704b3f). The worktree is clean; no follow-up fixes were applied after this head.

Two P2 findings remain:

  1. Direct one-token harness execution/configuration is not caught unless it appears as a JavaScript import, split literals, or a continuation. Examples include node scripts/fetch-holdout-repos.mjs, a YAML run: value, or a spawn-array entry.
  2. Comment masking is not fed into the direct eval-marker and shell/PowerShell/YAML continuation scans, so comment-only references can fail the guard.

Focused Cargo verification for the eventual fixes is still pending. This is a safe published resume point; do not mark ready or merge yet.

@TheGreenCedar TheGreenCedar marked this pull request as ready for review July 13, 2026 22:40
@TheGreenCedar TheGreenCedar merged commit 6c4e7d6 into dev/codestory-next Jul 13, 2026
17 checks passed
@TheGreenCedar TheGreenCedar deleted the codex/922-corpus-boundary branch July 13, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant