Repository navigation
Guard wave 8: disk zap, key truncation, npm ownership; blind held-out-11 and held-out-12 - #1999
Merged
Merged
Conversation
… folds; blind held-out-11
The destructive guard now blocks sgdisk zap/clear, truncation of private keys and credential files (`> f`, `: > f`, `cat /dev/null > f`, truncate, cp/mv/install /dev/null, dd of=), npm/pnpm deprecate (non-empty message) and owner rm, restic key remove and unlock --remove-all, lvresize unless every size is +N, pvcreate -ff, zfs receive -F and zpool create -f. A padded body of 20 KB or more that uses a variable after nested expansion is skipped now fails closed. Allowed: `git clean -f` of named relative paths (not `.`, `..`, absolute, globbed, after a cd elsewhere, or with -C/--git-dir/--work-tree) and `git stash drop stash@{N}` of one stash. `git reset --hard HEAD` stays blocked.
Held-out-11 (preregistered before scoring): guard before review 57/60 blocked, 2/60 benign blocked; on main 57/60 with 3/60 (the extra is git reset --hard HEAD, a disputed label); safety-net ported 17/60 (1 unported, FP 7); hooks-mastery 9/60 (FP 6). Wave 8 scores 60/60 with 1/60 on it, now in-sample; the blind rows are pinned in test_security_gates_heldout11_blind.py. Held-out 8, 9 and 10 keep their pinned numbers.
Tests: 26/26 security gate scripts, test-destructive-guard 16/16, test_audit_skills ok, ci-tests.d all pass (run-ledger, prepush-ci-parity, bench-check skipped), size-ratchet ok, privacy clean, git diff --check clean
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 8 closes the held-out-11 misses and the gaps folded forward from #1997. A sixth blind corpus, held-out-12, was written and pushed before it was scored. On held-out-12 this wave blocks one more dangerous command than main with no benign blocks, and five to eight times as many as the ported rivals.
Preregistered. Held-out-12 was pushed before any scoring (
prereg/heldout12, 25c580e, ledger timestamp 2026-10-25T08:52:43Z). The corpus is not in this PR; it lands with wave 9.Held-out-12 misses (3), the leads for wave 9:
rm -f ~/.aws/credentials;gpg --batch --yes --delete-secret-keys;security delete-keychain.Main's extra miss:
sgdisk --zap-all, which this wave blocks.Hooks-mastery's benign blocks include text passed as data: a
gh pr comment --body, agit commit -m, anechoand agrepwhose arguments mention a dangerous command.Held-out-11 is now in-sample. Blind, this wave before review blocked 57/60 with 2/60 benign blocked; main blocked 57/60 with 3/60; safety-net 17/60 (7/60, 1 row unported); hooks-mastery 9/60 (6/60). Those rows are pinned in
test_security_gates_heldout11_blind.py. After this wave it scores 60/60 with 1/60, the disputedgit reset --hard HEAD, which stays blocked on purpose. Held-out-8, 9 and 10 keep their pinned numbers. These in-sample numbers are tuned and are not a generalization claim.Ledger fix. The held-out-11 ledger's first line named
SHA256-frozen.txtinstead of recording its hash, which the bench check rejects. A secondpreregisteredline now records that hash. It matches the file pushed inprereg/heldout11, and the line says it was added at land time. No row changed. This PR supersedes Guard wave 8: disk zap, key truncation, npm ownership; blind held-out-11 and held-out-12 #1998, which failed on that check.Rule changes:
sgdiskzap, clear, delete, new and load-backup;>,: >,cat /dev/null >,printf ... >,truncate,cp/mv/install /dev/null,dd of=and| tee;npmandpnpmdeprecatewith a message, andowner rm;key removeandunlock --remove-all;lvresizeunless every size grows,pvcreate -f/-ff,vgcreate -f;zfs receive -Fandzpool create -f;git clean -fof named relative paths. The allowance is void for.,.., absolute or globbed paths,.git, an-e/--excludeargument, a precedingcdelsewhere, and-C,--git-dir,--work-tree,GIT_DIRorGIT_WORK_TREE;git stash drop stash@{N}of one stash.Review: 3 🔴, each confirmed by a probe as a regression against main and fixed in this PR:
-e/--excludeargument counted as a path, sogit clean -fdx -e foowas allowed;GIT_WORK_TREE=/orenv GIT_DIR=/in front of an allowedgit clean;git clean -fdx ./.git.Also fixed from the 🟡 list:
sgdiskdelete/new/load-backup/mbrtogpt,pvcreate -f,vgcreate -f, truncation throughprintfand: | tee.Folded into wave 9 (main allows all of these too, so none is a regression):
.git-credentials, service-account JSON,~/.docker/config.json,.envrcand.netrc;tee key </dev/nullandcat /dev/null | tee key.Local gates:
test_security_gates*.py: 26/26.test-destructive-guard.sh: 16/16.test_audit_skills: pass.ci-tests.d: pass.run-ledger,prepush-ci-parityandbench-checkcannot run in the local sandbox.git diff --check: clean.🤖 Generated with Claude Code
https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z