Skip to content

Guard wave 8: disk zap, key truncation, npm ownership; blind held-out-11 and held-out-12 - #1999

Merged
remigiusz-antczak merged 1 commit into
mainfrom
land/guard-w8b
Oct 25, 2026
Merged

remigiusz-antczak merged 1 commit into
mainfrom
land/guard-w8b

Conversation

@remigiusz-antczak

Copy link
Copy Markdown
Collaborator

Wave 8 closes the held-out-11 misses and the gaps folded forward from #1997. A sixth blind corpus, held-out-12, was written and pushed before it was scored. On held-out-12 this wave blocks one more dangerous command than main with no benign blocks, and five to eight times as many as the ported rivals.

Held-out-12 (blind, 60 dangerous / 60 benign) Blocked Benign blocked
Perun guard, this PR before review fixes (b9aa22871) 57/60 0/60
Perun guard, this PR after review fixes 57/60 0/60
Perun guard, main (8eb9fc0) 56/60 0/60
claude-code-safety-net, ported subset 12/60 (1 unported) 3/60
claude-code-hooks-mastery, ported 7/60 10/60
  • Preregistered. Held-out-12 was pushed before any scoring (prereg/heldout12, 25c580e, ledger timestamp 2026-10-25T08:52:43Z). The corpus is not in this PR; it lands with wave 9.

  • Held-out-12 misses (3), the leads for wave 9:

    • rm -f ~/.aws/credentials;
    • gpg --batch --yes --delete-secret-keys;
    • security delete-keychain.
  • Main's extra miss: sgdisk --zap-all, which this wave blocks.

  • Hooks-mastery's benign blocks include text passed as data: a gh pr comment --body, a git commit -m, an echo and a grep whose arguments mention a dangerous command.

  • Held-out-11 is now in-sample. Blind, this wave before review blocked 57/60 with 2/60 benign blocked; main blocked 57/60 with 3/60; safety-net 17/60 (7/60, 1 row unported); hooks-mastery 9/60 (6/60). Those rows are pinned in test_security_gates_heldout11_blind.py. After this wave it scores 60/60 with 1/60, the disputed git reset --hard HEAD, which stays blocked on purpose. Held-out-8, 9 and 10 keep their pinned numbers. These in-sample numbers are tuned and are not a generalization claim.

  • Ledger fix. The held-out-11 ledger's first line named SHA256-frozen.txt instead of recording its hash, which the bench check rejects. A second preregistered line now records that hash. It matches the file pushed in prereg/heldout11, and the line says it was added at land time. No row changed. This PR supersedes Guard wave 8: disk zap, key truncation, npm ownership; blind held-out-11 and held-out-12 #1998, which failed on that check.

Rule changes:

  • Now blocked:
    • sgdisk zap, clear, delete, new and load-backup;
    • truncating a private key or credential file through >, : >, cat /dev/null >, printf ... >, truncate, cp/mv/install /dev/null, dd of= and | tee;
    • npm and pnpm deprecate with a message, and owner rm;
    • restic key remove and unlock --remove-all;
    • lvresize unless every size grows, pvcreate -f/-ff, vgcreate -f;
    • zfs receive -F and zpool create -f;
    • a body padded past 20 KB that uses a variable after nested expansion now fails closed instead of skipping expansion.
  • Now allowed:
    • git clean -f of named relative paths. The allowance is void for ., .., absolute or globbed paths, .git, an -e/--exclude argument, a preceding cd elsewhere, and -C, --git-dir, --work-tree, GIT_DIR or GIT_WORK_TREE;
    • git stash drop stash@{N} of one stash.

Review: 3 🔴, each confirmed by a probe as a regression against main and fixed in this PR:

  • an -e/--exclude argument counted as a path, so git clean -fdx -e foo was allowed;
  • GIT_WORK_TREE=/ or env GIT_DIR=/ in front of an allowed git clean;
  • git clean -fdx ./.git.

Also fixed from the 🟡 list: sgdisk delete/new/load-backup/mbrtogpt, pvcreate -f, vgcreate -f, truncation through printf and : | tee.

Folded into wave 9 (main allows all of these too, so none is a regression):

  • deleting .git-credentials, service-account JSON, ~/.docker/config.json, .envrc and .netrc;
  • truncation through tee key </dev/null and cat /dev/null | tee key.

Local gates:

  • test_security_gates*.py: 26/26. test-destructive-guard.sh: 16/16. test_audit_skills: pass.
  • ci-tests.d: pass. run-ledger, prepush-ci-parity and bench-check cannot run in the local sandbox.
  • Size-ratchet: ok. Privacy: clean. git diff --check: clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z

… folds; blind held-out-11

The destructive guard now blocks sgdisk zap/clear, truncation of private keys and credential files (`> f`, `: > f`, `cat /dev/null > f`, truncate, cp/mv/install /dev/null, dd of=), npm/pnpm deprecate (non-empty message) and owner rm, restic key remove and unlock --remove-all, lvresize unless every size is +N, pvcreate -ff, zfs receive -F and zpool create -f. A padded body of 20 KB or more that uses a variable after nested expansion is skipped now fails closed. Allowed: `git clean -f` of named relative paths (not `.`, `..`, absolute, globbed, after a cd elsewhere, or with -C/--git-dir/--work-tree) and `git stash drop stash@{N}` of one stash. `git reset --hard HEAD` stays blocked.

Held-out-11 (preregistered before scoring): guard before review 57/60 blocked, 2/60 benign blocked; on main 57/60 with 3/60 (the extra is git reset --hard HEAD, a disputed label); safety-net ported 17/60 (1 unported, FP 7); hooks-mastery 9/60 (FP 6). Wave 8 scores 60/60 with 1/60 on it, now in-sample; the blind rows are pinned in test_security_gates_heldout11_blind.py. Held-out 8, 9 and 10 keep their pinned numbers.

Tests: 26/26 security gate scripts, test-destructive-guard 16/16, test_audit_skills ok, ci-tests.d all pass (run-ledger, prepush-ci-parity, bench-check skipped), size-ratchet ok, privacy clean, git diff --check clean

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z
@remigiusz-antczak
remigiusz-antczak merged commit 7453990 into main Oct 25, 2026
1 of 2 checks passed
@remigiusz-antczak
remigiusz-antczak deleted the land/guard-w8b branch October 25, 2026 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant