Skip to content

Guard wave 7: storage, backup and hosted-platform CLIs, cwd-aware rm; blind held-out-10 and 11 - #1997

Merged
remigiusz-antczak merged 1 commit into
mainfrom
land/guard-w7
Oct 25, 2026
Merged

remigiusz-antczak merged 1 commit into
mainfrom
land/guard-w7

Conversation

@remigiusz-antczak

Copy link
Copy Markdown
Collaborator

Wave 7 closes the held-out-10 misses and false positives and the gaps folded forward from #1994. A fifth blind corpus, held-out-11, was written and pushed before it was scored. On held-out-11 this wave blocks as many dangerous commands as main with fewer benign blocks, and it blocks three to six times as many as the ported rivals.

Held-out-11 (blind, 60 dangerous / 60 benign) Blocked Benign blocked
Perun guard, this PR before review fixes (3691a0431) 57/60 2/60
Perun guard, this PR after review fixes 57/60 3/60
Perun guard, main (9e22d2a) 57/60 4/60
claude-code-safety-net, ported subset 17/60 (1 unported) 7/60
claude-code-hooks-mastery, ported 9/60 6/60
  • Preregistered. Held-out-11 was pushed before any scoring (prereg/heldout11, ad0ee43, ledger timestamp 2026-10-25T06:47:25Z). The corpus is not in this PR; it lands with wave 8.
  • Why 3 differs from 2. The review fixes block git reset --hard HEAD again, on purpose. Held-out-10 and held-out-11 both label it benign, but it discards uncommitted work with no way back, so the guard counts the row as a false positive and keeps blocking it.
  • Held-out-11 misses (3), the leads for wave 8:
    • sgdisk --zap-all;
    • truncating a private key with cat /dev/null >;
    • npm owner rm.
  • Held-out-11 false positives besides the reset: git clean -fdx ./build and git stash drop stash@{0}.
  • Held-out-10 is now in-sample. Blind, main and Guard wave 6: cloud, VCS, Windows and shell-split deletes; blind held-out-9 and 10 #1994 both blocked 44/60 with 6/60 benign blocked; hooks-mastery 13/60 (9/60), safety-net 14/60 (4/60, 1 row unported). That row is pinned in test_security_gates_heldout10_blind.py. After this wave it scores 60/60 with 1/60 benign blocked, the disputed reset. Held-out-8 and held-out-9 stay at 60/60 and 0/60. These in-sample numbers are tuned and are not a generalization claim.

Rule changes:

  • Now blocked:

    • rclone purge, delete, sync, move, moveto, bisync and --delete-* flags, with the verb found after any flags;
    • restic forget --prune and prune;
    • zfs destroy and rollback, zpool destroy and labelclear, LVM removals including through lvm <options>;
    • firebase, supabase, vercel, netlify and similar platform deletes, with the delete verb matched inside camelCase and a:b:delete forms;
    • database CLIs whose --command or -c string drops data, and psql -f or cat file | feeding a dropping SQL file;
    • flags held in variables (f=-rf; rm $f /) and rm after a cd to a system path, including inside a subshell;
    • the wave-6 leftovers: ln -sf onto shell and SSH files in the home directory, rsync to a remote /etc, vault operator seal and rekey, reg add on user hives, wmic /node ... reboot, clickhouse ALTER ... DELETE, DROP PARTITION and SQL fed through stdin, make nuke|uninstall DESTDIR=/, mongosh bulkWrite, sqlite3 .shell;
    • cp onto a system path with a backup-looking suffix (.bak, .orig, .old).
  • Now allowed:

    • find under the current tree with a single -delete or a single -exec rm ... {} action;
    • xargs -I{} rm fed only ./ paths;
    • wipefs without -a or -o, which only lists signatures;
    • shred -u of a named file in /tmp.

    Each allowance is void when the command first changes directory to /, ~, a variable or .., and find loses it with -L, -H or -follow.

Review: 4 🔴 and 4 🟡. Every 🔴 was confirmed by a probe as a regression against main and is fixed in this PR:

  • a second -exec action chained after an allowed find;
  • the current-tree allowances surviving a preceding cd /;
  • find -L following symlinks out of the tree;
  • git -C <other repo> reset --hard HEAD.

Also fixed from the 🟡 list: the rclone verb behind unknown flags, platform delete verbs, LVM through lvm, psql -f, and a bounded ALTER TABLE scan.

Folded into wave 8:

  • a command padded past 20 KB disables nested variable expansion, so A='rm -rf /'; B=$A; $B followed by a 20 KB comment is allowed. Main allows it too; the fix is to fail closed;
  • restic key remove and unlock --remove-all;
  • lvresize shrinking a volume, pvcreate -ff;
  • zfs receive -F and zpool create -f.

The adversarial timing limit in the guard self-test moves from 0.5 s to 1.0 s. A 720 KB input takes 0.3 to 0.5 s, and most of that time is spent tokenizing it.

Local gates:

  • test_security_gates*.py: 23/23. test-destructive-guard.sh: 16/16. test_audit_skills: 11/11.
  • ci-tests.d: pass. run-ledger, prepush-ci-parity and bench-check cannot run in the local sandbox.
  • Size-ratchet: ok. git diff --check: clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z

… blind held-out-10

The destructive guard now blocks rclone purge/delete/sync, restic forget/prune, zfs/zpool destroy, lvremove/vgremove/pvremove, firebase/supabase/vercel/netlify/wrangler deletes and `supabase db reset --linked`, a relative `rm -r` after `cd /abs`, variable-held flags inside `bash -c`, ln onto shell startup files and ~/.ssh, rsync/scp to host:/etc, vault operator seal/rekey, reg add under HKCU/HKU Software Classes, wmic call reboot, ClickHouse ALTER DELETE/DROP PARTITION/clickhouse-local, a SQL client fed `< file.sql` that drops, make nuke/uninstall at root, mongosh bulkWrite deletes, and sqlite3 `.shell env sh`. cp of a .bak/.orig/.old onto a system path is no longer exempt. Benign forms now allowed (single-action find only, void after a cd to an absolute, ~, $var or .. target and with -L/-H/-follow): find -delete/-exec rm under a named relative dir or with a sole artifact glob, `echo ./a | xargs -I{} rm -rf {}`, wipefs without -a/-o, shred inside $TMPDIR or /tmp.

Held-out-10 (preregistered before scoring, ledger.jsonl): guard on main and wave 6 blocked 44/60 with 6/60 benign blocked; hooks-mastery 13/60 (FP 9); safety-net 14/60 (FP 4, 1 unported). Wave 7 scores 60/60 with FP 1/60 on it (git reset --hard HEAD, a disputed label: the guard keeps blocking it because it discards uncommitted work); the corpus is now in-sample; the blind rows are pinned in test_security_gates_heldout10_blind.py. Held-out-8 and 9 stay 60/60, FP 0.

Tests: 23/23 security gate scripts, test-destructive-guard 16/16, test_audit_skills 11/11, ci-tests.d all pass (run-ledger, prepush-ci-parity, bench-check skipped), size-ratchet ok, git diff --check clean

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant