Skip to content

Guard wave 6: cloud, VCS, Windows and shell-split deletes; blind held-out-9 and 10 - #1994

Merged
remigiusz-antczak merged 1 commit into
mainfrom
land/guard-w6
Oct 25, 2026
Merged

remigiusz-antczak merged 1 commit into
mainfrom
land/guard-w6

Conversation

@remigiusz-antczak

Copy link
Copy Markdown
Collaborator

Wave 6 closes the guard misses from held-out-8 and held-out-9 and the gaps the #1993 reviews folded forward. A fresh blind corpus (held-out-10, written after these rules) shows where the next wave has to go. Neither guard version moves on held-out-10, and both still block about three times as many dangerous commands as the ported rivals.

Held-out-10 (blind, 60 dangerous / 60 benign) Blocked Benign blocked
Perun guard, this PR 44/60 6/60
Perun guard, main (49162c7) 44/60 6/60
claude-code-safety-net, ported subset 14/60 (1 unported) 4/60
claude-code-hooks-mastery, ported 13/60 9/60
  • Preregistered. Held-out-10 was pushed before scoring (prereg/heldout10, 4a5cd20). The corpus is not part of this PR; it lands with wave 7.
  • Held-out-10 misses (16):
    • rclone purge, delete and sync-from-empty;
    • restic forget --prune;
    • zfs destroy, zpool destroy and lvremove;
    • firebase, supabase, vercel and netlify deletes;
    • a flag hidden in a variable (f=-rf; rm $f ...);
    • (cd /srv && rm -rf acme).
  • Held-out-10 false positives (6):
    • find ./build -name '*.o' -delete;
    • find . -name '*.pyc' -exec rm -f {} +;
    • echo ./build | xargs -I{} rm -rf {};
    • wipefs /dev/sdb without -a, which only lists signatures;
    • shred -u of a file in /tmp;
    • git reset --hard HEAD.
  • Held-out-9 is now in-sample. Its blind result, scored on the Guard waves 3-5: container, remote, Windows and wrapper passthrough; blind held-out-6 to 9 #1993 rules before this wave, was 55/60 blocked with 0/60 benign blocked (main before Guard waves 3-5: container, remote, Windows and wrapper passthrough; blind held-out-6 to 9 #1993: 48/60). That row is pinned in test_security_gates_heldout9_blind.py. After this wave, held-out-8 and held-out-9 both score 60/60 with 0/60 benign blocked; those numbers are tuned and are not a generalization claim.

Rule changes:

  • Now blocked:
    • tmutil delete; cdk, cdktf and sst destroy, including through npx, pnpm dlx, yarn and bunx; doctl, flyctl and heroku deletes;
    • hg strip and purge with global flags; make clean with a root variable in either position;
    • set -- rm ...; "$@" and IFS-split commands;
    • aws kms schedule-key-deletion behind aws global flags;
    • gem uninstall -aIx; diskutil apfs delete and erase in any case;
    • kubectl run with hostPID, privileged or hostPath overrides, behind kubectl global flags;
    • sqlite3 .read |sh; ln and rsync onto system paths;
    • vault lease and token revoke and policy delete; git reflog delete; reflog expire to now in the =, space and -c gc.reflogExpire forms;
    • chroot -u; mongosh remove and deleteOne; clickhouse DROP;
    • wmic call create and uninstall; reg add, copy and save on HKLM;
    • rm -r under /Library and other system directories;
    • Windows del of system files, .. paths, and a bare * or *.*;
    • xargs rm fed by a .. source.
  • Now allowed:
    • extension-scoped find . -name '*.ext' -delete;
    • cp to /etc/*.bak;
    • reflog expire with an age other than now;
    • del /s /q *.log in the current directory.
  • Speed: the set -- scan is linear. A 100 KB adversarial input now takes 0.1 s, down from 1.04 s.

Review: 0 🔴 and 6 🟡, every one confirmed by a probe. Fixed in this PR:

  • kubectl global flags before run;
  • package-runner wrappers for cdk;
  • aws and hg global flags;
  • DIR=/ make clean;
  • case-insensitive diskutil verbs;
  • the space form of reflog expire;
  • .. sources feeding xargs rm;
  • the quadratic set -- scan.

Folded into wave 7:

  • cp exemptions for any .bak, .orig or .old suffix on a system path;
  • ln -sf onto files in the home directory;
  • rsync to a remote /etc;
  • vault operator seal;
  • reg add HKU|HKCU;
  • wmic /node: ... reboot;
  • clickhouse ALTER ... DELETE and SQL fed through stdin;
  • make nuke|uninstall DESTDIR=/;
  • mongosh bulkWrite.

Also folded: the blind pin only checks the text in RESULTS.md and does not re-score.

Local gates:

🤖 Generated with Claude Code

https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z

…-out-9

The destructive guard now blocks tmutil delete, cdk/cdktf/sst destroy, doctl/flyctl/heroku deletes, hg strip/purge, make clean with a root variable, `set -- rm ...; "$@"` (also IFS-split), aws kms schedule-key-deletion, gem uninstall of everything, diskutil apfs delete/erase, kubectl run with hostPID/privileged/hostPath, sqlite3 `.read |sh`, ln/rsync onto a system path, vault lease/token revoke and policy delete, git reflog delete, chroot -u, mongosh remove/deleteOne, clickhouse DROP, wmic call create/uninstall, reg add/copy/save on HKLM, rm -r under /Library and other system dirs, and Windows `del` of system files, `..` paths and a bare `*` or `*.*`. Allowed stay: find ./dist -name '*.map' -delete, cp to /etc/*.bak, git reflog expire with a non-now age, del /s /q *.log in the cwd.

Held-out-9 (prereg/heldout9 ffb22e7, pushed before scoring): #1993 guard 55/60 blocked, 0/60 benign blocked; main before #1993 48/60, 0/60; hooks-mastery 8/60 (FP 2); safety-net 9/60 (FP 0, 1 unported). Held-out-8 and 9 are in-sample at 60/60, FP 0 after this wave; their blind rows are pinned in new *_blind tests.

Tests: 20/20 security gate scripts, test-destructive-guard 16/16, test_audit_skills 11/11, ci-tests.d all pass (run-ledger, prepush-ci-parity, bench-check skipped), size-ratchet ok, test-ci-gates 162 PASS with 4+ dcr-gates opt-in FAIL from sandbox EPERM on port bind (surface_check selftest), git diff --check clean

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z
@remigiusz-antczak
remigiusz-antczak merged commit 9e22d2a into main Oct 25, 2026
1 check passed
@remigiusz-antczak
remigiusz-antczak deleted the land/guard-w6 branch October 25, 2026 06:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant