Skip to content

Guard: container and remote passthrough, system-file writes, nested decode-exec; blind held-out-6 and 7 - #1991

Closed
remigiusz-antczak wants to merge 1 commit into
mainfrom
land/guard-w34
Closed

remigiusz-antczak wants to merge 1 commit into
mainfrom
land/guard-w34

Conversation

@remigiusz-antczak

Copy link
Copy Markdown
Collaborator

This extends the destructive guard to commands hidden inside containers, remote shells, system-file appends and nested decoders, and scores it on two fresh blind corpora that the rule author never read.

Held-out-6 (blind, 60 dangerous / 60 benign) Blocked Benign blocked
Perun guard, this PR 45/60 2/60
Perun guard, #1990 alone (879a0c1) 28/60 2/60
Perun guard, main before #1990 (e42fed4) 24/60 6/60
claude-code-hooks-mastery, ported 15/60 8/60
claude-code-safety-net, ported subset 11/60 (1 unported) 2/60
  • Held-out-7 (prereg/heldout7, b626135, pushed before scoring) gives a second blind result for the final rules: 51/60 blocked with 1/60 benign blocked. For comparison, main scores 46/60 with 2/60, claude-code-safety-net 13/60 with 1/60, and hooks-mastery 18/60 with 9/60. Its 9 misses are five Windows deletes and formats, plus rm under nohup, watch, xargs and parallel.
  • Held-out-6 was preregistered and pushed before scoring (prereg/heldout6, 1bcd79f). The blind score before the review fixes was also 45/60 with 2/60 benign blocked.
  • The 15 remaining misses are listed in its RESULTS.md. Windows and PowerShell deletes and formats account for 5 of them. Others: vault, gh api -X DELETE, glab, mongosh deleteMany, reflog expire plus gc, cp onto /etc files, openssl enc piped to a shell, and kubectl exec ... bash -c "dropdb".
  • Held-out-5 is now in-sample for the guard, at 59/60 with 0/60 benign blocked. Its blind 50/60 with 4/60 benign blocked is pinned by scripts/test_security_gates_heldout5_blind.py.

Rule changes in destructive_guard.py:

  • Inner command checked: docker, podman and nerdctl exec/run (image-first parsing, --entrypoint, compose); kubectl exec and oc exec, with or without --. Nesting past the depth cap fails closed.
  • Now blocked: >> and tee onto system paths (/root, /private/etc, and /var except /var/tmp and /var/folders), with .. and case variants normalized; nested decode-then-execute, including openssl enc and python base64 decoders. Also from held-out-5: gh ... delete, mv to /dev/null, vault kv metadata delete, curl -X DELETE, brew uninstall --ignore-dependencies, podman system reset, and eval of an echoed command.
  • Now allowed: extension-scoped find -delete under the current directory, mkfs.ext* -n, sqlite3 :memory: without attach or .shell, and rmtree(tempfile.mkdtemp()).

The first independent review found 4 🔴, 5 🟡 and 1 ❓, all fixed:

  • docker run --entrypoint hid the inner command.
  • Unknown flags pushed the command out of view.
  • Recursion depth was hard-coded.
  • The find -delete exemption covered any root.

The re-review confirmed those fixes and found no ReDoS (0.15 s worst case on 100 KB input). It also found 1 🔴 and 3 🟡 that main had too. This PR closes them as wave 4:

  • sqlite3 .shell, .system, .sh and .read | commands, including via -cmd, a heredoc or a herestring, are now checked as shell commands. ATTACH, .open and .output of a system path are refused.
  • The inner command of kubectl run|debug -- CMD, oc debug and oc rsh is checked.
  • rm under /private/{etc,var,root,usr,opt,home,Users} is treated like the unaliased path. /private/var/folders and /private/var/tmp stay scratch.
  • A write target such as $HOME/../../etc/x is judged by what remains after the .. run.

Held-out-6 still scores 45/60 with 2/60 benign blocked with the wave-4 rules.

The wave-4 review found 0 🔴 and 5 🟡. All five are folded into the next guard wave:

  • Quoted .read '|cmd'.
  • ATTACH with parenthesized, concatenated or file: URI paths.
  • $D/./.. and mid-path .. write targets, and cp onto them.
  • chroot /host rm under oc debug node.
  • kubectl run --overrides.

It also noted a pre-existing slow path of 2.6 s on a 20,000-newline sqlite3 argument; the new regexes are linear.

Local gates:

  • test_security_gates*.py: 11/11 pass. Pins moved only in the improving direction. One run of the held-out-4 timing pin (1 s bound) failed while the host load average was about 71; three reruns passed, and the slowest blob measures 0.68 s under that load.
  • test-destructive-guard.sh: 16/16.
  • bench_check --changed: 2 dirs clean. Size-ratchet: ok. Privacy: clean. git diff --check: clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z

…ecode-exec; blind held-out-6 and 7

destructive_guard.py now checks the inner command of docker/podman/nerdctl exec|run (image-first parsing, --entrypoint, compose) and kubectl/oc exec, with or without --, and fails closed past the recursion depth cap. It blocks >> and tee onto system paths (adds /root, /private/etc and all of /var except /var/tmp and /var/folders; `..` and case variants are normalized), scans decode-exec substitutions to the matching paren, adds openssl enc and python base64 decoders, and fails closed at the decode caps only when a decoder matches somewhere. From held-out-5 it also blocks gh delete, mv to /dev/null, vault kv metadata delete, curl -X DELETE, brew uninstall --ignore-dependencies, podman system reset and eval of an echoed command, and allows extension-scoped find -delete under the current directory, mkfs.ext* -n, sqlite3 :memory: without attach or .shell, and rmtree(tempfile.mkdtemp()).

Wave 4 closes gaps that main also had: sqlite3 .shell/.system/.sh/.read | commands (also via -cmd, heredoc or herestring) are checked as shell commands, and ATTACH/.open/.output of a system path is refused; the inner command of kubectl run/debug, oc debug and oc rsh is checked; rm under /private/{etc,var,root,usr,opt,home,Users} is treated like the unaliased path (/private/var/folders and /private/var/tmp stay scratch); and a write target built from an unexpanded variable or ~ followed by .. segments is judged by what remains, so $HOME/../../etc/x is refused. Held-out-6 still scores 45/60, 2/60 benign blocked with these rules.

Held-out-6 (preregistered on prereg/heldout6 at 1bcd79f, pushed before scoring; not read by the rule author) is the blind result: 45/60 blocked, 2/60 benign blocked, unchanged after the review fixes. Main before #1990 scored 24/60 (FP 6), #1990 alone 28/60 (FP 2), the claude-code-safety-net subset 11/60 (FP 2) and hooks-mastery 15/60 (FP 8). Held-out-7 (preregistered on prereg/heldout7 at b626135, pushed before scoring) is a second blind result: 51/60 blocked, 1/60 benign blocked, against main 46/60 (FP 2), safety-net 13/60 (FP 1) and hooks-mastery 18/60 (FP 9). Held-out-5 is now in-sample at 59/60, FP 0/60; its blind 50/60, FP 4/60 row is pinned separately.

Tests: 11/11 security gate scripts, test-destructive-guard 16/16, size-ratchet, bench_check green

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z
@remigiusz-antczak

Copy link
Copy Markdown
Collaborator Author

Superseded by #1993, which squashes this branch with guard wave 5 and the audit-skills triage that fixes this PR's CI failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant