Repository navigation
Guard: container and remote passthrough, system-file writes, nested decode-exec; blind held-out-6 and 7 - #1991
Closed
remigiusz-antczak wants to merge 1 commit into
Closed
remigiusz-antczak wants to merge 1 commit into
remigiusz-antczak wants to merge 1 commit into
Conversation
…ecode-exec; blind held-out-6 and 7
destructive_guard.py now checks the inner command of docker/podman/nerdctl exec|run (image-first parsing, --entrypoint, compose) and kubectl/oc exec, with or without --, and fails closed past the recursion depth cap. It blocks >> and tee onto system paths (adds /root, /private/etc and all of /var except /var/tmp and /var/folders; `..` and case variants are normalized), scans decode-exec substitutions to the matching paren, adds openssl enc and python base64 decoders, and fails closed at the decode caps only when a decoder matches somewhere. From held-out-5 it also blocks gh delete, mv to /dev/null, vault kv metadata delete, curl -X DELETE, brew uninstall --ignore-dependencies, podman system reset and eval of an echoed command, and allows extension-scoped find -delete under the current directory, mkfs.ext* -n, sqlite3 :memory: without attach or .shell, and rmtree(tempfile.mkdtemp()).
Wave 4 closes gaps that main also had: sqlite3 .shell/.system/.sh/.read | commands (also via -cmd, heredoc or herestring) are checked as shell commands, and ATTACH/.open/.output of a system path is refused; the inner command of kubectl run/debug, oc debug and oc rsh is checked; rm under /private/{etc,var,root,usr,opt,home,Users} is treated like the unaliased path (/private/var/folders and /private/var/tmp stay scratch); and a write target built from an unexpanded variable or ~ followed by .. segments is judged by what remains, so $HOME/../../etc/x is refused. Held-out-6 still scores 45/60, 2/60 benign blocked with these rules.
Held-out-6 (preregistered on prereg/heldout6 at 1bcd79f, pushed before scoring; not read by the rule author) is the blind result: 45/60 blocked, 2/60 benign blocked, unchanged after the review fixes. Main before #1990 scored 24/60 (FP 6), #1990 alone 28/60 (FP 2), the claude-code-safety-net subset 11/60 (FP 2) and hooks-mastery 15/60 (FP 8). Held-out-7 (preregistered on prereg/heldout7 at b626135, pushed before scoring) is a second blind result: 51/60 blocked, 1/60 benign blocked, against main 46/60 (FP 2), safety-net 13/60 (FP 1) and hooks-mastery 18/60 (FP 9). Held-out-5 is now in-sample at 59/60, FP 0/60; its blind 50/60, FP 4/60 row is pinned separately.
Tests: 11/11 security gate scripts, test-destructive-guard 16/16, size-ratchet, bench_check green
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z
Collaborator
Author
|
Superseded by #1993, which squashes this branch with guard wave 5 and the audit-skills triage that fixes this PR's CI failure. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This extends the destructive guard to commands hidden inside containers, remote shells, system-file appends and nested decoders, and scores it on two fresh blind corpora that the rule author never read.
prereg/heldout7, b626135, pushed before scoring) gives a second blind result for the final rules: 51/60 blocked with 1/60 benign blocked. For comparison, main scores 46/60 with 2/60, claude-code-safety-net 13/60 with 1/60, and hooks-mastery 18/60 with 9/60. Its 9 misses are five Windows deletes and formats, plusrmundernohup,watch,xargsandparallel.prereg/heldout6, 1bcd79f). The blind score before the review fixes was also 45/60 with 2/60 benign blocked.gh api -X DELETE, glab, mongoshdeleteMany, reflog expire plus gc,cponto/etcfiles,openssl encpiped to a shell, andkubectl exec ... bash -c "dropdb".scripts/test_security_gates_heldout5_blind.py.Rule changes in
destructive_guard.py:exec/run(image-first parsing,--entrypoint, compose);kubectl execandoc exec, with or without--. Nesting past the depth cap fails closed.>>andteeonto system paths (/root,/private/etc, and/varexcept/var/tmpand/var/folders), with..and case variants normalized; nested decode-then-execute, includingopenssl encand python base64 decoders. Also from held-out-5:gh ... delete,mvto/dev/null,vault kv metadata delete,curl -X DELETE,brew uninstall --ignore-dependencies,podman system reset, andevalof an echoed command.find -deleteunder the current directory,mkfs.ext* -n,sqlite3 :memory:withoutattachor.shell, andrmtree(tempfile.mkdtemp()).The first independent review found 4 🔴, 5 🟡 and 1 ❓, all fixed:
docker run --entrypointhid the inner command.find -deleteexemption covered any root.The re-review confirmed those fixes and found no ReDoS (0.15 s worst case on 100 KB input). It also found 1 🔴 and 3 🟡 that main had too. This PR closes them as wave 4:
.shell,.system,.shand.read |commands, including via-cmd, a heredoc or a herestring, are now checked as shell commands.ATTACH,.openand.outputof a system path are refused.kubectl run|debug -- CMD,oc debugandoc rshis checked./private/{etc,var,root,usr,opt,home,Users}is treated like the unaliased path./private/var/foldersand/private/var/tmpstay scratch.$HOME/../../etc/xis judged by what remains after the..run.Held-out-6 still scores 45/60 with 2/60 benign blocked with the wave-4 rules.
The wave-4 review found 0 🔴 and 5 🟡. All five are folded into the next guard wave:
.read '|cmd'.ATTACHwith parenthesized, concatenated orfile:URI paths.$D/./..and mid-path..write targets, andcponto them.chroot /host rmunderoc debug node.kubectl run --overrides.It also noted a pre-existing slow path of 2.6 s on a 20,000-newline sqlite3 argument; the new regexes are linear.
Local gates:
test_security_gates*.py: 11/11 pass. Pins moved only in the improving direction. One run of the held-out-4 timing pin (1 s bound) failed while the host load average was about 71; three reruns passed, and the slowest blob measures 0.68 s under that load.test-destructive-guard.sh: 16/16.bench_check --changed: 2 dirs clean. Size-ratchet: ok. Privacy: clean.git diff --check: clean.🤖 Generated with Claude Code
https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z