Bench: rival head-to-head on the security-gates corpus (ported rules)

.gitattributes

@@ -40,3 +40,4 @@ docs/bench/comms-e4-confirm-1019/**/outputs/** -whitespace
 docs/bench/comms-e4-confirm-1019/**/reads/** -whitespace
 docs/bench/comms-e4-confirm-1019/**/grades/** -whitespace
 docs/bench/comms-e4-confirm-1019/judge/** -whitespace
+docs/bench/security-gates-1024/rivals/* -whitespace

changelog.d/rival-h2h.md

@@ -0,0 +1,2 @@
+### Added
+- Rival head-to-head on the frozen security-gates corpus (`docs/bench/security-gates-1024/rivals/score_rivals.py`, pins in `SOURCES.md`): rules of gitleaks v8.30.1 (22 of 31 leaks, 0 of 34 false positives), hooks-mastery (13 of 46 dangerous, 5 of 45 benign blocked) and a documented subset of claude-code-safety-net (ported-subset result 27 of 46, 4 rows unported, 0 of 45 benign) ported to Python, against Perun's 30 of 31, 1 of 34, 46 of 46 and 0 of 45. Rules ported to Python, not the rival binaries; ported-rule fidelity is the main threat. README and `docs/perun-vs-rivals.md` rows updated; score pins run in CI via `scripts/test_rival_h2h.py`.

docs/bench/security-gates-1024/README.md

@@ -9,7 +9,9 @@ Frozen synthetic corpora for the H5 (secret gate) and H8 (destructive-command gu
 | destructive-dangerous.txt | 46 dangerous command strings, including interpreter, IFS, brace, quote-split and wrapper tricks |
 | destructive-benign.txt | 45 benign command strings |
 
-Run: `python3 docs/bench/security-gates-1024/score.py` (offline, deterministic, nothing executed). It fails on hash drift against the SHA256 lines below. Rival tools (gitleaks and others) were not run: not installed, owner gate.
+Run: `python3 docs/bench/security-gates-1024/score.py` (offline, deterministic, nothing executed). It fails on hash drift against the SHA256 lines below. Rival rules are scored by `python3 -I docs/bench/security-gates-1024/rivals/score_rivals.py` (pins and gaps in `rivals/SOURCES.md`).
+
+Caveat for the rival rows: rules ported to Python, not the rival binary; ported-rule fidelity is the main threat. Nothing downloaded was executed (data only). The safety-net port is a subset scored three-state (block, allow, unported); its number is a ported-subset result, not the tool's score. Unported rule families: heredocs, PowerShell and cmd, awk, parallel, policy files, strict and paranoid modes, worktree and temp-root relaxations, shell variable assignment tracking, filesystem-dependent path resolution, the raw-text fallback, the interpreter exec-sink check, `git rm`, `git rebase --abort`, `git reflog delete`, disk-erase tools and nested `$(...)` analysis. hooks-mastery targets only `rm -rf` and `.env` by design, so 13/46 is not a general quality comparison. `go2py` keeps Python's Unicode `\d`, `\w`, `\s` and `\b` where Go RE2 uses ASCII, which slightly favours gitleaks.
 
 Caveat: Perun fixes in the same change post-date this corpus, so the after-numbers are in-sample. The guard is a best-effort string check, not a security boundary.
 
@@ -30,6 +32,17 @@ d718d24caf1112adcfccc4bce1ea94a5170bbe2b0512d3f42919bbb060382d16 destructive-ben
 | Destructive guard blocked | 29/46 | 46/46 |
 | Destructive guard false positives | 0/45 | 0/45 |
 
+Rival rows (ported rules, same corpus, `rivals/score_rivals.py`):
+
+| Tool (pinned) | Secret recall | Secret FP | Destructive blocked | Destructive FP |
+|---|---|---|---|---|
+| Perun | 30/31 | 1/34 | 46/46 | 0/45 |
+| gitleaks v8.30.1 default config | 22/31 | 0/34 | n/a | n/a |
+| claude-code-safety-net (subset port) | n/a | n/a | 27/46 ported-subset result (4 rows unported) | 0/45 |
+| hooks-mastery (exact port) | n/a | n/a | 13/46 | 5/45 |
+
+gitleaks misses 9: 5 fixtures are off the provider's real format (AWS keys with non-base32 digits, 35-character `ghs_`, short fine-grained PAT, short Anthropic key), 3 are header-only PEM lines, 1 is a connection URL. Decode-and-rescan and path rules are not ported.
+
 Remaining secret miss: `gh_ghs` is a 35-character `ghs_` fixture (the real format is 36), kept frozen and counted as a miss. The AWS documentation key stays a false positive because an ERE cannot exclude the `EXAMPLE` suffix. Held-out variants and decoys for each fix are in `scripts/test_security_gates_corpus.py`. The older 32-string guard corpus in `scripts/test-destructive-guard.sh` went from 30/32 to 31/32 (a secret piped to `curl -d @-` is still allowed).
 
 Freeze note: the before-fix scores were taken on the first draft. Afterward two secret lines (`aws_secret`, `generic_json`) gained `@@` markers so the files pass the repo privacy gate; the scorer strips markers, so the scored text is unchanged. `SHA256-frozen.txt` and `ledger.jsonl` were written after the fixes (see the ledger note); they pin the corpora from here on.

docs/bench/security-gates-1024/rivals/SOURCES.md

@@ -0,0 +1,17 @@
+# Rival rule sources (data only, never executed)
+
+Rules ported to Python, not the rival binary; ported-rule fidelity is the main threat. Fetched 2026-10-24 over HTTPS from raw.githubusercontent.com at pinned tags or commits. `score_rivals.py` reads only the two vendored files below.
+
+| Rival | Pin | Source | sha256 |
+|---|---|---|---|
+| gitleaks default config | tag v8.30.1 | https://raw.githubusercontent.com/gitleaks/gitleaks/v8.30.1/config/gitleaks.toml (vendored as `gitleaks-v8.30.1.toml`, 97731 bytes upstream; the 17 `AIza` occurrences (16 public sample Google keys in an allowlist, 1 rule regex) carry `@@` markers after `AI` so the repo privacy gate passes, and deleting every `@@` restores the upstream bytes) | e163e53b9e7e8a8511e77271e2b323ed057759542a6d988258afe3a1fa329caf |
+| hooks-mastery `pre_tool_use.py` | commit 052ad1cbd5aeb1ec4a1def22012d1293c6225625 | https://raw.githubusercontent.com/disler/claude-code-hooks-mastery/052ad1cbd5aeb1ec4a1def22012d1293c6225625/.claude/hooks/pre_tool_use.py (vendored as `hooks-mastery-pre_tool_use.py.txt`) | 78006866f793ccd394bc52011582ce48707ceef9d3496e474ab7bcb63365a5da |
+| claude-code-safety-net (read, not vendored) | commit 71baf747e2a5246bdf613a2894f2d6e7a5ad80bf | https://github.com/kenryu42/claude-code-safety-net, files under `src/` listed below | see below |
+
+`src/` files read for the safety-net port (sha256 at the pinned commit): `core/rules/constants.ts` d1228cac292aad14b17f4bfc8cbf71fde852ba907872b437145b7ce0292225cd; `core/rules/destructive.ts` 73792ec819e890dcb97ee2a71485fabd104966b4e9e225c0c8af04287df6e82f; `gate/analyzer/derived-input.ts` 1d7d603589d3b1d5bb7eac5bc1e1537f939a4f3414a665239b1b4a6fab5d60e2; `gate/analyzer/device.ts` db9cb6772b368ed74f6cde4bf25c65c45e31378eea6100f8449dbb979950b8fa; `gate/analyzer/find.ts` 18070b2193d3a9f69e6b1c5d523b489553e0f5b817b993cb75d706c45b75710f; `gate/analyzer/git/rules.ts` 8b9fda4189581f23910095ad690d3b3ea1a7f8e4ead6af9f71fcd94eeda0aad8; `gate/analyzer/interpreters.ts` abb0363f2744f5724c914f218ac52e44a709ad8bb72f1b5723ff145a9e47c09b; `gate/analyzer/linear-danger-scanner.ts` 3967d3d4aa4d4786a75ee53c3ad655db0d30ced34870031fe165ec7306044708; `gate/analyzer/recursive-delete-targets.ts` 3887f945708153191e27d3ac4f97b99ad49b1bc0e4f9f77fca0bb7f1d863ec6f; `gate/analyzer/rm.ts` 414ad4312999587e41cd000caf9730815b887b1c3c03cf80e64c09a9ae7ee4bf; `gate/analyzer/rsync.ts` ff540f8fed8a3c1f5d27516e48e885061a2816040728f30688f9a48a5f4aad48; `gate/analyzer/segment.ts` b9abfe4d8108c539e276a1324b55df3bc998197a0afe1f3112191fe9822d0a7b; `gate/analyzer/wrapper-prelude.ts` a403df0e67d17631e31c9909f7387b988db81f3bb6546f09f830a3035cfce3e5; `gate/analyzer/xargs.ts` 8fcbf29f6b9ffd3a4880afdb1451665d802b7c136d65f7fa92b7364a38d4200d.
+
+## Known gaps in the ports
+
+- gitleaks: no decode-and-rescan (base64, hex, percent), no file-type path rules, no commit or git-history context; fragments are scored as a file body ending in a newline; Go regex features are translated to Python `re` and a test fails if any pattern stops compiling.
+- hooks-mastery: none for the two checks that file implements; it has no other command rules, so every non-`rm` dangerous row is a rival miss by construction.
+- safety-net (subset, default non-strict mode): ported are `rm` target classification, the git subcommand rules, `find -delete` and `-exec rm`, `xargs`, `rsync --delete`, `shred`, `dd`, `mkfs`, block-device redirects, shell `-c`, `eval`, shell reading a pipe, and interpreter one-liners (raw `rm` text and native delete of root or home). Not ported: heredocs, PowerShell and cmd, awk, parallel, policy files, strict and paranoid modes, worktree and temp-root relaxations, shell variable assignment tracking, filesystem-dependent path resolution (a fictional cwd is assumed), the raw-text fallback for unparseable input, the exec-sink literal check in interpreters, `git rm`, `git rebase --abort`, `git reflog delete`, disk-erase tools, and nested `$(...)` analysis. Rows whose command word is built from a variable, brace or `${IFS}` are scored `unported` because the matching rules are strict-mode only and shell-assignment tracking is not ported.