Skip to content

Union the gitleaks default rules into the privacy gate; blind held-out-3 - #1995

Closed
remigiusz-antczak wants to merge 1 commit into
mainfrom
land/union-secrets-d
Closed

remigiusz-antczak wants to merge 1 commit into
mainfrom
land/union-secrets-d

Conversation

@remigiusz-antczak

Copy link
Copy Markdown
Collaborator

This supersedes #1992, rebased onto main after #1993. It also fixes the two test-ci-gates.sh checks that failed CI on #1992: the frozen held-out-1024 secret-leaks.txt skip, and the missing awk-v-ok annotation.

This adds the gitleaks default rules to the privacy gate alongside .banlist.txt, so a file fails when either one matches. On held-out-3 the union catches more leaks than either layer alone, at the cost of about twice gitleaks' false-positive rate.

Held-out-3 (60 leaks / 60 decoys) Leaks caught Decoys flagged
Union, this PR 54/60 16/60
gitleaks v8.30.1 defaults, ported 47/60 8/60
.banlist.txt only (main) 41/60 12/60
  • Blindness is partial. The decision to union came after held-out-2. Two held-out-3 rows were given extra @@ markers before scoring, and the ledger discloses both. Treat the table as close to out-of-sample, not strictly blind.
  • False positives double against gitleaks. The extra 8 come from the banlist half, which main already has.
  • Implementation:
    • The gitleaks port moved from the rival bench into scripts/secret_scan.py; the gate and score_rivals.py both import it.
    • The vendored v8.30.1 config (MIT, pinned in SOURCES.md) sits next to it as data.
    • Frozen-corpus files that hold only placeholders are skipped by exact path plus sha256 in scripts/secret-scan-allow.txt, so the repo stays at 0 hits. A test proves that appending a key to a pinned file fails the gate.
    • Lines are cut to 4 KB before gitleaks matching; a 40,000-line timing test pins the cost.

The independent review raised 4 🔴 against the gitleaks layer. I probed each against the whole gate on this branch and on main:

  • A gitleaks:allow comment and an .svg or go.sum path both still fail the gate, because the banlist half matches them.
  • A key past column 4096 still fails when anything separates it from the preceding text.
  • A key glued to preceding letters at column 4090 passes on main and on this branch.
  • A file with a NUL byte in its first 8 KB is skipped as binary by both versions.

None of these is a regression. The NUL-byte skip and the unguarded allow file (🟡) are the next fixes for the secret gate.

Local gates:

  • scripts/ci-tests.d: 139/139. run-ledger.sh was excluded because it times out on lsof in the sandbox.
  • Privacy: 0 hits. Size-ratchet: ok. bench_check: clean. git diff --check: clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z

…lind on held-out-3

The privacy gate now flags a file when either .banlist.txt or the gitleaks default rules match. The gitleaks port moved out of the rival bench into scripts/secret_scan.py, which the gate and score_rivals.py both import; the vendored v8.30.1 config (MIT, pinned in SOURCES.md) moved next to it as data. The exact-path frozen-corpus skip is kept and extended to held-out-3; 15 frozen paths holding only placeholders or synthetic values are allowed by exact path in scripts/secret-scan-allow.txt, so the repo stays at 0 hits. The gitleaks half fans out over 8 processes; wall time on this repo was 426s against 659s without it in one noisy back-to-back run.

Held-out-3 (rules frozen, scored once, two rows given extra @@ markers before scoring and disclosed in the ledger): union 54/60 leaks FP 16/60, banlist only 41/60 FP 12/60, gitleaks 47/60 FP 8/60.

Tests: 139/139 ci-tests.d (run-ledger.sh excluded, sandbox lsof timeout), test-ci-gates.sh (privacy and awk-v checks pass; 3 sandbox-only fails: peer pains, reap_own pgrep, tracker hygiene; 7 dcr-gates opt-in need port bind), privacy, size-ratchet, bench_check pass

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QNFFPWVcPUbeuvwM5fk2Z
@remigiusz-antczak

Copy link
Copy Markdown
Collaborator Author

Superseded by #1996 (same commit rebased onto current main; branch protection requires up-to-date).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant