Repository navigation
fix(cli): redact secret-shaped values from the model bundle - #39
Merged
Merged
Conversation
Problem: buildBundle copied up to 10 hotspot files into the prompt bundle as they were, about 60,000 characters. A key hard-coded in a hotspot file went to the model provider, to the --via agent CLI, or into .complex-md/prompt.md for the user's agent. Add cli/src/redact.js, with no dependency: 10 rules for known token formats (AWS, GitHub, OpenAI, Anthropic, Slack, Stripe, Google, private key blocks, JSON web tokens) and 1 assignment rule that keeps a value only when its entropy is 3.5 bits per character or more and it holds a digit. Each match becomes [redacted:<rule>]. excerpt() redacts each file whole before the cut, and a second pass covers the rest of the bundle. generate prints 1 line with the count per rule, never a value. The front matter that generate writes never passes through redaction. Tests: 10 in redact.test.js (each rule, near misses, the CLI's own source unchanged, the template path), 1 in generate.test.js on the provider request, 1 in generate-via.test.js on the stub CLI's stdin. npm test: 806 tests, 0 failures. Intent: keep hard-coded credentials in hotspot files out of every third party that reads the bundle. Assisted-by: Claude
Problem: --help, cli/README.md, SECURITY.md and the site FAQ said the model call carries the hotspot excerpts, with no word on the values that the previous commit now replaces. Each text now says that each secret-shaped value goes as [redacted:<rule>]. The README names the rule families and says that a secret in another shape is sent as it is. truth, site and home tests: 60 of 60 pass. npm run build at the root passes. Intent: describe exactly what the model call carries. Assisted-by: Claude
|
COMPLEX.md: 1 hotspot touched, 3 co-change partners not in this pull request.
Co-change partners not in this pull request
Read from |
2 of 3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Problem:
buildBundleincli/src/generate.jscopied up to 10 hotspot files into the prompt bundle as they were, about 60,000 characters. A key hard-coded in a hotspot file went to the model provider, to the agent CLI that--vianames, or into.complex-md/prompt.mdfor the user's agent.Change:
cli/src/redact.js, with no dependency. It replaces each secret-shaped value with[redacted:<rule>]. 11 rules:aws-access-key:AKIA,ASIA,ABIA,ACCAandA3Tprefixes, 16 characters of[A-Z2-7].github-token:ghp_,gho_,ghu_,ghs_,ghr_with 36 characters, andgithub_pat_with 22 and 59.anthropic-key:sk-ant-<kind><2 digits>-and 32 or more characters.openai-key: theT3BlbkFJmarker, andsk-proj-,sk-svcacct-,sk-admin-with 40 or more characters.slack-token(xoxb-and the otherxoxkinds,xapp-) andslack-webhook(hooks.slack.com).stripe-key:sk_live_andrk_live_.google-api-key:AIzaand 35 characters.private-key: aBEGIN ... PRIVATE KEYblock to itsENDline, or to the end of the base64 when the END line is missing.jwt: 3 base64url parts, the first 2 starting witheyJ.generic-assignment(heuristic): a value assigned with=,:=,:or=>to a name that holds key, secret, token, password, pwd or credential. The value is replaced only when its Shannon entropy is 3.5 bits per character or more and it holds a digit and a letter. An environment lookup, a template slot, a placeholder word, a URL, and a lowercase lookup key of short parts (such asweb-article-v1) stay. Only the value is replaced, so the name and the quotes stay.The token formats and expressions are written for this project.
excerpt()redacts each file whole before it cuts the excerpt, so a cut never leaves half of a key that no rule matches. A second pass covers the rest of the bundle (paths, blind spots).buildBundlereturnsredacted(counts per rule).generateprints 1 line when the count is above 0, for exampleredacted 3 secret-shaped values in the prompt bundle (aws-access-key 1, generic-assignment 2); each is sent as [redacted:<rule>].The line never holds a value.generatewrites is computed from the signals and never passes through redaction. Template prose does not read the excerpts, so the template path writes the same map.--help,cli/README.md(Privacy),SECURITY.mdand the FAQ incontent/index.html. The README also says that a secret in another shape is sent as it is.Co-change partners:
cli/bin/complex-md.jschanged withcli/src/generate.js(the help text).cli/README.mdchanged, andcontent/spec.mddid not: its 2 sentences on the model call (the "One tool, two entry points" list and the FAQ) stay true, and a change to the spec text needs its own source.Limits: redaction matches shapes, so a secret in a format with no rule, or a short or low-entropy password, is still sent. A credential file by path is already dropped from the map by
cli/src/secrets.js.How it was tested
cd cli && npm test: exit 0, 806 tests, 804 pass, 0 fail, 2 todo (the 2 todo tests were there before).cd cli && node --test test/truth.test.js test/site.test.js test/home.test.js: 60 of 60 pass.npm run buildat the root: passes.cli/test/redact.test.js(10 tests):cli/srcandcli/bin, and the generation prompt, pass through with 0 replacements;buildBundleon a repository whose top hotspot holds every fake value: no original value in the bundle, every rule placeholder present, the computed front matter verbatim;generate --templatewrites the same map astemplateMap, and the saved.complex-md/prompt.mdholds no original value.cli/test/generate.test.js: a stub provider records the request. The request holds no original value, the run prints 1 count line, and the written map has the computed front matter.cli/test/generate-via.test.js: the stubclaudeCLI now saves its stdin. The stdin holds no original value.cli/test/fixtures/fake-secrets.mjsare plainly fake (FAKE, fake, 0123) and built from fragments at run time, so no whole token is in the repository.AI assistance
Claude (Claude Code) wrote the code, the tests, the text changes and this description.
Assisted-by:or aCo-Authored-By:trailer.🤖 Generated with Claude Code