v2.0.0: 93% Recall at 8% Noise Rate (SecretBench), "Dark Matter" Coverage and Performance

DeepSecrets 2.0 is here! 🚀

Now tested against the SecretBench Benchmark with:

• 93% Recall
• 8% Noise Rate
• ~9K Extra Findings outside the benchmark scope

Improvements

Coverage

• Nested Formats Parsing: LexerTokenizer is now able to detect code nesting and correctly parse situations like "inline YAML inside YAML inside Markdown".
• CheapVarDetector: Detects potential variable declarations even in "unlexable" code.
• Edge-cases: Better variable extraction for edge cases in Shell, JavaScript, Markdown, PHP, and C#.
• Deeper Language Support: Expanded native tracking surface for R(d), Ruby, and Nix configurations.
• Tighter Regexes: Improved "classic" backup regexes for secrets from AWS, Stripe, MailChimp, and our favorite -----BEGIN constructions.

Precision

• Confidence Scoring: Every finding candidate is dynamically scored using a system that evaluates naming layouts, value entropy, and semantic "naturalness" to drop false positive rate.

Performance and Stability

• ~30% faster and more reliable for large files (up to 200 MB) with rich semantics.
• The UI now shows the progress and estimates for each file, as well as the overall progress.

Important Changes

Switching to SARIF reports

Warning

The legacy JSON report format is now deprecated and will be removed in the next major release. For now, you can still select it via --outformat json, which will trigger a deprecation warning.

We are switching to the industry-standard SARIF (v2.1.0) format to provide seamless integration with orchestration, CI/CD pipelines, and ASPM systems like GitHub Security and DefectDojo:

• Virtual Subrules: (e.g., S105-LOW) to enforce proper precision and security-severity inside third-party dashboards.
• Smart Tracking (partialFingerprints): Populates fingerprints of findings. Even with automatic masking enabled, downstream systems can track moving secrets without duplicating them.

Warning

Breaking Change for Existing Alerts:
If you have used DeepSecrets before and already have a set of deduplicated findings in GitHub or DefectDojo, switching to Virtual Subrules will dynamically alter rule IDs (e.g., changing them to S105-LOW or S105-HIGH). This will likely cause your platforms to treat them as new issues, creating a one-time wave of duplicate alerts. I am truly sorry for this temporary inconvenience, but this change is vital for proper semantic precision mapping going forward.

📦 Quick Upgrade

pip install --upgrade deepsecrets

Full Changelog: v1.4.0...v2.0.0

v1.4.0: New UI

What's Changed

Improvements

Updated UI

• Progress Bars for every file for better verbosity

• Updated Report view

• Improved parsing of config files
• Fixes for Dangerous Variable Detection
  
  
  

Full Changelog: v1.3.0...v1.4.0

v1.3.0: SARIF support and masking by default

Improvements

• Added support for SARIF output format (--outformat dojo-sarif). The tool is now compatible with DefectDojo (#9, kudos to @dmarushkin)
• Secrets inside reports are now masked by default (Use --disable-masking to control this behaviour but be careful)
• Added support for Python 3.13

Bugfixes

• Updated dependencies, including pydantic (#10 )

Full Changelog: v1.2.0...v1.3.0

v1.2.0

Improvements

• The default multiprocessing context has been changed from fork (fast but unstable) to spawn (slower but more stable) to address the tool freezing issue (#5)

Warning

This fix has made the tool a bit slower. It's okay.

• Added a handle for multiprocessing context choice (--multiprocessing-context)
• The debug mode now shows the current progress of processed tokens once a second to indicate the liveness of the tool.
• Added a handle to control the maximum size of a file for analysis (--max-file-size)
• Better logging

Bugfixes

• Fixed an issue causing a crash while processing a ruleset with predefined false findings.

Full Changelog: v1.1.4...v1.2.0

v1.1.4

Bugfixes

• Fix for "Password in URI" rule in the built-in ruleset #4
• Improved docs on "excluded paths" #3

Minor changes

• Support for Python 3.12
• Updated dependencies

Full Changelog: v1.1.3...v1.1.4

v1.1.3

Cumulative update with a bunch of improvements

• Reduced false positive rate with 'key-value' pairs
• More variable suppression rules for Golang