fix: acceptance tests for secrets test output [PS-357]

[alexandru-manea-snyk]

Pull Request Submission Checklist

• Follows CONTRIBUTING guidelines
• Commit messages
  are release-note ready, emphasizing
  what was changed, not how.
• Includes detailed description of changes
• Contains risk assessment (Low | Medium | High)
• Highlights breaking API changes (if applicable)
• Links to automated tests covering new functionality
• Includes manual testing instructions (if necessary)
• Updates relevant GitBook documentation (PR link: ___)
• Includes product update to be announced in the next stable release notes

What does this PR do?

This PR adds acceptance tests to validate the SARIF and human-readable outputs of the secrets test command. These tests are designed to codify our rendering expectations and serve as a shared contract for the CLI team to iterate against.

Where should the reviewer start?

• test/jest/acceptance/snyk-secrets/snyk-secrets-test-user-journey.spec.ts;
• the document linked in the ticket;

How should this be manually tested?

Run the acceptance tests locally.

What's the product update that needs to be communicated to CLI users?

N/A

Risk assessment (Low | Medium | High)?

Low - extends test suite.

What are the relevant tickets?

• PS-357

[snyk-io]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues
✅	Code Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[snyk-pr-review-bot]

PR Reviewer Guide 🔍

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Unsafe run access 🟠 [major] In setupIsolatedIgnoreEnv, the code parses the CLI output and immediately accesses jsonOutput.runs[0].results. This is dangerous because if the CLI fails to produce any runs (e.g., due to an execution error or an empty scan), jsonOutput.runs could be an empty array or undefined. Accessing the first element of an empty array will result in undefined, and subsequent access to .results will throw a TypeError, crashing the test runner before any assertions can provide useful feedback. const jsonOutput = JSON.parse(jsonStdout); const results = jsonOutput.runs[0].results || []; Property name mismatch 🟠 [major] There is a discrepancy in how finding IDs are extracted in setupIsolatedIgnoreEnv compared to how they are validated in later tests. The setup logic extracts IDs from r.fingerprints?.identity, but the validation logic in the SARIF tests explicitly checks for a property named fingerprint. If the CLI output only provides fingerprint, the extraction at line 129 will return an empty array, no ignores will be applied during setup, and the test for rendering ignores will fail to see the expected suppressed results. results.map((r: any) => r.fingerprints?.identity).filter(Boolean),

📚 Repository Context Analyzed This review considered 6 relevant code sections from 4 files (average relevance: 1.00) CONTRIBUTING.md (2 segments, lines 1-218) dangerfile.js (lines 1-100) package.json (lines 1-223) package-lock.json (2 segments, lines 18661-18977)

[j-luong]

issue: same here

let's add a comment on these skipped tests stating why they're skipped, otherwise we should remove these tests instead of skipping them

[j-luong]

pre-approved to unblock once the issues are resolved

[snyk-pr-review-bot]

PR Reviewer Guide 🔍

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Fragile JSON Parsing 🟡 [minor] The setupIsolatedIgnoreEnv function uses JSON.parse(jsonStdout) on the raw stdout of the Snyk CLI. Snyk CLI often outputs non-JSON data to stdout, such as update notifications, deprecation warnings, or legal notices, especially in CI environments. If such text is present, JSON.parse will throw a SyntaxError, causing the test to fail. It is safer to use a utility that extracts the JSON block from the output or ensure the CLI is invoked with flags that suppress all non-JSON output (like SNYK_SYSTEM_NO_UPDATE_NOTIFIER). const jsonOutput = JSON.parse(jsonStdout);

📚 Repository Context Analyzed This review considered 7 relevant code sections from 6 files (average relevance: 1.00) CONTRIBUTING.md (2 segments, lines 1-218) dangerfile.js (lines 1-111) package.json (lines 1-223) src/cli/main.ts (lines 235-517) cliv2/cmd/cliv2/main.go (lines 1-202) package-lock.json (lines 18640-18951)