docs: gate SAST on PR-introduced findings, not ruleset drift#104
Conversation
github-actions
Bot
added
documentation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a new section to the reusable workflow security documentation explaining how to gate SAST checks on PR-introduced findings rather than ruleset drift. It details the problem of ruleset drift blocking unrelated PRs and outlines solutions such as scanner baselines and GitHub-native code scanning checks. The feedback suggests replacing the placeholder O/R with OWNER/REPO in the API path for better consistency with other reference files.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
There was a problem hiding this comment.
Pull request overview
Adds guidance to the GitHub Project Skill docs on how to keep required SAST checks effective without letting auto-updating rulesets (e.g., --config auto) break unrelated PRs due to newly surfaced pre-existing findings (“ruleset drift”). The new section recommends baseline/diff-aware gating so required checks fail only on PR-introduced findings relative to the merge-base.
Changes:
- Document the “ruleset drift” failure mode for required SAST checks using auto-updating rulesets.
- Recommend diff/baseline-aware gating as the correct mitigation (vs pinning rulesets or switching to report-only).
- Outline two implementation approaches: scanner-side baselining (e.g., Semgrep
--baseline-commit) and GitHub code scanning PR checks (new alerts only).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
0c0f838 to
5103d72
Compare
…, not drift A required SAST check with an auto-updating ruleset (opengrep/semgrep --config auto) turns a green main red on unrelated PRs when the ruleset drifts. Pinning stops new-rule adoption; report-only lets real new vulns through. The right shape is diff/baseline-aware gating (Semgrep --baseline-commit — verify the fork's flag — or GitHub code-scanning 'new alerts only'). Signed-off-by: Sebastian Mendel <info@sebastianmendel.de>
5103d72 to
a2ebb1a
Compare
|
Field-tested during a netresearch/t3x-nr-llm docs PR: a required
SAST (Opengrep)check (--config auto) flipped an unrelated docs PR from green to red because the community ruleset had drifted and now flagged 19 pre-existing findings on the repo's own workflow files — zero on the PR's changes.Adds a
reusable-workflow-security.mdsection on the correct fix: diff/baseline-aware gating (block only on findings the PR introduces vs the merge-base) — not pinning the ruleset (freezes new-rule adoption) and not report-only (lets real new vulns through). Mechanisms: Semgrep--baseline-commit(with a note to verify the fork's exact flag) or GitHub code-scanning 'new alerts only'.markdownlint clean.