fix(compliance-audit): suppress gh_api stdout on failure to fix false CODEOWNERS positive#212
fix(compliance-audit): suppress gh_api stdout on failure to fix false CODEOWNERS positive#212don-petry wants to merge 69 commits into
Conversation
…se positives When gh api returns a 404, it outputs the error JSON to stdout (not stderr). The previous gh_api() forwarded all stdout unconditionally, causing callers like check_codeowners() to receive concatenated 404 JSON blobs instead of empty strings. This made found=true for a missing file and treated the error JSON as CODEOWNERS content, triggering a false codeowners-org-leads-not-first finding. Fix: capture output into a variable and only echo it when the exit code is 0, so failed API calls produce no stdout. Closes #208 Co-authored-by: Don Petry <don-petry@users.noreply.github.com>
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe PR captures ChangesCompliance audit script and ignore list edits
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@petry-projects/org-leads — this PR is ready for review and merge. It fixes a false positive in the compliance audit script. |
There was a problem hiding this comment.
Pull request overview
This PR updates the gh_api() retry wrapper in scripts/compliance-audit.sh to prevent gh api stdout (including error JSON bodies from failed requests like 404s) from being forwarded to callers, which previously caused false positives in checks like check_codeowners().
Changes:
- Capture
gh apioutput insidegh_api()and only emit it when the command succeeds (exit code 0).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: a191bfcfa6fc014ce4d88ea3ba70860a890233d4
Review mode: triage-approved (single reviewer)
Summary
Single-line CI workflow change pinning petry-projects/.github/.github/workflows/agent-shield-reusable.yml from the mutable @v1 tag to the exact commit SHA 0cb4bba11d7563bf197ad805f12fb8639e4879e4, with the # v1 human-readable comment retained and the with: required-files: AGENTS.md input preserved. Verified via gh api repos/petry-projects/.github/git/refs/tags/v1 that the pinned SHA matches the actual commit currently behind the v1 tag. Conforms to the org action-pinning policy and follows the same pattern as the recently merged #127 (auto-rebase-reusable.yml SHA pin).
Linked issue analysis
Closes #114 — a compliance-audit finding for unpinned-actions-agent-shield.yml flagging that agent-shield.yml had 1 action not pinned to SHA. The PR addresses exactly that line; no other unpinned references remain in this workflow.
Findings
No issues found.
- SHA pin verified against upstream tag
v1(matches0cb4bba11d7563bf197ad805f12fb8639e4879e4). - Repo-specific
with:inputs preserved unchanged. - No secrets, permissions, or trigger surface modified.
- Pre-existing missing trailing newline on the file is unchanged by this PR; out of scope.
CI status
All required checks green: AgentShield, Claude Code, CodeQL (Analyze actions), Dependency audit (ecosystem detect), SonarCloud / SonarCloud Code Analysis (Quality Gate passed, 0 new issues), CodeRabbit. Dependabot auto-merge and ecosystem-specific audit jobs correctly skipped (no matching ecosystems / not a Dependabot PR). CodeRabbit posted a rate-limit notice but its status check reports SUCCESS; gemini-code-assist skipped due to unsupported file type. Mergeable: yes; merge state BLOCKED only on the required human review.
Reviewed automatically by the PR-review agent (single-reviewer mode: opus 4.7). Reply if you need a human review.
Dev-Lead — fix-bot-comment (applied)Changes committed and pushed. |
don-petry
dismissed stale reviews from coderabbitai[bot] and donpetry-bot
via
2010e83
Dev-Lead — review-changes (applied)Changes committed and pushed. |
Dev-Lead — review-changes (applied)Changes committed and pushed. |
Dev-Lead — review-changes (applied)Changes committed and pushed. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/compliance-audit.sh`:
- Around line 1088-1090: The add_finding call that reports the missing
`.github/copilot-instructions.md` uses severity "warning" but should be "error";
update the add_finding invocation that has identifier
"missing-copilot-instructions" (the call starting with add_finding "$repo"
"standards" "missing-copilot-instructions") to use "error" instead of "warning"
so the finding is emitted as an error; keep the rest of the parameters (title,
description, reference) unchanged and ensure the message still references
`standards/copilot-instructions-standard.md`.
- Around line 1242-1261: The case branch handling remediation for workflow
findings is using the label "workflows)" but findings are emitted as
"ci-workflows"; update the case pattern in the switch that checks case
"$category" to match "ci-workflows)" (or include an alternative pattern like
"workflows|ci-workflows)") so workflow findings map to the specific
remediation_steps block (refer to the case "$category" switch and the existing
"workflows)" branch to locate and modify the label).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 01efbb01-34e6-4ba4-b487-063cf4875c4d
📒 Files selected for processing (2)
.gitignorescripts/compliance-audit.sh
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/compliance-audit.sh`:
- Around line 1088-1090: The add_finding call that reports the missing
`.github/copilot-instructions.md` uses severity "warning" but should be "error";
update the add_finding invocation that has identifier
"missing-copilot-instructions" (the call starting with add_finding "$repo"
"standards" "missing-copilot-instructions") to use "error" instead of "warning"
so the finding is emitted as an error; keep the rest of the parameters (title,
description, reference) unchanged and ensure the message still references
`standards/copilot-instructions-standard.md`.
- Around line 1242-1261: The case branch handling remediation for workflow
findings is using the label "workflows)" but findings are emitted as
"ci-workflows"; update the case pattern in the switch that checks case
"$category" to match "ci-workflows)" (or include an alternative pattern like
"workflows|ci-workflows)") so workflow findings map to the specific
remediation_steps block (refer to the case "$category" switch and the existing
"workflows)" branch to locate and modify the label).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 01efbb01-34e6-4ba4-b487-063cf4875c4d
📒 Files selected for processing (2)
.gitignorescripts/compliance-audit.sh
🛑 Comments failed to post (2)
scripts/compliance-audit.sh (2)
1088-1090:
⚠️ Potential issue | 🟠 Major | ⚡ Quick winMissing Copilot instructions should be reported as error severity, not warning.
At Line 1088, the finding is emitted as
"warning"even though the policy text says every repository must have this file. This under-reports a mandatory compliance breach.Suggested fix
- add_finding "$repo" "standards" "missing-copilot-instructions" "warning" \ + add_finding "$repo" "standards" "missing-copilot-instructions" "error" \Based on learnings: Applies to
.github/copilot-instructions.md: Every repository MUST have its own.github/copilot-instructions.mdfile tailored to the repo.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.add_finding "$repo" "standards" "missing-copilot-instructions" "error" \ "Missing \`.github/copilot-instructions.md\`. Every repo must have its own Copilot instructions file — Copilot instruction files are repository-scoped and do not propagate from the \`petry-projects/.github\` repo. Copy the canonical template from \`standards/copilot-instructions-standard.md\` in \`petry-projects/.github\`, then tailor it with this repo's specific tech stack, project structure, local dev commands, required environment variables, and testing configuration." \ "standards/copilot-instructions-standard.md"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/compliance-audit.sh` around lines 1088 - 1090, The add_finding call that reports the missing `.github/copilot-instructions.md` uses severity "warning" but should be "error"; update the add_finding invocation that has identifier "missing-copilot-instructions" (the call starting with add_finding "$repo" "standards" "missing-copilot-instructions") to use "error" instead of "warning" so the finding is emitted as an error; keep the rest of the parameters (title, description, reference) unchanged and ensure the message still references `standards/copilot-instructions-standard.md`.
1242-1261:
⚠️ Potential issue | 🟠 Major | ⚡ Quick winCategory key mismatch breaks remediation mapping for workflow findings.
At Line 1253,
case "$category"usesworkflows), but findings are emitted asci-workflowsthroughout this file. Workflow findings will always hit the default branch and lose the tailored remediation steps.Suggested fix
- workflows) + ci-workflows) remediation_steps="Copy the relevant workflow template from \`standards/workflows/\` verbatim — do not generate from scratch:🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/compliance-audit.sh` around lines 1242 - 1261, The case branch handling remediation for workflow findings is using the label "workflows)" but findings are emitted as "ci-workflows"; update the case pattern in the switch that checks case "$category" to match "ci-workflows)" (or include an alternative pattern like "workflows|ci-workflows)") so workflow findings map to the specific remediation_steps block (refer to the case "$category" switch and the existing "workflows)" branch to locate and modify the label).
Dev-Lead — review-changes (applied)Changes committed and pushed. |
|
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
Summary
codeowners-org-leads-not-firstcompliance finding inscripts/compliance-audit.shgh_api()retry wrapper was forwardinggh apistdout unconditionally — including the error JSON body emitted on a 404 responsecheck_codeowners()received concatenated 404 JSON blobs instead of an empty string, makingfound=truefor a missing file and treating the error JSON as CODEOWNERS contentThe CODEOWNERS file at
.github/CODEOWNERSwas already correct (* @petry-projects/org-leads); no CODEOWNERS change is needed.Test plan
codeowners-org-leads-not-firstfinding should be absentgh_apicall sites are unaffected (callers that use|| echo ""or redirect stdout to/dev/nullstill work correctly)Closes #208
Generated with Claude Code
Summary by CodeRabbit