fix(ci): make the test workflow green on clean runners#40
Conversation
The test workflow has been red on main since ~2026-05-22: two tests pass locally but fail on clean CI runners (no ripgrep, no provider auth). Caught only because bun test is green locally. - .github/workflows/test.yml: install ripgrep before bun test. The repo_context grep/glob tools shell out to rg; without it M17 A11 fails (empty resultPaths) and 15 rg-integration tests skip. Installing rg fixes A11 and turns the 15 skips into real runs. - tests/ci-workflows.test.ts: RED-first test pinning install-rg-before-test ordering. - tests/operator-mode.test.ts: the active-run SHIP e2e asserted the SHIP-approval message, but with no authenticated provider the non-interactive provider-health guard fails closed first. Broaden the assertion to accept either valid fail-closed guard; keep the no-silent-proceed negative assertions. The SHIP-approval guard stays covered by the runApprove operator-mode test. 3811 pass / 2 skip / 0 fail; typecheck clean. Cross-family review: Codex gpt-5.5 xhigh -> push (one comment nit closed).
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis PR introduces ripgrep as a CI tooling dependency with workflow integration tests, and updates an operator-mode guard test to handle multiple failure message scenarios. The workflow installs ripgrep before typecheck/test phases, a verification test confirms correct step ordering, and an unrelated operator-mode test now accepts either approval-guard or provider-health-guard failure messages. ChangesRipgrep CI Installation and Verification
Operator Mode Guard Failure Handling
🎯 1 (Trivial) | ⏱️ ~4 minutes
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint skipped: no ESLint configuration detected in root package.json. To enable, add Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request updates the test suite to improve CI reliability. It adds a test in tests/ci-workflows.test.ts to verify that ripgrep is installed before running tests, ensuring repo_context tool tests are not silently skipped. It also updates tests/operator-mode.test.ts to handle environment-specific error messages when running in non-interactive operator mode. There are no review comments, and I have no feedback to provide.
There was a problem hiding this comment.
Pull request overview
Fixes CI test workflow failures on clean runners by installing ripgrep (required by repo_context grep/glob tools) before tests run, and broadens an e2e assertion to accept either valid non-interactive fail-closed guard.
Changes:
- Add an
Install ripgrepstep intest.yml(apt on Linux, brew on macOS) beforebun test. - Add a workflow test pinning install-before-test ordering for ripgrep.
- Broaden the SHIP-continuation e2e assertion to also accept the provider-health fail-closed message used on CI runners without authenticated providers.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| .github/workflows/test.yml | Installs ripgrep before bun test on both Linux and macOS runners. |
| tests/ci-workflows.test.ts | RED-first test asserting ripgrep is installed before the test step. |
| tests/operator-mode.test.ts | Broadens SHIP non-interactive assertion to accept the provider-health guard message; keeps no-silent-proceed negatives. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
What
The `test` workflow has been red on `main` since ~2026-05-22. Two tests pass locally but fail on clean CI runners (environment-dependent), unnoticed because `bun test` is green locally and no status checks are required.
Root cause + fix
`M17 A11 — runAudit dispatches the repo_context tool loop` — the repo_context grep/glob tools shell out to `ripgrep`; CI runners have no `rg`, so A11's grep returns empty `resultPaths` and fails, while 15 sibling rg-integration tests silently skip. Fix: install `ripgrep` in `test.yml` before `bun test` (apt on Linux, brew on macOS) — fixes A11 and turns the 15 skips into real runs. A RED-first test in `ci-workflows.test.ts` pins install-before-test ordering.
`active-run SHIP continuation — non-interactive operator guard` — the e2e asserted the SHIP-approval message, but on a runner with no authenticated provider the non-interactive provider-health guard fails closed first (`requires healthy real providers; refusing silent fake fallback`). The command still exits non-zero. Fix: broaden the assertion to accept either valid fail-closed guard; keep the no-silent-proceed negatives. The SHIP-approval guard stays covered by the `runApprove — operator mode` SHIP test.
Verification
Follow-up
Once green, consider making the `bun test` jobs required status checks so CI can't silently rot again.
Summary by CodeRabbit
Release Notes
Chores
Tests