Skip to content

fix(ci): make the test workflow green on clean runners#40

Merged
omerakben merged 1 commit into
mainfrom
fix/ci-green
May 30, 2026
Merged

fix(ci): make the test workflow green on clean runners#40
omerakben merged 1 commit into
mainfrom
fix/ci-green

Conversation

@omerakben

@omerakben omerakben commented May 29, 2026

Copy link
Copy Markdown
Owner

What

The `test` workflow has been red on `main` since ~2026-05-22. Two tests pass locally but fail on clean CI runners (environment-dependent), unnoticed because `bun test` is green locally and no status checks are required.

Root cause + fix

`M17 A11 — runAudit dispatches the repo_context tool loop` — the repo_context grep/glob tools shell out to `ripgrep`; CI runners have no `rg`, so A11's grep returns empty `resultPaths` and fails, while 15 sibling rg-integration tests silently skip. Fix: install `ripgrep` in `test.yml` before `bun test` (apt on Linux, brew on macOS) — fixes A11 and turns the 15 skips into real runs. A RED-first test in `ci-workflows.test.ts` pins install-before-test ordering.

`active-run SHIP continuation — non-interactive operator guard` — the e2e asserted the SHIP-approval message, but on a runner with no authenticated provider the non-interactive provider-health guard fails closed first (`requires healthy real providers; refusing silent fake fallback`). The command still exits non-zero. Fix: broaden the assertion to accept either valid fail-closed guard; keep the no-silent-proceed negatives. The SHIP-approval guard stays covered by the `runApprove — operator mode` SHIP test.

Verification

  • 3811 pass / 2 skip / 0 fail locally; typecheck clean.
  • Cross-family review: Codex `gpt-5.5` xhigh -> `push` (confirmed the A11 cause in source, the install step's safety, and that the operator broadening preserves the fail-closed safety property).
  • This PR's own CI run is the empirical proof — both `bun test` jobs should be green with the 15 rg-gated tests running.

Follow-up

Once green, consider making the `bun test` jobs required status checks so CI can't silently rot again.

Summary by CodeRabbit

Release Notes

  • Chores

    • CI workflow now automatically installs ripgrep dependency using platform-specific package managers
  • Tests

    • Added integration test to verify CI workflow configuration
    • Enhanced operator mode test to handle multiple operator guard failure scenarios

Review Change Stack

The test workflow has been red on main since ~2026-05-22: two tests pass
locally but fail on clean CI runners (no ripgrep, no provider auth). Caught
only because bun test is green locally.

- .github/workflows/test.yml: install ripgrep before bun test. The
  repo_context grep/glob tools shell out to rg; without it M17 A11 fails
  (empty resultPaths) and 15 rg-integration tests skip. Installing rg fixes
  A11 and turns the 15 skips into real runs.
- tests/ci-workflows.test.ts: RED-first test pinning install-rg-before-test
  ordering.
- tests/operator-mode.test.ts: the active-run SHIP e2e asserted the
  SHIP-approval message, but with no authenticated provider the
  non-interactive provider-health guard fails closed first. Broaden the
  assertion to accept either valid fail-closed guard; keep the
  no-silent-proceed negative assertions. The SHIP-approval guard stays
  covered by the runApprove operator-mode test.

3811 pass / 2 skip / 0 fail; typecheck clean. Cross-family review: Codex
gpt-5.5 xhigh -> push (one comment nit closed).
Copilot AI review requested due to automatic review settings May 29, 2026 19:56
@coderabbitai

coderabbitai Bot commented May 29, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f59db701-9759-4984-9c15-5a03156a34d5

📥 Commits

Reviewing files that changed from the base of the PR and between bd94f86 and 7ddd035.

📒 Files selected for processing (3)
  • .github/workflows/test.yml
  • tests/ci-workflows.test.ts
  • tests/operator-mode.test.ts

📝 Walkthrough

Walkthrough

This PR introduces ripgrep as a CI tooling dependency with workflow integration tests, and updates an operator-mode guard test to handle multiple failure message scenarios. The workflow installs ripgrep before typecheck/test phases, a verification test confirms correct step ordering, and an unrelated operator-mode test now accepts either approval-guard or provider-health-guard failure messages.

Changes

Ripgrep CI Installation and Verification

Layer / File(s) Summary
Ripgrep installation in test workflow
.github/workflows/test.yml
Test job installs ripgrep via APT on Linux and Homebrew on other runners before bun run typecheck.
Ripgrep installation step verification
tests/ci-workflows.test.ts
New test loads test.yml, finds ripgrep install step and bun test step, and asserts ripgrep step runs first.

Operator Mode Guard Failure Handling

Layer / File(s) Summary
Multiple guard failure message acceptance
tests/operator-mode.test.ts
Non-interactive operator test broadens stderr expectation to match either SHIP approval guard or provider-health guard fail-closed messages with expanded comments documenting both routes.

🎯 1 (Trivial) | ⏱️ ~4 minutes

🐰 A rabbit bounds through CI logs bright,
Finding ripgrep guards the build's tight fight,
Tests now accept what guards may say,
Multiple paths to approve the way,
Workflow and guards dance side by side today! 🌿

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Description check ❓ Inconclusive The description covers the what, root cause, fix, and verification, but does not include the structured template sections for Testing, Cross-model review, and Breaking changes. Complete the description by filling in the template sections: Testing (with bun test results), Cross-model review (Codex briefing/response links), and Breaking changes checkboxes for clarity.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: fixing the CI test workflow to pass on clean runners.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/ci-green

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the test suite to improve CI reliability. It adds a test in tests/ci-workflows.test.ts to verify that ripgrep is installed before running tests, ensuring repo_context tool tests are not silently skipped. It also updates tests/operator-mode.test.ts to handle environment-specific error messages when running in non-interactive operator mode. There are no review comments, and I have no feedback to provide.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes CI test workflow failures on clean runners by installing ripgrep (required by repo_context grep/glob tools) before tests run, and broadens an e2e assertion to accept either valid non-interactive fail-closed guard.

Changes:

  • Add an Install ripgrep step in test.yml (apt on Linux, brew on macOS) before bun test.
  • Add a workflow test pinning install-before-test ordering for ripgrep.
  • Broaden the SHIP-continuation e2e assertion to also accept the provider-health fail-closed message used on CI runners without authenticated providers.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
.github/workflows/test.yml Installs ripgrep before bun test on both Linux and macOS runners.
tests/ci-workflows.test.ts RED-first test asserting ripgrep is installed before the test step.
tests/operator-mode.test.ts Broadens SHIP non-interactive assertion to accept the provider-health guard message; keeps no-silent-proceed negatives.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@omerakben omerakben merged commit baefd14 into main May 30, 2026
4 checks passed
@omerakben omerakben deleted the fix/ci-green branch May 30, 2026 03:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants