Skip to content

fix(test): Bun proof scripts exit 0 on a failed assertion #1092

Description

@vivek7405

Problem

Every test/bun/*.mjs proof script that boots a server through startServer exits 0 when its assertions fail under Bun, so CI reports green on a real regression.

Verified with a minimal repro (boot startServer, then assert.equal(1, 2)):

node t5.mjs  -> exit=1   (correct)
bun  t5.mjs  -> exit=0   (swallowed)

Cause: startServer calls installProcessHandlers (packages/server/src/listener-core.js L388), which registers an uncaughtException handler that runs the graceful shutdown, and makeShutdown ends in process.exit(0) on a clean close (L426). On Bun a top-level assertion failure in the module body routes through that handler, so the shutdown's exit code wins. Node reports the module-evaluation rejection before the handler can exit, which is why only Bun is affected in practice. The scripts pass a quiet logger, so the logged error is swallowed too and the run is completely silent.

Impact: .github/workflows/ci.yml runs roughly fifteen of these directly (run: bun test/bun/smoke.mjs, listener.mjs, compression.mjs, file-storage.mjs, app-boot.mjs, seed.mjs, blog-db.mjs, path-alias.mjs, timeouts.mjs, binding-prefixes.mjs, slot-ssr-parity.mjs, and others) and trusts the step's exit code. Those steps currently cannot fail.

Found while adding test/bun/forwarded-proto.mjs in #1091: its counterfactual (revert the fix, expect red) came back exit 0.

Not affected: node scripts/run-bun-tests.js, which spawns bun test <file> (L119) and classifies on both status and the reported fail count, so the *.test.mjs wrappers DO fail correctly there. The hole is specific to the direct plain-script CI steps.

Design / approach

Two candidate fixes, not mutually exclusive:

  1. Per-script (the shape fix(server): honor X-Forwarded-Proto/Host on the Bun listener #1091 already uses): capture the failure, run cleanup, then process.exit(1) explicitly. Self-contained and obviously correct, but it must be repeated in every script and a new script can silently forget it.
  2. Systemic: have the proof scripts opt out of the framework's process handlers, or give startServer a way not to install them (a test-only flag), or set process.exitCode = 1 before the shutdown handler runs. Preferred, since it fixes every existing script at once and cannot be forgotten by the next one.

Whichever is chosen, add a guard so this cannot silently regress: a meta-test that runs a deliberately-failing proof script under Bun and asserts a non-zero exit.

Implementation notes (for the implementing agent)

  • Where to look: packages/server/src/listener-core.js installProcessHandlers() (L388) and makeShutdown() (L410 onward, the process.exit(0) at L426). The scripts themselves are test/bun/*.mjs; the CI steps are in .github/workflows/ci.yml around L104 to L204. The working counterexample to copy is test/bun/forwarded-proto.mjs (from fix(server): honor X-Forwarded-Proto/Host on the Bun listener #1091).
  • Landmines:
    • Do NOT simply stop installing the handlers in production. They are the graceful-shutdown path (SIGINT/SIGTERM drain, the 10s force-exit) and are load-bearing for a real deploy.
    • installProcessHandlers is guarded by a globalThis.__webjsProcHandlers latch, so a second startServer in one process is a no-op. A per-test opt-out must account for that.
    • The scripts pass a quiet logger, so any fix that only logs remains invisible. The exit code is the contract.
    • Verify the fix on BOTH runtimes: node currently exits 1 and must keep doing so.
  • Invariants: packages/ stays plain .js + JSDoc. Node/Bun parity (Support both Bun and Node runtimes (first-class create + run) #508) is the whole point of this test layer.
  • Tests: a meta-test asserting a failing proof script exits non-zero under Bun. A counterfactual is inherent here (the test IS the counterfactual).

Acceptance criteria

  • A test/bun/*.mjs proof whose assertion fails exits non-zero under Bun
  • Node keeps exiting non-zero for the same failure
  • Graceful shutdown on SIGINT/SIGTERM is unchanged for a real server
  • A guard test prevents this from silently regressing
  • The existing proof scripts are covered (systemic fix) or individually updated

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status
Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions