Integration tests for repair invariants: assertion immutability and budget caps

[myselfsiddharth]

Context

Two invariants keep the gate number honest, and both currently rest on a stub that can never violate them.

1. Assertions are immutable during repair. contracts/assertion.schema.json says repair MUST NOT weaken an assertion. ReplayRunner enforces it with deepFreeze plus three assertAssertionUnchanged calls per repair attempt. If a repair could relax an assertion, replay-validity would become self-fulfilling — the loop would keep loosening the check until the step passed.

2. The repair budget is bounded. maxRepairsPerRun defaults to 2, aligned with success_with_le_2_repairs on the run metric, which is directly one half of the PRD section 9 pass condition ("task-level success with ≤2 repairs ≥ 90%").

tests/unit/runner.test.ts exercises the loop, but only against StubRepairModelClient, which always returns corrected_action: null. A stub that never proposes anything cannot test what happens when a proposal is hostile or a budget is exhausted mid-flight. Once a real model is wired, these invariants are load-bearing against non-deterministic input.

Depends on: the repair-model-client issue.

What to build

tests/integration/repair-invariants.test.ts, driven by scripted fake RepairModelClient implementations — no network, no API key, fully deterministic.

Cases:

1. Assertion tamper is rejected. A client that returns a corrected_action and also mutates the assertion object it was handed (ctx.assertion) must cause assertAssertionUnchanged to throw. Test both the frozen-copy path and the live step.assertion path — ReplayRunner checks both, and both should be covered.
2. Strength downgrade is rejected. A client that attempts to change assertion.strength from strong to weak must fail the same way. This is the specific attack the contract language names.
3. Budget is honoured. With maxRepairsPerRun: 2 and a client that always proposes an action that keeps failing, exactly 2 repair attempts occur, the step ends REPAIR_EXHAUSTED, and repair_count === 2.
4. Budget is per run, not per step. With a program whose first step consumes both repairs, a later failing step must not get a fresh budget. Assert the run-level repair_count never exceeds the cap.
5. success_with_le_2_repairs is accurate. A run needing exactly 2 repairs and succeeding sets it true; a run needing 3 (with a raised cap) sets it false. This field feeds the section 9 pass condition directly, so verify it rather than trusting it.
6. Cost is accounted even on failure. A client that consumes tokens and then returns corrected_action: null must still have those tokens in cost_repair. Unbilled failed repairs would understate cost against the 70% kill line.
7. A thrown client error does not silently pass the step. A client that throws must produce a failure outcome, not an exception that escapes and aborts the matrix.

Constraints

• No network. No ANTHROPIC_API_KEY. These are fakes implementing RepairModelClient.
• Do not relax an assertion to make a test pass. If a test reveals the runner's enforcement has a hole, fix the runner — that is a genuine finding and the most valuable outcome this issue can have.
• Deterministic and fast; this runs on every PR.

How to test

npm run test:integration
npm run ci

Then confirm each guard is real rather than vacuous: temporarily comment out one assertAssertionUnchanged call in src/runner/replay.ts and verify the tamper tests fail. A test that passes with the protection removed is testing nothing. Restore before committing and describe the exercise in the PR body.

Before you open the PR

• All seven cases implemented and passing
• Each guard verified to fail when its protection is removed; described in the PR body
• No network, no API key, no flakiness
• Any runner hole found is fixed in this PR with the test that caught it
• npm run ci, npm run test:canary green
• docs/gate/runner.md invariants section references these tests by path
• Branch track1/b4-repair-invariant-tests