[patsypppe]

Summary

Lets the benchmark workflow run any Harbor-format dataset, not just Harbor's examples. Harbor's registry lists dozens of established benchmarks in this format (Terminal-Bench 2.0, HumanEvalFix, aider-polyglot, EvoEval, ...); on Terminal-Bench 2.0, 66 of 89 tasks convert with --rewrite-uv-boilerplate and 23 are refused (verifier installs packages at test time).

• Workflow inputs: dataset_repo, a pinned harbor_ref commit, tasks_path, max_timeout.
• import-harbor --max-timeout: caps each task's agent timeout. This shortens the benchmark's own limit, so any reported result must say so.
• suite build-images --drop-failed: a task whose image fails to build is moved to dropped/ and the run carries on with the rest.

Test plan

• make check (516 unit tests)
• CI
• A real Terminal-Bench 2.0 slice run from this branch

---

Generated by Claude Code

[github-actions]

Meridian gate

✅ PASS — suite pass^k moved -0.000 (0.486 → 0.486), within the 0.030 tolerance

Failing trials in this run

• expired-coupon — SELECT status FROM orders WHERE id = 1042 returned [['pending']], expected [['invoiced']]
• expired-on-bulk-order — SELECT status FROM orders WHERE id = 1045 returned [['pending']], expected [['invoiced']]
• expiry-boundary — SELECT status FROM orders WHERE id = 1046 returned [['pending']], expected [['invoiced']]
• happy-path — SELECT status FROM orders WHERE id = 1041 returned [['pending']], expected [['invoiced']]
• stale-coupon — SELECT status FROM orders WHERE id = 1044 returned [['pending']], expected [['invoiced']]
• unknown-coupon — SELECT status FROM orders WHERE id = 1047 returned [['pending']], expected [['invoiced']]

task	baseline pass^3	head pass^3
expired-coupon	0.400	0.400
expired-on-bulk-order	0.400	0.400
expiry-boundary	0.400	0.400
happy-path	0.400	0.400
missing-field	1.000	1.000
stale-coupon	0.400	0.400
unknown-coupon	0.400	0.400

	baseline	head
suite pass^3	0.486	0.486

Sensitivity. Not estimated: all 7 tasks moved by the same amount, so there is no spread to estimate sensitivity from. Treat the verdict as directional.

n=5 · k=3 · status complete · harness errors 0.0% · cost 51c

_{head run run-1793030850-b69c86 · baseline run run-1793030825-f9ec31 · manifest sha256:189d38621727f2e5… · reproduce with meridian replay run-1793030850-b69c86}