fix(actor): make spawn the default and run the exception in the tool prompt - #1942
Merged
Conversation
…prompt Agents kept reaching for `actor run`, which blocks the whole conversation until the subagent finishes, so parallelism was lost on ordinary delegation. The cause was the prompt itself: `run` was listed first as the straightforward path and nearly every example used it. Reorder and rewrite the actor tool description so `spawn` is presented first and as THE DEFAULT (background, returns actor_id immediately, subagents run in parallel), and `run` is a narrow exception gated on a crisp test: only a tiny, fast lookup whose result you cannot make your next decision without in this turn. Examples flip to spawn (including a 3-way parallel fan-out) with one labelled run exception, and a collection section documents notifications plus wait/status. Behavior is unchanged - description/prompt text, action describe() strings and schema ordering only. Also corrects orchestrator.txt, which described `actor spawn` as blocking; both remain forbidden there for real work.
Every other example in actor.txt is clean, copy-pasteable JSON. The run example carried a trailing prose annotation on the same line, and the model imitates the shape of these examples line-by-line rather than the parsed JSON — so the arrow could be copied verbatim into a real call. Move the note to its own line above, matching the prose convention the EXCEPTION example block further down already uses.
The existing coverage for the spawn-first rewrite only asserted the wording of actor.txt, which cannot show that an agent actually prefers spawn. Add the two things that can: - a deterministic assertion that the JSON schema handed to the provider offers spawn ahead of run (the wire-format consequence of moving spawnSchema ahead of runSchema; previously only the union's LENGTH was pinned, never its order) - an opt-in RUN_ACTOR_SPAWN_AB=1 test that drives the real headless CLI against mimo/mimo-v2.5 and reads the emitted operation out of the structured tool part A/B over 6 trials per prompt per arm: a clear-delegation prompt went 0/6 spawn on the run-first description to 6/6 spawn on the spawn-first one, and a three-way fan-out went from three sequential runs to three parallel spawns in 6/6 trials. The blocking-lookup prompt still chose run on both arms, so the change did not over-correct.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Agents kept reaching for
actor run, which blocks the whole conversation until the subagent finishes, instead ofactor spawn, which returns anactor_idimmediately and lets several subagents work in parallel while the agent keeps talking to the user.The cause was the tool's own prompt: in the operations list
runcame first and read as the straightforward path ("spawn a subagent and BLOCK until completion; result returned inline"), and every single example inactor.txtusedrun. The model naturally defaulted to the blocking verb and lost parallelism.Change
Description/prompt only — tool behavior is unchanged.
packages/opencode/src/tool/actor.txt—spawnis now presented first and as THE DEFAULT (background, immediateactor_id, parallel);runis documented as a RARE EXCEPTION with a crisp admission test: only when you cannot make your very next decision without the result in THIS turn and the work is a tiny, fast lookup. Added a "Collecting spawned work" section (notification →wait→status). Examples flipped: threespawnexamples including a 3-way parallel fan-out, plus exactly onerunexample explicitly labelledTHE EXCEPTION.packages/opencode/src/tool/actor.shell.txt— same inversion for the shell invocation style:actor spawndocumented first as the default,actor runsecond as the exception, parallel fan-out promoted into the primary examples, and the "When to use what" table rewritten.packages/opencode/src/tool/actor.ts— theactionliteraldescribe()strings now sayTHE DEFAULT(spawn) /RARE EXCEPTION … BLOCKS the whole conversation(run), andspawnSchemaprecedesrunSchemain the discriminated union so the model reads spawn first in the generated JSON schema. Discriminated-union parsing is keyed onaction, so ordering is behavior-neutral for parsing — but it does change theanyOfbranch order the model is handed.packages/opencode/src/session/prompt/orchestrator.txt— corrected a factual error: it claimedactor run/actor spawnboth "start a BLOCKING subagent".spawnis not blocking. Both remain forbidden there for real work (the orchestrator routes real work tosession create), so orchestrator policy is unchanged.Does it actually change behaviour? Measured, not asserted from wording
The original evidence for this PR was 14 regex assertions over the
.txtfiles. Prompt-text assertions are not behavioural evidence — they show the file says "spawn is the default", not that an agent picksspawn. So the claim was measured with a paired A/B against a real model.Method: the real headless CLI (
run --format json), modelmimo/mimo-v2.5, 6 trials per prompt per arm (a perfect split at n=6 is p≈0.002 by Fisher's exact, and 6 tolerates one stochastic outlier where 5 would not). The two arms differ only in the four files this PR touches; everything else — prompt, model, working directory, config — is identical. Theactortool is denied by permission ({"actor": {"**": "deny"}}), so the call is recorded with its input but no subagent is actually launched. The emitted operation is read from the structured tool part ($.state.input.operation.actionof thepartrow / thetool_useevent), never from prose. The event-derived counts and the persistedpartrows agreed for 36/36 sessions.origin/main)runis CORRECT here)runis CORRECT here)spawncounts trials whose every emittedactorop wasspawn;runcounts trials that emitted at least onerun;neithermeans the agent never calledactorand did the work inline.Verdict — the behaviour claim holds, and the effect is large. On a clear-delegation prompt the blocking path went from 3/6 to 0/6 while
spawnwent from 0/6 to 6/6 (p≈0.002). The fan-out prompt is the sharpest result: run-first emitted three sequentialruns in 6/6 trials — precisely the lost parallelism this PR was written to fix — while spawn-first emitted three parallelspawns in 6/6 trials.Over-correction check — negative, it did not over-correct. On the prompt where
runis the right answer, spawn-first still choserun(2/6, versus 1/6 on run-first); it never refused the blocking path. Reported honestly: on that prompt most trials on both arms skippedactorentirely and just grepped, which is arguably the most correct answer of all for a one-key lookup, so this arm has low power — it can only support "NEW does not forbidrun", not a finer claim.Secondary observation: on the clear-delegation prompt the 3 run-first "neither" trials did the whole investigation inline with ~20 sequential
grep/readcalls (152–224 s), i.e. run-first lost the work to the main context rather than delegating it.Approximate cost: 36 trials ≈ 1.46M input + 5.0M cached + 21k output tokens on
mimo-v2.5, each trial killed as soon as the actor batch closed. The DB records$0.0000because that model entry carries no pricing metadata.Test
Two clearly separate tiers.
(a) Deterministic, runs in CI.
packages/opencode/test/tool/actor.test.ts— newflattened schema offers spawn ahead of run in the operation union. This asserts the wire format the model is actually handed: theanyOfbranch order of the JSON schema aftertransformSchema. It is the machine-checkable half of "spawn is the default", and it was the real gap — the neighbouring test already pinned the union's length (7) but never its order.packages/opencode/test/tool/actor-prompt-spawn-first.test.ts— the pre-existing 14 wording assertions, kept unchanged. They are cheap regression guards for the text, and that is now all they are cited as; they are no longer offered as proof of behaviour.(b) Real-model, opt-in, a no-op in CI.
packages/opencode/test/tool/actor-spawn-preference.test.ts— new, gatedRUN_ACTOR_SPAWN_AB=1following thetest/workflow/verify-wow.test.tsprecedent (aRUN_*env var plus a placeholder test so the file is not empty in CI). It drives the real headless CLI and asserts, over 4 trials of a clear-delegation prompt, that no trial emitsrunand that at least half emitspawn. Credentials are read from~/.config/mimocode/mimocode.jsonbecausetest/preload.tsstrips provider API keys from the environment — gating on a stripped key is the trap that leavestest/session/structured-output-integration.test.tspermanently skipped. Run it with:RUN_ACTOR_SPAWN_AB=1 bun test test/tool/actor-spawn-preference.test.tsCI runs
bun run test:ci(bun test --timeout 30000, 4 shards) with no credentials and noRUN_*set, so tier (a) runs and tier (b) skips.The new deterministic test was proved real by pointing it at the OLD schema (
git checkout origin/main -- src/tool/actor.ts), which fails as expected:Verification
bun typecheck→ exit 0bun test test/tool --timeout 120000→ 747 pass / 0 fail / 11 skip (758 tests, 55 files), against an identical-command baseline on the unmodified tree of 745 pass / 0 fail / 10 skip (755 tests, 54 files). Failure sets are identical (both empty); the delta is exactly the two new tests plus the gated file's skip. Wall time 165 s vs 170 s, so no load confounder.RUN_ACTOR_SPAWN_AB=1 RUN_ACTOR_SPAWN_AB_TRIALS=2→ 1 pass, loggingemitted actor operations per trial: [[],["spawn"]].bun test test/session/orchestrator-prompt.test.ts test/agent/orchestrator.test.ts→ pass (orchestrator wording assertions still hold)Noted, not changed
src/tool/read.ts:237andsrc/session/system.ts:113suggestactor run <type> … --model <vision>for image analysis — that is a genuine one-shot lookup whose result gates the current turn, i.e. the sanctioned exception.src/skill/compose/.bundle/subagent/SKILL.md:65,169dispatches its implementer withactor run … --task T1. That skill's plan → implement → review flow is strictly sequential, so switching it would change workflow semantics rather than just wording.Follow-up:
bd0be88e9— run-exception note moved off the JSON example lineReview nit, fixed in-branch (already an ancestor of this PR's head
9e7f022a2).actor.txt:17used to end a valid JSON example with a trailing prose annotation(
… }} ← exception only), while every other example line in the file is clean,copy-pasteable JSON. Since the model imitates the shape of these lines, the
annotation was a real echo risk (this repo has measured
mimo-v2.5manglingtool-call shapes). The note now sits on its own line above the example:
Diff is 1 file, +2/-1. Scope check: the "at most one
action:run" assertion slicesfrom
indexOf("## Examples"), so this top-of-file compact list was never counted —no assertion needed changing and none was touched.
actor.shell.txt's trailing#annotations are that file's own declared line-comment syntax (:15), so theyare copy-safe and were deliberately left alone;
orchestrator.txtcontains noJSON example lines at all.
Re-verified on the branch head after landing:
bun typecheckexit 0 (12/12), andbun test test/tool/actor-prompt-spawn-first.test.ts test/session/orchestrator-prompt.test.ts test/tool/actor.shell.test.ts test/tool/actor.test.ts→ 78 pass / 0 fail (182 expect() calls, 4 files, 179 s).