docs(agent-eval-adoption): dynamic driver + optimizePrompt; fix stale topology claims#6
Conversation
…tale topology claims
The skill predated agent-runtime 0.33.0 and was steering adopters wrong:
- "Topologies — Refine + FanoutVote shipped; others deferred" told people the
Decompose topology was NOT shipped and to fork a custom Driver. createDynamicDriver
+ createSandboxPlanner (agent-authored topology) ship in 0.33.0 and subsume it.
- createFanoutVoteDriver signature was stale ({variants, scoreFn} → actual {n, selector}).
- No mention of optimizePrompt (identity-gated prompt-surface optimization).
Changes:
- Refresh the Topologies section: add createDynamicDriver/createSandboxPlanner,
move Decompose out of "deferred", correct the fanout-vote signature. Council +
Pipeline remain the only deferred topologies.
- New "Prompt-surface optimization — optimizePrompt (identity-gated)" subsection:
the reusable [SURFACE]-prompt optimization recipe + the gepaDriver footguns
(text/H2-only, domain-real scenarios, extend-don't-fork, cost via ctx.cost).
- Update frontmatter + plugin.json descriptions for discoverability (0.33.x).
drewstone
left a comment
There was a problem hiding this comment.
✅ Auto-approved tangletools PR — 93f4cd24
This PR was opened by the trusted tangletools automation account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: tangletools_author · 2026-05-31T01:22:32Z
✅ No Blockers —
|
| deepseek | kimi-code | aggregate | |
|---|---|---|---|
| Readiness | 82 | 82 | 82 |
| Confidence | 65 | 65 | 65 |
| Correctness | 82 | 82 | 82 |
| Security | 82 | 82 | 82 |
| Testing | 82 | 82 | 82 |
| Architecture | 82 | 82 | 82 |
Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.
🟠 MEDIUM Key docs section retains stale package version references inconsistent with updated frontmatter — plugins/agent-eval-adoption/skills/agent-eval-adoption/SKILL.md
The frontmatter description was updated to reference
@tangle-network/agent-eval (0.50.x+)and@tangle-network/agent-runtime (0.33.x+), and the body documentscreateDynamicDriver/optimizePromptasagent-runtime 0.33.0+APIs. However, the Key docs section at lines 1118–1126 was left unchanged and still lists@tangle-network/agent-eval@0.36.xREADME and@tangle-network/agent-runtime@0.28.x/loops. An adopter following the Key docs index may look at the wrong package version documentation and fail to find the newly documented APIs. Fix: update Key docs to@tangle-network/agent-eval@0.50.x+README and `@tangle-netw
🟠 MEDIUM Stale version references in Key docs section after description bump — plugins/agent-eval-adoption/skills/agent-eval-adoption/SKILL.md
L1118 references
@tangle-network/agent-eval@0.36.xbut the description frontmatter (L3) was bumped to0.50.x+. L1122 references@tangle-network/agent-runtime@0.28.x/loopsbut the description now says0.33.x+and new features (createDynamicDriver L145-146, optimizePrompt L180) require 0.33.0+. The agent-runtime mismatch is clearly wrong and will misdirect readers to stale API docs. Fix: update L1118 to0.50.xand L1122 to0.33.xto match the frontmatter version target.
🟡 LOW Comment references wrong variable name in optimizePrompt example — plugins/agent-eval-adoption/skills/agent-eval-adoption/SKILL.md
L194 comment says
assign result.prompt unconditionallybut the destructuring at L187 bindspromptdirectly (notresult.prompt). The comment should saypromptto match the destructured variable. Minor reader confusion risk.
🟡 LOW optimizePrompt example destructures prompt but prose refers to result.prompt — plugins/agent-eval-adoption/skills/agent-eval-adoption/SKILL.md
Lines 187–194 destructure the return value as
const { prompt, improved, decision, delta } = await optimizePrompt(...), yet the comment on line 194 says// assign result.prompt unconditionally, and the prose on line 199 says`result.prompt` is the baseline UNLESS `decision === 'ship'`. A reader copying the d
tangletools · 2026-05-31T01:29:30Z · trace
The
agent-eval-adoptionskill predated agent-runtime 0.33.0 and was activelysteering adopters wrong:
shipped; others deferred" and explicitly told people the Decompose
topology was NOT shipped and to fork a custom
Driver. ButcreateDynamicDrivercreateSandboxPlanner(agent-authored topology) ship in 0.33.0 and subsume it.createFanoutVoteDriversignature was wrong ({variants, scoreFn}→ actual{n, selector}).optimizePrompt(identity-gated prompt-surface optimization).Changes:
createDynamicDriver/createSandboxPlanner,move Decompose out of "deferred", correct the fanout-vote signature. Council +
Pipeline remain the only deferred topologies.
optimizePrompt(identity-gated)":the reusable
[SURFACE]-prompt optimization recipe (extract prompt → domainscenarios → judge dims →
runWithPrompt→ gepaDriver + held-out gate) and thefootguns learned the hard way — gepaDriver is text/H2-only, scenarios must be
domain-real, extend-don't-fork an existing
runImprovementLoop, cost viactx.cost.plugin.jsondescriptions updated for discoverability (0.33.x).Pairs with the self-contained
agent-runtime-adoptionskill now shipped insidethe agent-runtime repo (tangle-network/agent-runtime#77), so external consumers
of that package need nothing from this marketplace.