Repository navigation
Provider registry: subscription-auth harness backends; cursor-agent Strict backend (M1) - #81
Merged
Merged
Conversation
…sor-agent Strict backend (M1) Strict previously required a raw Anthropic/OpenAI API key. This lands the provider registry (doc/RFC-provider-registry.md): one table of ways to reach a model, with the existing raw-API path unified under it as provider "api" (byte-identical parity, existing suite is the gate) and harness-kind providers for subscription-authed vendor CLIs. cursor-agent ships as the first harness backend; claude-code/codex are declared and ship in M2/M3. - providers/: ProviderSpec + Transport contracts, registry, HarnessLLM (tool-less create() adapter; raises on tools), cursor transport (stream-json parse, sanitized allowlist env, scratch-cwd one-shots), productized session audit (from the Composer eval arm) - tool_bridge.py: stdio MCP server exposing the run's ToolScope-checked tools to harness sessions via tools.dispatch, plus read containment (reads confined to worktree+run dir — ToolScope path-guards only writes) and a separate bridge_trace.jsonl - config: STRICT_BACKEND selection (explicit; Strict refuses when unset, CLI defaults to api), ResolvedTarget gains provider_id/kind, tier_target returns harness targets with no endpoint/credential - run_agent_step branches on target.kind: harness steps delegate the whole step (same prompt bundle, output contract, coercion, salvage); MoA members keep the in-process loop - strict_readiness + doctor report backend selection/CLI gaps with exact fixes; .env.template documents the new keys Offline coverage: registry selection, harness target shape, HarnessLLM guards, bridge spec round-trip + containment refusals, cursor transport against a fake CLI (session, audit flags, timeout, one-shot, env sanitization). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…it widened to all path-bearing tools Findings from the first live Strict runs on a real Cursor subscription (PR #81 e2e, three runs on vllm-omni#6199): - headless cursor-agent does not auto-approve configured MCP servers; the tool bridge was silently ignored until --approve-mcps was passed. With the flag, bridged grep/read_file calls flow through tools.dispatch and land in bridge_trace.jsonl (verified live: 26 and 8 bridge events). - live usage keys are camelCase (inputTokens/outputTokens); the parser accepted only snake_case and recorded zeros. - the audit checked containment only on readToolCall; native grepToolCall (and friends) carry paths too and sailed past a read-only audit — every path-bearing native call is now checked, and the misleading bridge_calls trace field is renamed other_tool_calls. - cursor-agent spools MCP tool RESULTS under ~/.cursor/projects/*/ agent-tools/ and reads them back; that read-back is bridge-output consumption, now exempt from root containment instead of flagging every bridge-using session. Final verification run: audit_ok=true, zero violations, bridge engaged, review success on the PR-time tree. Full offline suite green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
Live e2e: PR review in all three modes ✅Ran the M1 acceptance smoke end-to-end on this branch — real MCP stdio sessions against the thin server, real PRs, real backends. All three modes pass; the cursor runs surfaced four real bugs, fixed in a8311cc.
Live findings → fixes (a8311cc)
Env notes: harness subprocess env sanitization verified live ( Full offline suite green after fixes (26 provider/bridge/cursor tests). Run artifacts: 🤖 Generated with Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #80 (RFC — provider registry). Implements M1: the unified provider registry, the MCP tool bridge, and the cursor-agent harness backend. claude-code (M2) and codex (M3) are declared in the registry but unshipped — selecting them reports the milestone in
strict_readiness/doctor instead of failing mid-run.What this does
Strict previously required a raw Anthropic/OpenAI API key. With this PR,
.envselects the execution backend explicitly:Selection is explicit by design: a Strict run with
STRICT_BACKENDunset refuses upfront with the exact fix line (same philosophy asTierNotConfiguredError); the CLI path defaults toapiso existing setups keep working..env.templatenow writesSTRICT_BACKEND=api.Design (details in #80 /
doc/RFC-provider-registry.md)ResolvedTargetgainsprovider_id/kind;tier_targetstays the only resolution site. The only runtime branch is inrun_agent_step: harness targets delegate the whole agent step (run_session) with the same prompt bundle, output contract, coercion and salvage; api targets are byte-identical to before (existing suite is the parity gate). MoA members keep the in-process loop — they carry their own API endpoints.tool_bridge.pyis an stdio MCP server the harness session launches; every builtin tool call passestools.dispatchwith the step's deserializedToolScope. Reads are additionally confined to the PR-time worktree + run dir (ToolScopepath-guards only writes) — the.env-exfiltration guard. Bridge tool events land inbridge_trace.jsonl.eval/dataset/run_cursor_arm.py, which caught a real out-of-bounds read): containment violations and read-only write attempts are traced (harness_sessionevent) and surfaced as refusals.ANTHROPIC_BASE_URL/API keys (a gateway-pointed base URL would misroute subscription calls).HarnessLLMserves tool-lesscreate()calls (reducer, repair, intent) via one-shot CLI invocations in an empty scratch cwd, and raises loudly if anything passes tools.Known M1 gaps (disclosed in the RFC)
StepContext(skill/memory retrieval, repo_map) are not reconstructed in the bridge — harness sessions get builtins +doc_search/doc_read.Verification
test_providers.py,test_tool_bridge.py,test_provider_cursor.py, extendedtest_mcp.pyreadiness tests).ruff checkclean on all touched files;git diff --checkclean.STRICT_BACKEND=apiparity: zero changes tollm.py/agent_loop.py; existing tests unchanged except the threestrict_readinesstests that now set the backend explicitly (the designed migration).🤖 Generated with Claude Code