feat(flow): local models, a spend cap in every mode, and a pre-run estimate

FLOW.md

@@ -771,6 +771,21 @@ Two shapes that surprise, both learned the hard way:
   (anomalyco/opencode#45424) which this fork's vendored core does not have yet: against an engine
   without it *and* without the config override above, those models read as runnable and still fail
   to dispatch. Drop the id list from `COMPATIBLE_PROVIDERS` if that PR is rejected.
+- **A Claude subscription is not a provider, and no feature should pretend otherwise.** Anthropic
+  does not allow Pro/Max to drive another app, and OpenRouter's BYOK takes API keys only. Claude
+  reaches a card by API key, direct or through OpenRouter (whose BYOK is also how Bedrock/Vertex
+  credits pay for it). README "Using Claude" is the user-facing account.
+- **A local model is declared, with four fields the engine gets wrong by default** (`lib/local.ts`):
+  a `local-` id (a models.dev id such as `lmstudio` is hidden until a key is stored), `tools: true`
+  (default off), a limit (default `{0,0}`), and a **zero price row** — priced at nothing, so the
+  unpriced-never-$0 rule and a gauntlet's cap both still hold. Never special-case local in
+  `usage.ts`; the config price is the whole mechanism.
+- **Every run has a spend cap** (`spendCapOf`; absent means the $5 default, unlike the toggles,
+  because absent-means-off would leave the common case uncapped). Non-gauntlet runs hard-stop in
+  `enforceCap` on the repriced step and end `error` with `RunLog.capped`; a gauntlet keeps its
+  graceful `exhausted()` path at its own cap. The pre-run estimate in `graph/estimate.ts` is the
+  one guessed number here — it stays a range, and seat advice (`weak-seat`) fires only on an
+  inversion inside the canvas, never on a model merely being cheap.
 - `bun test` has **no `localStorage`** (it is a browser global). When persisting a preference
   the browser stores there — custom roles, the default-model preference (`graph/default-model.ts`),
   the walkthrough-dismissed flag — make a Solid signal the source of truth and treat

packages/flow/README.md

@@ -321,6 +321,75 @@ served to the browser.
 | `GET /flow/api/zen-models` | model ids opencode zen really serves, or `null` if unreadable |
 | `GET /flow/api/repackage` | which OpenAI-compatible providers the global config already repackages |
 | `POST /flow/api/repackage` | repackage them there (fixed id list only, backed up, restart required) |
+| `GET /flow/api/local` | local model servers, whether each answers, what it serves, what is declared |
+| `POST /flow/api/local` | `{ id, url? }` — declare that server's models at $0 (backed up, restart required) |
+| `DELETE /flow/api/local/:id` | remove a declared local server |
+
+### Using Claude
+
+Claude runs on a card through the paid API, billed per token. There are two
+ways to reach it:
+
+1. **An Anthropic API key** from `console.anthropic.com`. Connect **Anthropic**
+   in the provider panel and paste it. The runner routes `@ai-sdk/anthropic`
+   directly, so nothing else is needed.
+2. **Through OpenRouter.** Connect **OpenRouter**, then **Repackage** it. It
+   bills OpenRouter credits, or — with OpenRouter's BYOK (bring your own key)
+   setting — your own Anthropic key, or an **AWS Bedrock or Google Vertex**
+   key. That last one is how Claude spend comes out of cloud credits: this
+   engine cannot route Bedrock or Vertex on its own, but OpenRouter can.
+
+A Claude Pro or Max **subscription cannot be used**. Anthropic does not allow a
+consumer subscription to drive another application, and OpenRouter's BYOK takes
+API keys only, so routing through it does not change that.
+
+To keep a Claude run cheap: put the strong model where it pays off — the card
+that dispatches, the critic, the synthesizer — and a cheaper one (Haiku,
+Sonnet) on the cards that do the work. Preflight points out the reverse (a
+`weak-seat` warning) and never changes a model itself. The spend panel shows how
+much of the input came from the provider's prompt cache, which Anthropic bills
+at a tenth of the input rate.
+
+### Local models
+
+Ollama, LM Studio, or any server that speaks OpenAI chat (llama.cpp's server,
+vLLM) can run a card for nothing. Start the server and pull or load a model; the
+provider panel's **Local models** group lists what each one serves, and a click
+declares those models in the global config (`lib/local.ts`) and restarts the
+engine. Clicking again after pulling a new model picks it up.
+
+What is written, and why each part is not optional:
+
+- Ids are prefixed `local-` (`local-ollama/qwen3`). The catalog hides a provider
+  with a models.dev id until a key is stored, and `lmstudio` is one.
+- `tools: true`, because an undeclared model has tool calling switched off and
+  a card with no tools cannot read a file.
+- A context limit, because the engine's default is zero. LM Studio reports the
+  context each model is loaded with (`/api/v0/models`) and that is used; other
+  servers report none, so 32k is written. Raise it on the model in the config
+  file; a later refresh keeps the edit.
+- Only chat models. The OpenAI listing mixes in embedding models; LM Studio's
+  own route says which is which, and elsewhere an id naming `embed` is dropped.
+- A **price of zero**, written as a real price row. The model is priced, so
+  spend reads `$0` and a gauntlet's cap still works — the rule that an unpriced
+  model never shows as `$0` is unchanged.
+
+A small local model is weakest in the seat that writes a control block every
+turn (the orchestrator), which is where preflight's router warning fires for the
+same reason. Use one where a card does the work.
+
+### Spend cap and estimate
+
+Every run has a dollar cap, **$5 unless the canvas says otherwise** (`cap:` in
+the runbar; a gauntlet's own cap replaces it). The engine reprices on every step
+and stops the run the moment priced spend reaches it; the log records `capped`
+and the run ends as an error, not as `stopped`, because nobody decided it. A
+model with no published price makes the cap partial, and preflight says so.
+
+Beside it, the runbar shows a rough range before anything is spent
+(`graph/estimate.ts`): card turns counted from the graph, each priced between a
+short turn and a heavy one at list price. The turn count is exact; the turn size
+is a guess, which is why it is a range and the tooltip says what it assumed.
 
 `POST /api/session/{id}/wait` answers `503 Session wait is not available yet` on
 this server build, so idleness is derived from `/api/session/active` plus a

packages/flow/lib/local.test.ts

@@ -0,0 +1,139 @@
+import { describe, expect, test } from "bun:test"
+import { CUSTOM_LOCAL_ID, LOCAL_LIMIT, declare, declared, discover, normalizeUrl, runtimeFor, undeclare } from "./local"
+
+const ollama = runtimeFor("local-ollama")!
+
+describe("declare", () => {
+  test("writes a v2 provider with tools on, a limit, and a zero price row", () => {
+    const value = declare({ $schema: "x" }, ollama, ["llama3.2", "qwen3"])
+    const entry = value.providers["local-ollama"]
+    expect(entry.api).toEqual({ type: "aisdk", package: "@ai-sdk/openai-compatible", url: "http://localhost:11434/v1" })
+    expect(Object.keys(entry.models)).toEqual(["llama3.2", "qwen3"])
+    expect(entry.models.qwen3.capabilities.tools).toBe(true)
+    expect(entry.models.qwen3.limit).toEqual(LOCAL_LIMIT)
+    expect(entry.models.qwen3.cost).toEqual([{ input: 0, output: 0, cache: { read: 0, write: 0 } }])
+    expect(value.provider).toBeUndefined()
+  })
+
+  // One v1 key sends the whole file through the v1 migration, where
+  // `providers` is never read.
+  test("follows a v1 file's dialect", () => {
+    const value = declare({ plugin: ["./p.js"] }, ollama, ["llama3.2"])
+    const entry = value.provider["local-ollama"]
+    expect(entry.npm).toBe("@ai-sdk/openai-compatible")
+    expect(entry.api).toBe("http://localhost:11434/v1")
+    expect(entry.models["llama3.2"].tool_call).toBe(true)
+    expect(entry.models["llama3.2"].cost).toEqual({ input: 0, output: 0, cache_read: 0, cache_write: 0 })
+    expect(value.providers).toBeUndefined()
+  })
+
+  test("drops models the server no longer lists, keeps edits on the ones it does, and never keeps a price", () => {
+    const before = declare({}, ollama, ["old", "kept"])
+    before.providers["local-ollama"].models.kept.limit = { context: 131_072, output: 8_192 }
+    before.providers["local-ollama"].models.kept.cost = [{ input: 9, output: 9, cache: { read: 0, write: 0 } }]
+    const after = declare(before, ollama, ["kept"])
+    expect(Object.keys(after.providers["local-ollama"].models)).toEqual(["kept"])
+    expect(after.providers["local-ollama"].models.kept.limit.context).toBe(131_072)
+    expect(after.providers["local-ollama"].models.kept.cost[0].input).toBe(0)
+  })
+
+  test("undeclare removes it from either dialect and leaves the rest", () => {
+    const v2 = undeclare(declare({ providers: { openai: {} } }, ollama, ["m"]), "local-ollama")
+    expect(v2.providers).toEqual({ openai: {} })
+    const v1 = undeclare(declare({ plugin: [] }, ollama, ["m"]), "local-ollama")
+    expect(v1.provider).toEqual({})
+  })
+
+  test("declared reads back what was written, and nothing that is not local", () => {
+    const value = declare({ providers: { openrouter: { api: { url: "https://x" } } } }, ollama, ["m"])
+    expect(declared(value)).toEqual([{ id: "local-ollama", url: "http://localhost:11434/v1", models: ["m"] }])
+  })
+})
+
+describe("runtimeFor", () => {
+  test("accepts the fixed ids, with or without an address override", () => {
+    expect(runtimeFor("local-lmstudio")?.url).toBe("http://localhost:1234/v1")
+    expect(runtimeFor("local-ollama", "http://box:11434/v1/")?.url).toBe("http://box:11434/v1")
+  })
+
+  test("a custom server needs an http(s) address, and no other id is accepted", () => {
+    expect(runtimeFor(CUSTOM_LOCAL_ID)).toBeUndefined()
+    expect(runtimeFor(CUSTOM_LOCAL_ID, "file:///etc")).toBeUndefined()
+    expect(runtimeFor(CUSTOM_LOCAL_ID, "http://localhost:8000/v1")?.url).toBe("http://localhost:8000/v1")
+    expect(runtimeFor("openrouter", "http://localhost:8000/v1")).toBeUndefined()
+  })
+
+  test("normalizeUrl rejects what is not a url", () => {
+    expect(normalizeUrl("not a url")).toBeUndefined()
+    expect(normalizeUrl(42)).toBeUndefined()
+  })
+})
+
+describe("discover", () => {
+  test("reads model ids off an OpenAI-compatible listing", async () => {
+    const server = Bun.serve({
+      port: 0,
+      fetch: () => Response.json({ object: "list", data: [{ id: "qwen3" }, { id: "llama3.2" }, { id: "qwen3" }] }),
+    })
+    const found = await discover(`http://localhost:${server.port}/v1`)
+    server.stop(true)
+    expect(found).toEqual({ running: true, models: ["llama3.2", "qwen3"] })
+  })
+
+  // Measured against LM Studio: its OpenAI listing put an embedding model beside
+  // the chat ones, and its own /api/v0 route is what says which is which.
+  test("reads LM Studio's own listing to drop embedding models and take each loaded context", async () => {
+    const server = Bun.serve({
+      port: 0,
+      fetch: (request) =>
+        new URL(request.url).pathname === "/api/v0/models"
+          ? Response.json({
+              data: [
+                { id: "omnicoder-9b", type: "llm", loaded_context_length: 262_144 },
+                { id: "ornith-1.0-9b", type: "llm", max_context_length: 131_072 },
+                { id: "text-embedding-nomic-embed-text-v1.5", type: "embeddings" },
+              ],
+            })
+          : Response.json({
+              data: [{ id: "omnicoder-9b" }, { id: "ornith-1.0-9b" }, { id: "text-embedding-nomic-embed-text-v1.5" }],
+            }),
+    })
+    const found = await discover(`http://localhost:${server.port}/v1`)
+    server.stop(true)
+    expect(found.models).toEqual(["omnicoder-9b", "ornith-1.0-9b"])
+    expect(found.context).toEqual({ "omnicoder-9b": 262_144, "ornith-1.0-9b": 131_072 })
+  })
+
+  test("anywhere else, an id naming an embedding model is dropped", async () => {
+    const server = Bun.serve({
+      port: 0,
+      fetch: (request) =>
+        new URL(request.url).pathname === "/v1/models"
+          ? Response.json({ data: [{ id: "qwen3" }, { id: "nomic-embed-text:latest" }] })
+          : new Response("no", { status: 404 }),
+    })
+    const found = await discover(`http://localhost:${server.port}/v1`)
+    server.stop(true)
+    expect(found).toEqual({ running: true, models: ["qwen3"] })
+  })
+
+  test("declare writes a model's measured context in place of the floor", () => {
+    const value = declare({}, ollama, ["big"], { big: 262_144 })
+    expect(value.providers["local-ollama"].models.big.limit).toEqual({ context: 262_144, output: LOCAL_LIMIT.output })
+  })
+
+  test("a server that is not there is reported as not running, not thrown", async () => {
+    const server = Bun.serve({ port: 0, fetch: () => new Response("") })
+    const port = server.port
+    server.stop(true)
+    expect(await discover(`http://localhost:${port}/v1`)).toEqual({ running: false, models: [] })
+  })
+
+  test("an error status is running, with the reason", async () => {
+    const server = Bun.serve({ port: 0, fetch: () => new Response("no", { status: 404 }) })
+    const found = await discover(`http://localhost:${server.port}/v1`)
+    server.stop(true)
+    expect(found.running).toBe(true)
+    expect(found.error).toContain("404")
+  })
+})