feat(ai-openrouter): video generation adapter (/api/v1/videos) + image activity follow-ups#740
Open
tombeckenham wants to merge 19 commits into
Conversation
* chore(ai-mcp): scaffold @tanstack/ai-mcp package
* feat(ai): expose abortSignal on ToolExecutionContext
* feat(ai-mcp): transport config types and resolver with isolated stdio
* feat(ai): thread chat-run abort signal into tool execution context
* feat(ai-mcp): core types and error classes
* test(ai-mcp): in-memory MCP server helper
* feat(ai-mcp): convert MCP tool definitions to TanStack ServerTools
* feat(ai-mcp): MCPClient connect, tools discovery, and lifecycle
* feat(ai-mcp): definition-binding tools() with MCPToolNotFoundError + duplicate detection
* fix(ai-mcp): type capabilities from the server descriptor generic; drop redundant lazy cast
* fix(ai-mcp): resolve capabilities cast lint in client
* style(ai-mcp): resolve eslint errors across the package
* feat(ai-mcp): createMCPClients multi-server pool with auto-prefix
* feat(ai-mcp): type tools() output from the generated ServerDescriptor generic
* feat(ai-mcp): mcpResourceToContentPart converter + resource test helper
* docs(skills): document MCP tools and tool-context abortSignal
* docs: add MCP server support guide
* chore: changeset for @tanstack/ai-mcp
* feat(ai-mcp): client resources() / readResource() / resourceTemplates()
* feat(ai-mcp): mcpPromptToMessages converter + prompt test helper
* feat(ai-mcp): client prompts() / getPrompt()
* feat(ai-mcp): codegen config schema and loader
* feat(ai-mcp): codegen server introspection
* chore(ai-mcp): sync pnpm-lock.yaml with tsup ^8.5.1 bump
* feat(ai-mcp): codegen emit of ServerDescriptor types from JSON Schema
* feat(ai-mcp): generate CLI command
* test(ai-mcp): cover loadConfig JSON fallback and missing-config error
* test(e2e): MCP server tool discovery and execution in chat()
* docs: document MCP prompts API
* fix(ai-mcp): expose defineConfig via ./cli subpath export
* fix(ai-mcp): emit flat lib build and expose defineConfig from main entry (no jiti in lib)
* ci: apply automated fixes
* feat(ai): MCPToolSource interface and chat mcp option types
* feat(ai): export MCP chat option types
* feat(ai): MCPManager — encapsulates chat mcp discovery + lifecycle
* feat(ai): add mcp option to TextActivityOptions
* feat(ai): wire MCPManager into chat() runners
* test(ai): chat({ mcp }) discovery and lifecycle behavior
* fix(ai): remove unused serverTool import in chat-mcp test
* test(ai-mcp): assert MCPClient/MCPClients satisfy MCPToolSource
* chore: changeset for chat({ mcp })
* docs(skills): document chat({ mcp })
* docs: document chat({ mcp }) for managing MCP clients
* test(e2e): chat({ mcp }) managed-client discovery and execution
* docs(skills): fix inverted connection keep-alive semantics in examples
* fix(ai): export MCPDuplicateToolNameError and correct cross-client collision docs
* chore: ignore .test-d.ts type-test files in knip
* feat(ai-mcp): re-export Transport type and InMemoryTransport for the custom-transport escape hatch
* docs: split MCP docs into focused pages (core client, codegen, chat mcp, manual chat integration)
* example(ts-react-chat): MCP server routes — manual, chat({ mcp }), and pool (keyless stdio reference servers)
* example(ts-react-chat): MCP demo page with manual/chat/pool mode selector
* chore(ts-react-chat): keep dependencies alphabetically sorted
* fix(ts-react-chat): use gpt-5.5 for MCP example routes
* feat(ts-react-chat): add MCP Servers link to sidebar
* fix(openai-base): strip unsupported JSON Schema formats from tool/output schemas
* chore(ts-react-chat): regenerate routeTree in canonical order
* feat(ts-react-chat): pass stable per-mode threadId in MCP demo for devtools grouping
* ci: apply automated fixes
* feat(ts-react-chat): render tool calls and results in MCP demo UI
* ci: apply automated fixes
* test(mcp): cover structured-output runners + real pool passed into chat({ mcp })
* feat(ts-react-chat): switch LLM provider in MCP demo (OpenAI/Anthropic/Gemini/Groq)
* ci: apply automated fixes
* test(e2e): cover MCP resource/prompt read + connection lifecycle (keep-alive vs close)
* feat(ts-react-chat): add /api/mcp-status endpoint and OpenRouter provider (default)
* ci: apply automated fixes
* fix(ai-mcp): apply PR review fixes, document OAuth, use TanStack Start in examples
Review fixes:
- docs: correct chat() signature (adapter: openaiText('gpt-5.5'), no top-level model)
- docs/skills: fix close-before-consume lifecycle samples — close MCP clients in
middleware terminal hooks (onFinish/onAbort/onError), not try/finally or
await using around a streaming return
- skills: repair truncated/unbalanced code fence corrupting the Provider Skills
section in tool-calling SKILL.md
- example: settle parallel createMCPClient calls in api.mcp-chat.ts and close the
connected sibling on partial failure (no leaked stdio subprocess)
- example: probe capabilities in api.mcp-status.ts instead of catch-all-ing
resources()/prompts()
- ai-mcp: name the tool in MCP isError throws; short-circuit already-aborted
signals; pool.tools() now attributes the failing server by config key
- ai(chat): correct MCPConnectionPolicy JSDoc ('close' = when the run ends;
'keep-alive' = never closed by chat())
- types: collapse redundant AutomaticDescriptor; reword DescribedTool/codegen
claims to name-only typing (args stay untyped on discovery; Mode 2 types args)
New coverage (ai-mcp 30 → 46 tests):
- isError path, abort→callTool forwarding, already-aborted short-circuit,
structuredContent preference, mcpContentToTanstack branches
- replace tautological duplicate-name test with one exercising the real guard
- connect-failure wrapping, double-close idempotency, bound-defs prefix,
pool tools() failure attribution, stdio smoke test, authProvider forwarding
- types.test-d.ts pins the descriptor name-literal guarantee
Docs additions:
- Authentication section (headers + OAuth authProvider + finishAuth caveat) in
mcp.md and the ai-mcp skill
- all MCP route examples converted from Next.js App Router to TanStack Start
(createFileRoute + server handlers); no Next.js references remain in PR files
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ai-mcp): exclude task-required tools from discovery, throw on explicit binding
Newer MCP servers (e.g. @modelcontextprotocol/server-everything's
simulate-research-query) mark tools with `execution.taskSupport: 'required'`
(experimental MCP tasks). Plain `callTool` — what @tanstack/ai-mcp uses — is
rejected by the server with -32600, so offering these tools to the model
guarantees a failed tool call.
- tools(): auto-discovery now filters task-required tools via the new
`requiresTaskExecution()` guard — the model is never offered a tool that
cannot succeed
- tools([defs]): explicitly binding a task-required tool throws the new
`MCPTaskRequiredToolError` (exported) with guidance pointing at the SDK's
tasks API
- tests: in-memory server helper registering a real task-required tool;
discovery-exclusion + binding-throws coverage (48 tests)
- e2e: in-process MCP server registers a task-required tool; spec asserts it
never reaches the discovered tool list
- docs: Mode 1 exclusion callout, Mode 2 error mention, Error Reference row
in mcp.md; matching error-list updates in the ai-mcp skill
Actual task-based execution support (client.experimental.tasks.callToolStream)
is tracked in #704.
Also includes import-order/lint cleanup in examples/ts-react-chat
api.mcp-manual.ts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(mcp): rename guide pages to managed/manual, simplify examples
- rename mcp-chat.md → mcp-managed.md ("Managed MCP with chat()") and
mcp-with-chat.md → mcp-manual.md ("Manual MCP: typed tools, resources &
prompts") — the old slugs differed only by a preposition while the real
distinction is managed vs manual lifecycle; new names match the vocabulary
already used by the e2e/example routes (api.mcp-managed-test, api.mcp-manual)
- update all ids, cross-links, link texts, and docs/config.json nav labels
- Quick Start (mcp.md) now leads with the managed mcp option (zero lifecycle
code) instead of the manual middleware-close pattern; Lifecycle section
opens with the managed escape hatch before the manual rules
- mcp-managed.md: keep one full route example (plus keep-warm, where module
vs handler placement is the point); convert the other five examples to
focused fragments showing just the client setup + chat() call; drop the
repeated 8-line body-validation boilerplate (449 → 306 lines)
- reorder sidebar/frontmatter to adoption order:
mcp → mcp-managed → mcp-manual → mcp-codegen
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: apply CodeRabbit review fixes across ai-mcp, chat core, and examples
- emit.ts: guarantee valid/unique generated identifiers, escape string
literals via JSON.stringify when emitting TypeScript
- introspect.ts: move connect() inside try/finally, guard close() from
masking the original error, drain cursor-paginated list endpoints
- prompts.ts: never produce undefined content (JSON.stringify(x ?? null))
- manager.ts: await onDiscoveryError so async handlers fail fast
- chat: combine caller + middleware abort signals so ctx.abort() reaches
running tools via ctx.abortSignal (regression test added)
- mcp routes (e2e + example): close MCP client on non-stream error paths,
bridge request aborts during setup
- mcp-demo/threads: pass remark-gfm via remarkPlugins (not rehypePlugins)
- tests: fix import/order lint errors, assert pool cleanup on failure,
cover hostile-name escaping and interface collisions in emit
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Added extra link to the docs
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Tom Beckenham <34339192+tombeckenham@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…mparison (#708) docs: add addedAt to docs config entries and update MCP in AI SDK comparison - Add an addedAt field (YYYY-MM-DD) to every page entry in docs/config.json, dated from when each entry first appeared in the docs navigation. - Update the TanStack AI vs Vercel AI SDK comparison to reflect the new standalone host-side MCP client (@tanstack/ai-mcp): new MCP section, updated feature-table row, and removal of MCP as a Vercel-only advantage.
docs: give MCP its own sidebar section and shorten long nav labels - Move the four MCP pages out of Tools into a dedicated MCP sidebar section. - Shorten 'Manual MCP: Tools, Resources & Prompts' to 'Manual MCP'. - Shorten 'Sampling Options to modelOptions' to 'Sampling → modelOptions'.
…ing docs (#711) Document in CLAUDE.md and AGENTS.md that touching a docs page must update its docs/config.json entry: addedAt for new pages, updatedAt for content changes. Pure bug fixes (typos, broken links, code-fence languages, formatting) must not bump either date.
36 code blocks containing JSX (React/Solid/Preact components) were fenced as ```typescript. Re-fence them as ```tsx so editors and the docs site highlight the JSX correctly. No ```ts blocks contained JSX; the mislabeled ones were all ```typescript.
* feat(fal): surface billed cost as result.usage.unitsBilled The fal adapters discarded fal's response headers, so the actual billed cost of a generation was unrecoverable through the SDK. fal returns the real billed quantity in the `x-fal-billable-units` header on the result fetch; this surfaces it as `result.usage.unitsBilled` so consumers can compute exact media-generation cost without wrapping `fetch` themselves. - `TokenUsage` gains an optional `unitsBilled` (a bare count of priced units, sibling to `durationSeconds`; the unit name itself is provider -defined and looked up via the pricing API, not carried here). - A `config.fetch` wrapper reads `x-fal-billable-units` off every fal response, keyed by `x-fal-request-id` (the same value the client surfaces as `Result.requestId`), so the adapter's lookup always matches the fetch the units came from. `config.fetch` is used rather than `responseHandler` because the queue ops clobber a global handler. - All five fal media adapters (image, audio, video, speech, transcription) populate `result.usage.unitsBilled` when fal reports it. - `VideoUrlResult` gains a `usage` slot; `getVideoJobStatus` now emits the `video:usage` event and returns `usage` on completion. Closes #722 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * feat(examples): show fal unitsBilled in ts-react-media Surface the new `result.usage.unitsBilled` in the media example so the billed quantity is visible after a generation — a caption under each generated image/video ("Billed N fal units"). Verified against the live fal API: a fal-ai/flux/schnell generation reports unitsBilled: 1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(examples): use a runtime-valid size for grok-imagine-image fal's generated `size`/`resolution` type for `xai/grok-imagine-image` offers `16:9_1K` / `16:9_4K`, but the live API rejects those resolutions ("Input should be '1k' or '2k'") — the vendor enum is out of sync with the API. `'16:9_4K'` therefore type-checked but 422'd at runtime. Pass `aspect_ratio: '16:9'` via modelOptions instead and let the endpoint default the resolution; verified against the live fal API. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(fal): inject fetch instead of overriding the global Address CodeRabbit + review feedback: dependency-inject the underlying fetch rather than mutating globals. - `createBillingFetch(baseFetch = globalThis.fetch)` now takes the fetch to delegate to, and `FalClientConfig` gains an optional `fetch` override that `configureFalClient` wraps for usage capture. - The E2E route passes a per-request redirecting `fetch` via `falImage(model, { fetch })` instead of swapping `globalThis.fetch` — removing the concurrency race CodeRabbit flagged (no global mutation, no try/finally restore). - `billing.test.ts` injects a fake fetch directly instead of `vi.stubGlobal('fetch')`. - Changeset: reword "billed cost" → "billed units" to match the surfaced `usage.unitsBilled` field. Verified: full `pnpm test:pr` green, fal E2E spec green, and a live fal-ai/flux/schnell call still reports unitsBilled via the default (no-override) path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: apply automated fixes --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…test (#725) Follow-up to #723 addressing CodeRabbit review feedback. - Add unit coverage for `result.usage.unitsBilled` on the fal speech and transcription adapters (billed + unbilled cases), mirroring the existing audio/image/video adapter tests. - Remove the e2e `fal-billable-units` spec, its `/api/fal-billable-units` route, and the hand-rolled `/fal-queue` aimock mount. aimock has no seam to stamp the `x-fal-billable-units` response header the feature reads, so any e2e test required manipulating `fetch` to redirect fal's hardcoded `queue.fal.run` URLs. The billed-units behavior is now covered by unit tests across all five fal adapters instead. - Drop the now-orphaned `@tanstack/ai-fal` dependency from the e2e app and regenerate the route tree. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
) * chore(ai-gemini): sync model meta with Google's current model list Remove retired Gemini ids (gemini-3-pro-preview, the 09-2025 2.5 previews, and the 2.0 line), add the stable gemini-3.1-flash-lite, and migrate every workspace consumer off the removed ids. Also fixes the structured-output demo dropdown, which still offered the retired gemini-3-pro-preview as its default. All removals were verified against the live API, not just the docs page. Includes a couple of small fixes found along the way: missing gemini-3.5-flash entries in the tool-capability and modality maps, and stale display labels left next to migrated model values. Closes #620, closes #621 * docs(ai-gemini): list stable gemini-3.1-flash-lite and correct document-input note Add the stable gemini-3.1-flash-lite row to the skill reference's Key Chat Models table, and fix the modality note that wrongly claimed all Gemini text models accept document input — gemini-2.5-flash does not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ai-openai): migrate WebRTC realtime adapter to OpenAI GA API * Merge branch 'main' into fix/openai-realtime-ga-migration * fix(ai-openai): complete realtime Beta-to-GA migration Completes the GA migration started in this PR so the whole realtime flow works against OpenAI's GA API (the Beta shape was shut down 2026-05-12): - openaiRealtimeToken() mints ephemeral keys via POST /v1/realtime/client_secrets (the Beta /v1/realtime/sessions endpoint is retired) and parses the GA top-level value/expires_at response shape - session.update payloads use the GA shape via a new pure buildSessionUpdate() helper: required session.type, audio.input.*, audio.output.voice, output_modalities, max_output_tokens; temperature (removed in GA) is dropped with a debug log instead of getting the whole update rejected with unknown_parameter - server events handled under GA names (response.output_audio_transcript.*, response.output_audio.*, output_text/output_audio content parts) - removed the now-unused model local in createWebRTCConnection (the GA /calls endpoint rejects ?model=; the model is bound to the ephemeral key) - default model gpt-realtime; dead gpt-4o-(mini-)realtime-preview ids (shut down 2026-05-07) removed from OpenAIRealtimeModel, docs, and examples - unit tests for the session.update payload and client-secret request/response shapes; changeset added Live-verified against the OpenAI API: client_secrets 200 (ek_ token), /v1/realtime/calls 201 with SDP answer, and session.updated echoing voice, semantic VAD, tools, output_modalities, and max_output_tokens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai-openai): collapse output modalities to single GA-supported value The GA realtime API only accepts ['audio'] or ['text'] for output_modalities; the Beta API accepted ['audio', 'text'] and the provider-agnostic RealtimeSessionConfig still legitimately produces it (e.g. the example UI's audio+text mode). Sending both got the whole session.update rejected with: Invalid modalities: ['audio', 'text']. Collapse to ['audio'] when audio is requested — GA audio replies still stream text via response.output_audio_transcript.* events, so visible behavior is unchanged. Live-verified: session.updated accepted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Merge branch 'main' into fix/openai-realtime-ga-migration
* feat(ai): add 'error' terminal to ToolCallState (#718) When a tool execution produced an output error, the tool-call UIMessage part parked at "input-complete" forever, so UIs rendering lifecycle from part.state could not distinguish "still executing" from "failed" without reverse-engineering the error-shaped output or the sibling tool-result part. Add an 'error' member to ToolCallState (in @tanstack/ai, @tanstack/ai-client, and @tanstack/ai-event-client) and transition the tool-call part to it on an output error, making the tool-call state machine self-describing and symmetric with ToolResultState. - StreamProcessor maps output errors to state 'error' in addToolResult, handleToolCallEndEvent, and handleToolCallResultEvent. - The completion safety net (RUN_FINISHED / finalizeStream) finalizes the call's internal state but no longer downgrades a rendered 'error' part back to 'input-complete', including when an output-error result arrives before TOOL_CALL_END. - A failed client tool with an empty error message now still reaches 'error' (error-ness comes from the output-error state, not message truthiness). - isToolCallIncluded keeps errored tool calls in conversation history. Adds unit coverage in @tanstack/ai and @tanstack/ai-client plus an E2E assertion in tool-error.spec.ts. * refactor(ai-client): derive tool-result error signalling solely from result.state Pass an error message to processor.addToolResult only for output-error results; pass undefined otherwise. Previously the non-error branch forwarded result.errorText, so a stray errorText on a successful result could be misread as an error by addToolResult's message-truthiness check.
…ive Cloudflare Workers (#734) * fix(ai): render debug meta portably in ConsoleLogger for Cloudflare Workers (#730) workerd never forwards console.dir output to the terminal, so the default debug logger printed category headlines but dropped every meta payload on Cloudflare Workers. ConsoleLogger now picks a per-runtime strategy: Node keeps the depth-unlimited console.dir dump, Workers gets circular-safe pretty-printed JSON appended to the message, and other runtimes receive meta as a second console argument. Detection checks workerd's navigator.userAgent marker before process.versions.node, since nodejs_compat emulates a Node version string and would otherwise route Workers down the broken Node path. Adds a Miniflare-based e2e spec that runs the built logger inside real workerd and asserts payloads reach the log stream at full depth. * test(e2e): document why the workerd logger spec doesn't route through aimock aimock mocks provider LLM responses; this spec exercises console rendering inside the workerd runtime, a path provider HTTP never reaches. Wiring aimock in would mean bundling the whole chat() + adapter stack into the Miniflare worker without covering any additional fixed behavior. * ci: apply automated fixes
…ols (#696) * fix(ai): stop emitting duplicate TOOL_CALL_END for server-executed tools Adapters already stream START/ARGS/END for every tool call, but the post-execution phase in chat() pushed a second TOOL_CALL_END with no preceding TOOL_CALL_START. AG-UI-strict consumers (e.g. @ag-ui/client's verifyEvents) reject an orphan END and abort the stream. buildToolResultChunks now emits END only alongside the matching START it already gates on argsMap (the continuation re-execution path, which reconstructs a never-streamed call); the normal post-execution path contributes only TOOL_CALL_RESULT. The result/error state lives on the spec-compliant TOOL_CALL_RESULT, so nothing is lost. Fixes #519. * test(e2e): add #519 regression for duplicate TOOL_CALL_END Drive chat() with a server tool through an adapter that streams its own START/ARGS/END (as real adapters do) and assert the emitted lifecycle is balanced: exactly one TOOL_CALL_END per tool call, each preceded by a matching START — the invariant @ag-ui/client's verifyEvents enforces. Reproduces #519 (the orphan duplicate END) end-to-end. * test(e2e): consume open START on END in pairing invariant A second TOOL_CALL_END for the same toolCallId now fails the pairing check directly, matching how @ag-ui/client's verifyEvents closes open tool calls. Addresses CodeRabbit review feedback on #696. * fix(ai-event-client): emit devtools tool-result from TOOL_CALL_RESULT The devtools middleware sourced tool results from TOOL_CALL_END, which the adapter emits before execution (no result attached). With the #519 fix in @tanstack/ai, the post-execution END is no longer re-emitted, so results now travel on the spec-compliant TOOL_CALL_RESULT event. Handle it so server-executed tool output keeps reaching devtools.
…video-support' into 707-featai-openrouter-video-generation-adapter-apiv1videos-+-image-activity-follow-ups # Conflicts: # packages/ai-fal/tests/video-adapter.test.ts
…e activity follow-ups Closes #707. - Add openRouterVideo: async jobs adapter for OpenRouter's dedicated video API (submit -> poll -> download). Per-model size/duration/option types are generated from GET /api/v1/videos/models; frame roles map onto frame_images[] / input_references[] per the MediaInputRole taxonomy. - Teach the model-meta sync scripts the videos/models endpoint (openrouter.video-models.json + OPENROUTER_VIDEO_MODEL_META). - Image adapter follow-ups from the #624 review: throw on unmapped sizes (the size union used a Unicode multiplication sign so every non-square size silently dropped its aspect ratio), throw on numberOfImages > 1 (live-verified: the gateway ignores all count keys), expose image_config.strength. - Completed videos are returned as data: URLs (unsigned_urls 401 without the API key header) with gateway-reported cost on usage.cost. The SDK's getVideoContent is bypassed: its matcher only accepts application/octet-stream while the endpoint serves video/mp4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Contributor
🚀 Changeset Version Preview8 package(s) bumped directly, 23 bumped as dependents. 🟥 Major bumps
🟨 Minor bumps
🟩 Patch bumps
|
|
View your CI Pipeline Execution ↗ for commit 413f0a7
☁️ Nx Cloud last updated this comment at |
@tanstack/ai
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-mcp
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
The getVideoContent response-matcher bug is still present in 0.12.79 (the stream matcher only accepts application/octet-stream while the endpoint serves video/mp4), so the direct unsigned-URL download stays. Link the aimock feature request (CopilotKit/aimock#261) from the e2e matrix exclusion. Submit/poll/download lifecycle re-verified live on the new SDK. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Changes
Implements #707 (follow-up to #618 / #624). Stacked on #624 — based on
618-image-to-image-and-image-to-video-support; rebase ontomainonce that merges.openRouterVideoadapterPOST /api/v1/videos→ pollGET /api/v1/videos/{jobId}→ download) — Seedance 2.0, Veo 3.1, Wan, Kling, Sora 2 Pro through one key, on the same jobs/polling architecture as the Sora adapter.imageInputsrole mapping per the issue table:start_frame/end_frame→frame_images[](first_frame/last_frame),reference/character→input_references[],mask/control→ throw, unroled image → start frame. Frame roles are validated against each model'ssupported_frame_images.videoInputs/audioInputsthrow (unsupported by the API).size/duration/resolution/aspectRatiotypes and runtime validation are generated fromGET /api/v1/videos/models(OPENROUTER_VIDEO_MODEL_META);seed/generateAudioare dropped from the type for models whose metadata reports them unsupported. The sync scripts (fetch-openrouter-models.ts/convert-openrouter-models.ts) now fetch and convert the second endpoint.data:URLs —unsigned_urls401 without the API key header (verified live), so they can't go straight into a<video>tag. Downloads >10 MiB log the same OOM warning as the OpenAI adapter. Gateway-reported cost is surfaced asusage.cost.getVideoContent: its response matcher (as of@openrouter/sdk0.12.35) only acceptsapplication/octet-streamwhile the live endpoint servesvideo/mp4— worth reporting upstream. Download uses a config-injectablefetchseam.Image activity follow-ups (#624 review)
sizenow throws with the supported list. Root cause found: theOpenRouterImageModelSizeByNameunion used the Unicode×(U+00D7) while the lookup table used ASCIIx, so every typed size except1024x1024silently dropped its aspect ratio. Union fixed to ASCII;×still normalized at runtime.numberOfImages > 1now throws. Live-verified: the chat-completions pathway ignores every count key (numberOfImages,num_images,count) and always returns one image. Also confirmedimage_configcasing live: snake_caseaspect_ratiochanges output dimensions, camelCase is silently ignored — comment added at the request site.image_config.strength(0.0–1.0 i2i influence) exposed viamodelOptions.strength.Tests / E2E / Docs
feature-support.ts— aimock 1.29 only mocks the OpenAI-shaped/v1/videos, not OpenRouter'spolling_url/unsigned_urlsjob shape (same constraint noted in feat: multimodal prompt for generateImage/generateVideo (image-to-image, image-to-video) #624).media/video-generation.md,media/image-generation.md,adapters/openrouter.md(+config.jsondates), media-generationSKILL.md, changeset (minor).✅ Checklist
pnpm run test:pr.🚀 Release Impact
🤖 Generated with Claude Code