diff --git a/.changeset/openrouter-video-adapter.md b/.changeset/openrouter-video-adapter.md new file mode 100644 index 000000000..d1b9e7f7d --- /dev/null +++ b/.changeset/openrouter-video-adapter.md @@ -0,0 +1,7 @@ +--- +'@tanstack/ai-openrouter': minor +--- + +Add `openRouterVideo`, a video generation adapter for OpenRouter's dedicated async API (`POST /api/v1/videos`) — Seedance, Veo 3.1, Wan, Kling, and Sora 2 Pro through one API key. Follows the jobs/polling architecture (`generateVideo()` → `getVideoJobStatus()`), with per-model `size` / `duration` / provider-option types generated from OpenRouter's `GET /api/v1/videos/models` metadata and validated before submit. Image-conditioned prompts map `metadata.role` onto the wire: `start_frame` / `end_frame` → `frame_images[]` (`first_frame` / `last_frame`), `reference` / `character` → `input_references[]`; frame roles are validated against each model's `supported_frame_images`. Completed videos are downloaded server-side and returned as `data:` URLs (OpenRouter's download URLs require the API key), and the gateway-reported cost is surfaced as `usage.cost`. + +Image adapter fixes from the #624 review: requested `size` is now validated (the `WIDTHxHEIGHT` union previously used a Unicode `×`, so every size except `1024x1024` silently dropped its aspect ratio; unsupported sizes now throw with the supported list), `numberOfImages > 1` throws instead of silently returning one image (verified live: the gateway ignores all count keys in `image_config`), and `image_config.strength` (0.0–1.0 image-to-image influence) is exposed via `modelOptions.strength`. diff --git a/docs/adapters/openrouter.md b/docs/adapters/openrouter.md index cd8984511..eff6d0aec 100644 --- a/docs/adapters/openrouter.md +++ b/docs/adapters/openrouter.md @@ -219,6 +219,91 @@ fields are simply absent and the stream completes normally. Both `openRouterText` and `openRouterResponsesText` populate cost when OpenRouter returns it. +## Image Generation + +`openRouterImage` routes image generation through OpenRouter's +chat-completions surface (`modalities: ['image']`). Multimodal prompts are +supported — text and image parts are forwarded in order for +image-conditioned generation: + +```typescript +import { generateImage } from "@tanstack/ai"; +import { openRouterImage } from "@tanstack/ai-openrouter"; + +const result = await generateImage({ + adapter: openRouterImage("google/gemini-2.5-flash-image"), + prompt: "A watercolor lighthouse at dusk", + size: "1344x768", // mapped to image_config.aspect_ratio ('16:9') + modelOptions: { + image_size: "2K", // resolution (Gemini models) + strength: 0.35, // image-to-image influence, i2i-capable models only + }, +}); +``` + +Notes: + +- The pathway returns **exactly one image per request** — `numberOfImages > 1` + throws instead of silently under-delivering. Make multiple requests if you + need multiple candidates. +- `size` must be one of the ten supported `WIDTHxHEIGHT` values (it is + converted to `image_config.aspect_ratio`); anything else throws with the + supported list. + +## Video Generation (Experimental) + +`openRouterVideo` targets OpenRouter's dedicated **async video API** +(`POST /api/v1/videos`) — Seedance, Veo 3.1, Wan, Kling, and Sora 2 Pro +through your one OpenRouter key. It follows the jobs/polling architecture +shared by all TanStack AI video adapters: + +```typescript +// Server: create the job, then poll +import { generateVideo, getVideoJobStatus } from "@tanstack/ai"; +import { openRouterVideo } from "@tanstack/ai-openrouter"; + +const adapter = openRouterVideo("bytedance/seedance-2.0"); + +const { jobId } = await generateVideo({ + adapter, + prompt: [ + { type: "text", content: "Animate this product shot, slow push-in" }, + { + type: "image", + source: { type: "url", value: "https://your-cdn.com/product.png" }, + metadata: { role: "start_frame" }, + }, + ], + size: "1280x720", + duration: 8, +}); + +let status = await getVideoJobStatus({ adapter, jobId }); +while (status.status !== "completed" && status.status !== "failed") { + await new Promise((r) => setTimeout(r, 5000)); + status = await getVideoJobStatus({ adapter, jobId }); +} +// status.url is a data: URL (OpenRouter download URLs require the API key, +// so the adapter downloads server-side); status.usage?.cost is the real +// billed cost reported by the gateway. +``` + +```tsx +// Client: track the job with the useGenerateVideo hook +import { useGenerateVideo, fetchServerSentEvents } from "@tanstack/ai-react"; + +const { generate, result, videoStatus, isLoading } = useGenerateVideo({ + connection: fetchServerSentEvents("/api/generate/video"), +}); +// result?.url renders directly: