docs(ai-chat): document the chat.agent tools option (#3791)

ericallam · web-flow · commit 4c4ed22e82af · 2026-06-02T09:59:42.000+01:00
## Summary Documents the new `tools` option on `chat.agent` (companion to #3790). Adds a dedicated [Tools](/ai-chat/tools) guide: the three places tools show up (config, `toStreamTextOptions`, `streamText`), why declaring them on the config matters for `toModelOutput` across turns, static vs per-turn tools, the typed `run()` payload, `InferChatUIMessageFromTools`, the relationship to skills, and the manual `convertToModelMessages` path for `customAgent` loops. Threads the option through the rest of the guide: the reference tables, a happy-path section on the backend page, the types page, and the HITL / skills / tool-result-auditing patterns. Corrects the sub-agents guide, where the `toModelOutput` compression was implied to work across turns but silently degraded from turn 2 without config tools. Also unstacks the three callouts that were piled under the `chat.agent()` header on the backend page, and adds a changelog entry.
diff --git a/docs/ai-chat/backend.mdx b/docs/ai-chat/backend.mdx
@@ -12,32 +12,6 @@ import RcBanner from "/snippets/ai-chat-rc-banner.mdx";
 
 The highest-level approach. Handles message accumulation, stop signals, turn lifecycle, and auto-piping automatically.
 
-<Tip>
-  To fix a **custom** `UIMessage` subtype or typed client data schema, use the [ChatBuilder](/ai-chat/types#chatbuilder) via `chat.withUIMessage<...>()` and/or `chat.withClientData({ schema })`. Builder-level hooks can also be chained before `.agent()`. See [Types](/ai-chat/types).
-</Tip>
-
-<Info>
-  Every `chat.agent` conversation is backed by a durable Session — `externalId` is your `chatId`, `type` is `"chat.agent"`, `taskIdentifier` is the agent's task ID. The session is the run manager: it owns the chat's runs, persists across run lifecycles, and orchestrates handoffs (idle continuation, `chat.requestUpgrade`). You rarely need to touch the session directly (`chat.stream`, `chat.messages`, `chat.stopSignal` wrap everything), but `payload.sessionId` is available if you want to reach in — e.g. `sessions.open(payload.sessionId)` to write from a sub-agent or from outside the turn loop.
-</Info>
-
-<Warning>
-  **Always spread `chat.toStreamTextOptions()` into every `streamText` call.** It wires up the `prepareStep` callback that drives [compaction](/ai-chat/compaction), [steering](/ai-chat/pending-messages), and [background injection](/ai-chat/background-injection) — features that silently no-op if the spread is missing. It also injects the system prompt set via `chat.prompt()`, the resolved model (when a registry is provided), and telemetry metadata.
-
-  Spread it **first** in the options object so any explicit overrides win:
-
-  ```ts
-  streamText({
-    ...chat.toStreamTextOptions(),     // or: chat.toStreamTextOptions({ registry, tools }) — see below
-    messages,
-    abortSignal: signal,
-    // any explicit overrides go here
-    stopWhen: stepCountIs(15),
-  });
-  ```
-
-  Examples in this doc keep the spread implicit for brevity, but you should include it in real code.
-</Warning>
-
 ### Simple: return a StreamTextResult
 
 Return the `streamText` result from `run` and it's automatically piped to the frontend:
@@ -51,7 +25,7 @@ export const simpleChat = chat.agent({
   id: "simple-chat",
   run: async ({ messages, signal }) => {
     return streamText({
-      ...chat.toStreamTextOptions(), // prepareStep, system, telemetry — see callout above
+      ...chat.toStreamTextOptions(), // prepareStep, system, telemetry (see note below)
       model: anthropic("claude-sonnet-4-5"),
       system: "You are a helpful assistant.",
       messages,
@@ -62,6 +36,10 @@ export const simpleChat = chat.agent({
 });
 ```
 
+<Warning>
+  **Always spread `chat.toStreamTextOptions()` first** (as above) so your explicit overrides win. It wires up the `prepareStep` callback behind [compaction](/ai-chat/compaction), [steering](/ai-chat/pending-messages), and [background injection](/ai-chat/background-injection), all of which silently no-op without it, and injects the system prompt from `chat.prompt()`, the resolved model (when you pass a `registry`), and telemetry metadata. Examples below keep the spread implicit for brevity, so include it in real code.
+</Warning>
+
 ### Using chat.pipe() for complex flows
 
 For complex agent flows where `streamText` is called deep inside your code, use `chat.pipe()`. It works from **anywhere inside a task** — even nested function calls.
@@ -173,6 +151,33 @@ await waitUntilComplete();
 
 For piping streams from subtasks to the parent chat (via `target: "root"`), see the [Sub-agents pattern](/ai-chat/patterns/sub-agents).
 
+### Backed by a Session
+
+Every `chat.agent` conversation is backed by a durable [Session](/ai-chat/sessions): `externalId` is your `chatId`, `type` is `"chat.agent"`, and `taskIdentifier` is the agent's task ID. The session is the run manager. It owns the chat's runs, persists across run lifecycles, and orchestrates handoffs (idle continuation, `chat.requestUpgrade`). You rarely touch it directly, since `chat.stream`, `chat.messages`, and `chat.stopSignal` wrap everything, but `payload.sessionId` is there when you need to reach in, e.g. `sessions.open(payload.sessionId)` to write from a sub-agent or from outside the turn loop.
+
+### Tools
+
+Declare your tools on the agent config, then read them back (typed) from the `run()` payload. Declaring them on the config, not just on `streamText`, is what lets the SDK re-apply each tool's `toModelOutput` when it re-converts history on later turns.
+
+```ts
+const tools = { searchDocs };
+
+export const myChat = chat.agent({
+  id: "my-chat",
+  tools,
+  run: async ({ messages, tools, signal }) =>
+    streamText({
+      ...chat.toStreamTextOptions({ tools }),
+      model: anthropic("claude-sonnet-4-5"),
+      messages,
+      abortSignal: signal,
+      stopWhen: stepCountIs(15),
+    }),
+});
+```
+
+See [Tools](/ai-chat/tools) for `toModelOutput` across turns, per-turn dynamic tools, the typed run payload, and how config tools relate to skills.
+
 ### Lifecycle hooks
 
 `chat.agent({ ... })` accepts hooks that fire in a fixed order around each turn, plus dedicated suspend/resume hooks. The full reference lives on its own page:
diff --git a/docs/ai-chat/changelog.mdx b/docs/ai-chat/changelog.mdx
@@ -4,6 +4,30 @@ sidebarTitle: "Changelog"
 description: "Pre-release updates for AI chat agents."
 ---
 
+<Update label="June 1, 2026" description="4.5.0-rc.4" tags={["SDK"]}>
+
+## `tools` option on `chat.agent`: `toModelOutput` survives across turns
+
+`chat.agent` now takes a `tools` option. Until now tools only went to `streamText` inside `run()`, which meant the SDK had no tools when it re-converted the persisted `UIMessage` history at the start of each turn. Any tool with a `toModelOutput` (raw image bytes turned into an image content part, or a sub-agent transcript compressed to a summary) had its transform applied on turn 1 and skipped from turn 2 onward, so the raw output got stringified back into the prompt.
+
+Declare your tools on the config and the SDK threads them into that conversion, so `toModelOutput` is re-applied every turn. The resolved set is handed back, typed, on the `run()` payload as `tools`, so you declare them once:
+
+```ts
+const tools = { searchDocs, renderChart };
+
+export const myChat = chat.agent({
+  tools,
+  run: async ({ messages, tools, signal }) =>
+    streamText({ ...chat.toStreamTextOptions({ tools }), messages, abortSignal: signal }),
+});
+```
+
+`tools` also accepts a per-turn function (`(event) => ToolSet`) for tools that depend on the user or a feature flag. Only `inputSchema` and `toModelOutput` are read during conversion, never `execute`. No behavior change for agents that don't declare `tools`.
+
+A new `InferChatUIMessageFromTools<typeof tools>` helper derives the chat `UIMessage` type (with typed tool parts) directly from a tool set. See the new [Tools](/ai-chat/tools) guide.
+
+</Update>
+
 <Update label="May 23, 2026" description="4.5.0-rc.2" tags={["SDK", "Webapp", "Bug fix"]}>
 
 ## HITL continuations — slim wire by default + field-level merge
diff --git a/docs/ai-chat/overview.mdx b/docs/ai-chat/overview.mdx
@@ -78,6 +78,9 @@ Three primitives, related but distinct:
   <Card title="Backend" icon="server" href="/ai-chat/backend">
     `chat.agent` options, lifecycle hooks, and the raw-task primitives.
   </Card>
+  <Card title="Tools" icon="wrench" href="/ai-chat/tools">
+    Declare tools so `toModelOutput` survives across turns, typed in `run()`.
+  </Card>
   <Card title="Patterns" icon="puzzle-piece" href="/ai-chat/patterns/sub-agents">
     HITL approvals, branching, sub-agents, OOM/crash recovery.
   </Card>
diff --git a/docs/ai-chat/patterns/human-in-the-loop.mdx b/docs/ai-chat/patterns/human-in-the-loop.mdx
@@ -69,18 +69,21 @@ const askUser = tool({
 
 export const myChat = chat.agent({
   id: "my-chat",
-  run: async ({ messages, signal }) => {
+  tools: { askUser },
+  run: async ({ messages, tools, signal }) => {
     return streamText({
       model: anthropic("claude-sonnet-4-5"),
       messages,
-      tools: { askUser },
+      tools,
       abortSignal: signal,
       stopWhen: stepCountIs(15),
     });
   },
 });
 ```
 
+Declaring `tools` on the config (and reading them back from the payload) is the recommended shape for any agent with tools. See [Tools](/ai-chat/tools).
+
 ## Frontend: render the question and collect the answer
 
 Two pieces on the client:
diff --git a/docs/ai-chat/patterns/skills.mdx b/docs/ai-chat/patterns/skills.mdx
@@ -185,6 +185,8 @@ return streamText({
 
 Your tools win on name conflicts. (Pick names that don't collide with `loadSkill` / `readFile` / `bash` to keep things predictable.)
 
+Also declare those same tools on the agent's [`tools`](/ai-chat/tools) config. `toStreamTextOptions` merges them with the skill tools for the model call, while the config option threads them into history re-conversion so any `toModelOutput` survives across turns. The auto-injected skill tools (`loadSkill` / `readFile` / `bash`) don't define `toModelOutput`, so they don't need to be on the config.
+
 ## Bundling
 
 Bundling is **built-in to the CLI** — there's no extension to import. When you run `trigger deploy` or `trigger dev`:
diff --git a/docs/ai-chat/patterns/sub-agents.mdx b/docs/ai-chat/patterns/sub-agents.mdx
@@ -205,6 +205,10 @@ toModelOutput: ({ output: message }) => {
 
 This is important for token efficiency: the sub-agent might use 100K tokens exploring and reasoning, but the parent LLM only consumes the summary.
 
+<Warning>
+  `toModelOutput` only runs when the SDK has your tools at conversion time. On a multi-turn parent, the SDK re-converts the persisted history at the start of each turn, so you must declare the sub-agent tool on the agent config (`chat.agent({ tools })`) for the compression to survive. Without it, the summary holds on turn 1 but turn 2 onward re-ingests the full sub-agent output. In a `chat.customAgent` loop you own the conversion, so pass the tools to `convertToModelMessages(uiMessages, { tools })` yourself. See [Tools: toModelOutput across turns](/ai-chat/tools#tomodeloutput-across-turns).
+</Warning>
+
 ## ChatStream.messages()
 
 The `messages()` method on `ChatStream` wraps the AI SDK's `readUIMessageStream`. It reads the raw `UIMessageChunk` stream and yields complete `UIMessage` snapshots — each containing all parts received so far.
@@ -237,21 +241,24 @@ Sub-agent tools work inside both `chat.agent()` (managed) and `chat.customAgent(
 
 ```ts
 // Managed agent with sub-agent tool
+const tools = { research: researchTool };
+
 export const myAgent = chat.agent({
   id: "orchestrator",
-  run: async ({ messages, stopSignal }) => {
+  tools, // declare here so toModelOutput survives across turns
+  run: async ({ messages, tools, stopSignal }) => {
     return streamText({
       model: anthropic("claude-sonnet-4-6"),
       messages,
-      tools: { research: researchTool },
+      tools,
       abortSignal: stopSignal,
       stopWhen: stepCountIs(15),
     });
   },
 });
 ```
 
-For `chat.customAgent()`, define the tool and sub-agent Map inside the `run` closure so they survive across turns.
+For `chat.customAgent()`, define the tool and sub-agent Map inside the `run` closure so they survive across turns. Since you own the turn loop there, convert history with your tools in scope so `toModelOutput` is re-applied each turn: `convertToModelMessages(uiMessages, { tools })`. See [Tools: manual turn loops](/ai-chat/tools#manual-turn-loops-chatcustomagent).
 
 ## Streaming progress from a subtask to the parent chat
 
diff --git a/docs/ai-chat/patterns/tool-result-auditing.mdx b/docs/ai-chat/patterns/tool-result-auditing.mdx
@@ -8,7 +8,7 @@ import RcBanner from "/snippets/ai-chat-rc-banner.mdx";
 
 <RcBanner />
 
-When a chat agent uses tools (especially [human-in-the-loop](/ai-chat/patterns/human-in-the-loop) tools that wait on `addToolOutput` from the frontend), you often need to fire side effects exactly once per resolved tool call:
+When a chat agent uses [tools](/ai-chat/tools) (especially [human-in-the-loop](/ai-chat/patterns/human-in-the-loop) tools that wait on `addToolOutput` from the frontend), you often need to fire side effects exactly once per resolved tool call:
 
 - **Audit logs** — record every tool result for compliance.
 - **Billing** — charge per tool invocation.
diff --git a/docs/ai-chat/quick-start.mdx b/docs/ai-chat/quick-start.mdx
@@ -151,6 +151,7 @@ These steps assume you already have a Trigger.dev project with the SDK installed
 ## Next steps
 
 - [Backend](/ai-chat/backend) — Lifecycle hooks, persistence, session iterator, raw task primitives
+- [Tools](/ai-chat/tools): Declare tools so `toModelOutput` survives across turns, typed in `run()`
 - [Frontend](/ai-chat/frontend) — Session management, client data, reconnection
 - [Types](/ai-chat/types) — `chat.withUIMessage`, `InferChatUIMessage`, and related typing
 - [`chat.local`](/ai-chat/chat-local) — Per-run typed state across hooks, run, tools, subtasks
diff --git a/docs/ai-chat/reference.mdx b/docs/ai-chat/reference.mdx
@@ -45,6 +45,7 @@ Options for `chat.agent()`.
 | `compaction`                  | `ChatAgentCompactionOptions`                                | —                              | Automatic context compaction. See [Compaction](/ai-chat/compaction)                                 |
 | `pendingMessages`             | `PendingMessagesOptions`                                    | —                              | Mid-execution message injection. See [Pending Messages](/ai-chat/pending-messages)                  |
 | `prepareMessages`             | `(event: PrepareMessagesEvent) => ModelMessage[]`           | —                              | Transform model messages before use (cache breaks, context injection, etc.)                         |
+| `tools`                       | `ToolSet \| ((event: ResolveToolsEvent) => ToolSet \| Promise<ToolSet>)` | —                 | Tools for this agent. Threads each tool's `toModelOutput` through cross-turn history re-conversion, and hands the resolved set back on the run payload. Static set or per-turn function. See [Tools](/ai-chat/tools). |
 | `maxTurns`                    | `number`                                                    | `100`                          | Max conversational turns per run                                                                    |
 | `turnTimeout`                 | `string`                                                    | `"1h"`                         | How long to wait for next message                                                                   |
 | `idleTimeoutInSeconds`        | `number`                                                    | `30`                           | Seconds to stay idle before suspending                                                              |
@@ -87,6 +88,7 @@ The payload passed to the `run` function.
 | -------------- | ------------------------------------------ | -------------------------------------------------------------------- |
 | `ctx`          | `TaskRunContext`                           | Full task run context — same as `task` `run`’s `{ ctx }`             |
 | `messages`     | `ModelMessage[]`                           | Model-ready messages — pass directly to `streamText`                 |
+| `tools`        | `ToolSet`                                  | Resolved tools declared on the agent config (empty object when none). Pass straight to `streamText`. See [Tools](/ai-chat/tools). |
 | `chatId`       | `string`                                   | Your conversation ID (the session's `externalId`)                    |
 | `sessionId`    | `string`                                   | Friendly ID of the backing Session (`session_*`). Use with `sessions.open()` for advanced cases. Always set — every chat.agent run is bound to a Session. |
 | `trigger`      | `"submit-message" \| "regenerate-message"` | What triggered the request                                           |
@@ -191,6 +193,17 @@ Passed to the `onValidateMessages` callback.
 | `turn`    | `number`                                                        | Turn number (0-indexed)                  |
 | `trigger` | `"submit-message" \| "regenerate-message" \| "preload" \| "close"` | The trigger type for this turn        |
 
+## ResolveToolsEvent
+
+Passed to the `tools` function form on `chat.agent`, once per turn, to resolve the tool set for that turn. See [Tools](/ai-chat/tools#static-or-per-turn-tools).
+
+| Field          | Type                        | Description                                       |
+| -------------- | --------------------------- | ------------------------------------------------- |
+| `chatId`       | `string`                    | Chat session ID                                   |
+| `turn`         | `number`                    | Turn number (0-indexed)                           |
+| `continuation` | `boolean`                   | Whether this run is continuing an existing chat   |
+| `clientData`   | Typed by `clientDataSchema` | Custom data from the frontend                     |
+
 ## HydrateMessagesEvent
 
 Passed to the `hydrateMessages` callback. See [hydrateMessages](/ai-chat/lifecycle-hooks#hydratemessages).
@@ -532,6 +545,19 @@ type Msg = InferChatUIMessage<typeof myChat>;
 
 Use with `useChat<Msg>({ transport })` when using [`chat.withUIMessage`](/ai-chat/types). For agents defined with plain `chat.agent()` (no custom generic), this resolves to the base `UIMessage`.
 
+## `InferChatUIMessageFromTools`
+
+Type helper: derives the chat `UIMessage` type (with typed `tool-${name}` parts) directly from a tool set. Shorthand for `UIMessage<unknown, UIDataTypes, InferUITools<typeof tools>>`.
+
+```ts
+import type { InferChatUIMessageFromTools } from "@trigger.dev/sdk/ai";
+
+const tools = { search, readFile };
+type ChatUiMessage = InferChatUIMessageFromTools<typeof tools>;
+```
+
+Pin it on the agent with [`chat.withUIMessage<ChatUiMessage>()`](/ai-chat/types) and reuse it on the client. See [Tools](/ai-chat/tools#typing-messages-from-your-tools).
+
 ## AI helpers (`ai` from `@trigger.dev/sdk/ai`)
 
 | Export                                                                              | Status         | Description                                                                                                                                                                                |
diff --git a/docs/ai-chat/tools.mdx b/docs/ai-chat/tools.mdx
diff --git a/docs/ai-chat/types.mdx b/docs/ai-chat/types.mdx
diff --git a/docs/docs.json b/docs/docs.json
