Skip to content

chat: lazy-load final tool input - #374

Merged
roblourens merged 6 commits into
mainfrom
roblou/agents/tool-input-streaming-optimization
Jul 31, 2026
Merged

chat: lazy-load final tool input#374
roblourens merged 6 commits into
mainfrom
roblou/agents/tool-input-streaming-optimization

Conversation

@roblourens

@roblourens roblourens commented Jul 31, 2026

Copy link
Copy Markdown
Member

Summary

  • make partial arguments and invocation-message updates independently optional on chat/toolCallDelta, so hosts may send either or both
  • let hosts choose inline strings or ContentRef values for final toolInput
  • keep edited input action-only: reducers replace inline input, while referenced input is updated by the host before echoing confirmation
  • regenerate schemas and all five clients, with cross-language reducer fixtures

Validation

  • npm run test (354 tests, 100% reducer coverage)
  • TypeScript client typecheck and tests (65 tests)
  • Rust formatting, Clippy, workspace tests, and doc tests
  • Swift package tests (115 tests)
  • Kotlin Gradle tests
  • Go tests
  • second npm run generate pass and git diff --check

(Written by Copilot)

Replace streamed raw tool parameters with invocation-message updates and retain the final input as a resource-readable ContentRef. Keep edited input action-only and require the host to update the referenced resource before echoing confirmation. (Written by Copilot)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@roblourens
roblourens requested a review from Copilot July 31, 2026 03:09
Apply rustfmt output required by the Rust CI job. (Written by Copilot)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Updates the chat tool-call protocol to avoid streaming raw tool parameters via deltas and instead expose the final tool input as a lazy-loaded ContentRef, while keeping streaming invocation-message updates available to subscribers.

Changes:

  • Replace chat/toolCallDelta parameter chunk streaming with invocation-message-only updates.
  • Change tool input from inline string to ContentRef across state/actions/schemas and regenerate all clients.
  • Update reducer fixtures and docs to reflect the new lazy-load + edited-input host-rewrite model.
Show a summary per file
File Description
types/test-cases/reducers/260-edited-tool-input-survives-reconfirmation.json Adds fixture ensuring tool input ref persists across re-confirmation after edited confirmation.
types/test-cases/reducers/255-toolcallcomplete-ignores-requiresresultconfirmation-when-cancelling-auth-required.json Updates fixtures to use ContentRef for tool input and formatting adjustments.
types/test-cases/reducers/254-toolcallcomplete-ignores-successful-result-from-auth-required.json Updates fixtures to use ContentRef for tool input and formatting adjustments.
types/test-cases/reducers/253-toolcallcomplete-cancels-auth-required-tool-call.json Updates fixtures to use ContentRef for tool input and formatting adjustments.
types/test-cases/reducers/252-session-inputneededremoved-removes-tool-authentication.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/251-session-inputneededset-appends-tool-authentication.json Updates fixture tool input to ContentRef and reformats nested JSON arrays/objects.
types/test-cases/reducers/250-turncomplete-force-cancels-auth-required-tool-call.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/249-toolcallauthresolved-preserves-fields-and-clears-input-needed.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/248-toolcallauthrequired-sets-input-needed-status-and-preserves-metadata.json Updates fixture tool input to ContentRef and formatting adjustments.
types/test-cases/reducers/247-toolcallauthresolved-noop-when-not-auth-required.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/243-toolcallauthrequired-full-lifecycle-running-authrequired-running-completed.json Updates fixture tool input to ContentRef and formatting adjustments.
types/test-cases/reducers/242-toolcallready-completes-risk-assessment.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/220-toolcall-actions-update-meta.json Removes delta content/partialInput expectations and updates tool input to ContentRef.
types/test-cases/reducers/163-toolcallstart-carries-mcp-contributor-through-lifecycle.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/158-toolcallconfirmed-approved-with-editedtoolinput-overrides-original.json Adjusts fixture semantics for host-handled edited input; tool input becomes ContentRef.
types/test-cases/reducers/111-toolcall-pending-confirmation-sets-input-needed-status.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/101-toolcalldelta-without-invocationmessage.json Updates delta fixture to invocation-message-only behavior.
types/test-cases/reducers/100-delta-targeting-toolcall-partid-is-no-op.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/099-endturn-force-cancels-running-tool-call.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/098-toolcallresultconfirmed-wrong-status-is-no-op.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/097-toolcallcomplete-wrong-status-is-no-op.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/096-toolcallconfirmed-wrong-status-is-no-op.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/095-toolcalldelta-wrong-status-is-no-op.json Updates delta fixture to invocation-message-only behavior and tool input to ContentRef.
types/test-cases/reducers/090-toolcalldelta-wrong-turnid-is-no-op.json Updates delta fixture to invocation-message-only behavior.
types/test-cases/reducers/086-toolcall-contentchanged-replaces-existing.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/085-toolcall-contentchanged-noop-non-running.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/084-toolcall-contentchanged-updates-running.json Updates fixture tool input to ContentRef.
types/test-cases/reducers/025-tool-call-actions-for-unknown-toolcallid-are-no-op.json Updates delta fixture to invocation-message-only behavior.
types/test-cases/reducers/019-tool-call-full-lifecycle-start-delta-ready-confirmed-complete.json Removes delta content streaming and uses ContentRef for ready/final input.
types/channels-chat/state.ts Switches tool input to ContentRef and removes partialInput from streaming state.
types/channels-chat/reducer.ts Updates reducer logic for invocation-message-only deltas and preserves tool input ref across confirmation.
types/channels-chat/actions.ts Changes delta action shape (remove content, require invocationMessage) and updates tool input type to ContentRef.
scripts/generate-swift.ts Updates generated SnapshotState doc string to include content state.
scripts/generate-rust.ts Updates generated SnapshotState doc string and adds ContentRef to generated action imports.
scripts/generate-markdown.ts Formatting-only whitespace adjustments in markdown generator.
scripts/generate-kotlin.ts Updates generated SnapshotState doc string to include content state.
scripts/generate-go.ts Updates generated SnapshotState doc string to include content state.
schema/state.schema.json Updates tool input schema to ContentRef and removes partialInput from streaming tool-call state.
schema/notifications.schema.json Mirrors state schema changes for notifications (tool input as ContentRef, remove partialInput).
schema/errors.schema.json Mirrors action/state schema changes for errors (delta requires invocationMessage, tool input ContentRef).
schema/commands.schema.json Mirrors action/state schema changes for commands (delta requires invocationMessage, tool input ContentRef).
schema/actions.schema.json Updates action schema for tool-call delta (remove content, require invocationMessage) and tool input to ContentRef.
docs/guide/state-model.md Documents lazy ContentRef tool input and invocation-message-only deltas; updates editable-input flow.
docs/guide/customizations.md Updates customizations guide sequence to reflect invocation-message deltas and tool input refs.
docs/guide/actions.md Updates tool-call delta description to reflect invocation-message-only semantics.
docs/.changes/20260730-tool-input-content-ref.json Adds changelog entry describing tool input ref and new delta semantics.
clients/typescript/test/hosts.test.ts Formatting-only whitespace adjustment.
clients/typescript/src/client/state-mirror.ts Formatting-only whitespace adjustment.
clients/typescript/src/client/hosts/state-mirror.ts Formatting-only whitespace adjustment.
clients/swift/AgentHostProtocol/Sources/AgentHostProtocol/ToolCallStateExtensions.swift Extends base-field extraction to include toolInput: ContentRef?.
clients/swift/AgentHostProtocol/Sources/AgentHostProtocol/Reducers.swift Updates Swift reducer for invocation-message-only deltas and ContentRef tool input.
clients/swift/AgentHostProtocol/Sources/AgentHostProtocol/Generated/State.generated.swift Regenerates Swift state types (remove partialInput, tool input as ContentRef).
clients/swift/AgentHostProtocol/Sources/AgentHostProtocol/Generated/Actions.generated.swift Regenerates Swift action types (delta invocationMessage required, tool input as ContentRef).
clients/swift/AHPApp/AHPApp/Views/ResponsePartView.swift Updates Swift UI to display tool input via ContentRef.uri and removes partialInput use.
clients/swift/AHPApp/AHPApp/Views/ChatView.swift Updates preview/sample tool calls to use ContentRef.
clients/rust/crates/ahp/src/reducers.rs Updates Rust reducer to remove partial input accumulation and propagate ContentRef tool input.
clients/rust/crates/ahp-types/src/state.rs Regenerates Rust state types (remove partial_input, tool input as ContentRef).
clients/rust/crates/ahp-types/src/actions.rs Regenerates Rust action types (delta invocation_message required, tool_input as ContentRef).
clients/kotlin/src/test/kotlin/com/microsoft/agenthostprotocol/FixtureDrivenReducerTest.kt Formatting-only whitespace adjustment.
clients/kotlin/src/main/kotlin/com/microsoft/agenthostprotocol/generated/State.generated.kt Regenerates Kotlin state types (remove partialInput, tool input as ContentRef).
clients/kotlin/src/main/kotlin/com/microsoft/agenthostprotocol/generated/Actions.generated.kt Regenerates Kotlin action types (delta invocationMessage required, tool input as ContentRef).
clients/kotlin/src/main/kotlin/com/microsoft/agenthostprotocol/Reducers.kt Updates Kotlin reducer to remove partial input accumulation and propagate ContentRef tool input.
clients/go/ahptypes/state.generated.go Regenerates Go state types (remove PartialInput, tool input as *ContentRef).
clients/go/ahptypes/actions.generated.go Regenerates Go action types (delta InvocationMessage required, tool input as *ContentRef).
clients/go/ahp/reducers.go Updates Go reducer to remove partial input accumulation and propagate *ContentRef tool input.

Review details

  • Files reviewed: 61/65 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread clients/go/ahp/reducers.go
Comment thread docs/guide/state-model.md Outdated
Avoid aliasing the Go action payload in reducer state and document the ContentRef nonce field. (Written by Copilot)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread types/channels-chat/actions.ts Outdated
/** Raw tool input */
toolInput?: string;
/** Reference to the final raw tool input, readable with `resourceRead` */
toolInput?: ContentRef;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts on allowing this to be a string | ContentRef and leaving it to the host to decide?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would a host decide one way or another?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like it would be the client that cares- based on whether it wants to render raw tool args, or whether it is very bandwidth-sensitive

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated this to string | ContentRef, with the host choosing the representation. I also restored optional partial argument chunks on chat/toolCallDelta; invocationMessage remains required. (Written by Copilot)

@connor4312

connor4312 commented Jul 31, 2026

Copy link
Copy Markdown
Member

@roblourens streaming tool input is used for MCP apps to progressive rendering (e.g. in Claude https://claude.com/blog/claude-builds-visuals at 0:30 in their video; FastMCP supports this too) so I'm not sure we should just remove the ability.

@roblourens

Copy link
Copy Markdown
Member Author

Ah, so for some types of tools it's unavoidable and we want the host to decide that? Makes sense

roblourens and others added 3 commits July 31, 2026 11:10
Let hosts choose between inline strings and ContentRef values for final tool input, and restore optional partial argument chunks on tool-call deltas. (Written by Copilot)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove unnecessary display-ready and raw qualifiers while retaining only comments that define wire behavior. (Written by Copilot)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Make invocationMessage optional so hosts may stream only parameter content, only progress text, or both. (Written by Copilot)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@roblourens
roblourens marked this pull request as ready for review July 31, 2026 19:54
@roblourens
roblourens enabled auto-merge (squash) July 31, 2026 19:54
@roblourens
roblourens merged commit 62422ee into main Jul 31, 2026
9 checks passed
@roblourens
roblourens deleted the roblou/agents/tool-input-streaming-optimization branch July 31, 2026 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants