feat(a2ui-middleware): streaming surfaces, dynamic schema, and action handlers#1321
Open
ataibarkai wants to merge 13 commits intomainfrom
Open
feat(a2ui-middleware): streaming surfaces, dynamic schema, and action handlers#1321ataibarkai wants to merge 13 commits intomainfrom
ataibarkai wants to merge 13 commits intomainfrom
Conversation
…2UIOperations
Tools can now return {"a2ui_operations": [...]} for explicit detection.
Legacy bare-array and single-object formats still work.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shared constant for the container key used by copilotkit.a2ui.render() (Python) and A2UIMessageRenderer (React). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add metadata field to AG-UI ToolSchema - Add A2UIToolSchema type and schemas config to A2UIMiddlewareConfig - On TOOL_CALL_START for registered tools, emit surfaceUpdate + beginRendering - On TOOL_CALL_ARGS, partial-parse and emit dataModelUpdate progressively - Add toA2UIContents/toA2UITypedValue helpers for JS→A2UI type conversion Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…yParseA2UIOperations
Only the explicit { a2ui_operations: [...] } container is now recognized.
Bare arrays and single operation objects are no longer auto-detected.
Tests updated accordingly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Config is now an explicit array of { toolName, surface } objects
instead of a Record<string, schema>. Clearer developer experience.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
A2UISurfaceConfig now accepts actionHandlers — same interface as the tool-declared a2ui_action_handlers. Streaming surfaces pass actionHandlers through in both schema emission (TOOL_CALL_START) and progressive data updates. tryParseA2UIOperations now returns A2UIParseResult with optional actionHandlers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…oolCallId
messageId now includes toolCallId: "a2ui-surface-{surfaceId}-{toolCallId}"
so each tool call creates a distinct activity message. Prevents second
invocations of the same tool from overwriting earlier surfaces in chat.
Updated all four paths: createA2UIActivityEvents, emitStreamingData,
auto-detected TOOL_CALL_RESULT, and processSendA2UIToolCall.
2 new tests: distinct messageIds for send_a2ui and auto-detected tools.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace hardcoded "operations" key with A2UI_OPERATIONS_KEY constant in emitStreamingData and createA2UIActivityEvents. Updates tests to match the new container key format. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Unified streaming pipeline for both fixed-schema and dynamic render_a2ui tool calls. The middleware extracts schema from streaming args, then progressively parses data items using the same extractCompleteItems + emitStreamingData pipeline as fixed-schema surfaces. - Track render_a2ui tool calls alongside fixed-schema streaming surfaces - extractCompleteItems: add bracket depth tracking to stop at array boundary - extractCompleteItemsWithStatus: report arrayClosed for schema timing - extractStringField: extract string values from partial JSON - Auto-inject beginRendering for early surface display - Deferred schema emission until components array fully closed - groupBySurface helper for per-surface snapshot emission Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract actionHandlers from render_a2ui streaming args and include them in ACTIVITY_SNAPSHOT content. Re-emit snapshot when handlers arrive after initial data emission. Support dynamic actionHandlers fallback in emitStreamingData alongside fixed-schema handlers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Skip extractCompleteItems/extractStringField/JSON.parse calls when the TOOL_CALL_ARGS delta contains no structural JSON characters (} or ]). This eliminates ~90% of parsing work during streaming since most deltas are string content, not object/array boundaries. - Check delta for closing braces/brackets before extraction calls - Guard JSON.parse for actionHandlers with string-end check - Apply same optimization to send_a2ui_json_to_client path - ~17x reduction in parsing overhead per stream Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Open
6 tasks
Python Preview PackagesVersion
Install with uvAdd the TestPyPI index to your [[tool.uv.index]]
name = "testpypi"
url = "https://test.pypi.org/simple/"
explicit = trueThen install the packages you need: # Core SDK
uv add 'ag-ui-protocol==0.0.0.dev1773817668' --index testpypi
# Integrations (each already depends on the matching ag-ui-protocol preview)
uv add 'ag-ui-langgraph==0.0.0.dev1773817668' --index testpypi
uv add 'ag-ui-crewai==0.0.0.dev1773817668' --index testpypi
# NOTE: ag-ui-agent-spec depends on pyagentspec (git-only, not on PyPI).
# You will need to install pyagentspec separately from its git repo.
uv add 'ag-ui-agent-spec==0.0.0.dev1773817668' --index testpypi
uv add 'ag_ui_adk==0.0.0.dev1773817668' --index testpypi
uv add 'ag_ui_strands==0.0.0.dev1773817668' --index testpypiInstall with pippip install \
--index-url https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple/ \
ag-ui-protocol==0.0.0.dev1773817668
Commit: 7ebcf16 |
@ag-ui/a2a-middleware
@ag-ui/a2ui-middleware
@ag-ui/mcp-apps-middleware
@ag-ui/middleware-starter
@ag-ui/a2a
@ag-ui/adk
@ag-ui/ag2
@ag-ui/agno
@ag-ui/aws-strands
@ag-ui/claude-agent-sdk
@ag-ui/crewai
@ag-ui/langchain
@ag-ui/langgraph
@ag-ui/langroid
@ag-ui/llamaindex
@ag-ui/mastra
@ag-ui/pydantic-ai
@ag-ui/server-starter
@ag-ui/server-starter-all-features
@ag-ui/vercel-ai-sdk
create-ag-ui-app
@ag-ui/client
@ag-ui/core
@ag-ui/encoder
@ag-ui/proto
commit: |
- Remove old a2ui_chat (used outdated createA2UIMessageRenderer API) - Add a2ui_fixed_schema: pre-built flight card schema, no streaming - Add a2ui_fixed_schema_streaming: progressive card rendering via streamingSurfaces - Add a2ui_dynamic_schema: secondary LLM generates UI schema from conversation - Add a2ui_advanced: dynamic schema + useA2UIActionHandler for optimistic button responses - Add local-install script (.pnpmfile.cjs + local-install.sh) for linking CopilotKit locally - Configure CopilotRuntime with a2ui + streamingSurfaces for streaming flight search - Auto-detect cross-repo symlinks in next.config.ts for outputFileTracingRoot Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Now that the CopilotKit infrastructure fix (useRenderTool overrides built-in) is in place, the custom violet progress renderer with live component/item counters works correctly without the built-in duplicate. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
mme
requested changes
Mar 18, 2026
| * been received). Callers that need the complete array (e.g., schema | ||
| * components) should wait for `arrayClosed === true`. | ||
| */ | ||
| export function extractCompleteItemsWithStatus( |
Contributor
There was a problem hiding this comment.
This relies too much on AI generated code for something we should use a library for. Use https://github.com/dscape/clarinet or an equivalent instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends the A2UI middleware with three rendering modes and interactive action handlers. Companion to CopilotKit#3453.
Architecture
1. Fixed Schema (non-streaming)
The middleware intercepts
TOOL_CALL_RESULTevents and checks fora2ui_operationsin the JSON payload. When found, it maps the operations array toACTIVITY_SNAPSHOTevents that the frontend A2UI renderer consumes. The agent tool returns the full surface (schema + data + beginRendering) in one shot viaa2ui.render().2. Fixed Schema (streaming)
streamingSurfacesconfig in the CopilotRuntime declares tool-to-schema mappings. When the middleware sees aTOOL_CALL_STARTfor a registered tool, it immediately emits the schema (surfaceUpdate+beginRendering). AsTOOL_CALL_ARGSstream in, partial JSON parsing extracts complete items from thedataKeyarray and emits progressivedataModelUpdates. Cards appear one-by-one.3. Dynamic Schema
The
render_a2uitool name is auto-detected. The secondary LLM generates both schema and data as tool call args. The middleware extractscomponentsfirst (waits for the array to close), then streamsitemsprogressively — same partial-parse approach as streaming surfaces.4. Action Handlers
Pre-declared optimistic UI responses attached to surfaces. When a button is clicked, the matching handler's operations replace the surface instantly (no round-trip). Supports exact name match and
"*"catch-all. Available on both tool results (a2ui_action_handlerskey) and streaming surface config (surface.actionHandlers).Dojo Items
a2ui_fixed_schema,a2ui_fixed_schema_streaming,a2ui_dynamic_schema— LangGraph Python (FastAPI) agents + frontend pagesKey Changes
middlewares/a2ui-middleware/src/— streaming surfaces, dynamic schema detection, action handler pass-through, partial JSON parsing, surface isolation by toolCallIdintegrations/langgraph/python/examples/agents/a2ui_*— three agent implementationsapps/dojo/src/— menu entries, agent configs, frontend pages for all three A2UI featuresWhat's Missing (for the person taking this over)
local-installscript was prototyped but reverted due to version compatibility issues between local CopilotKit and the dojo's runtime. The backup branchatai/0313/a2ui-etc-dojo-backuppreserves that workNote to Reviewer
Feel free to modify any and all implementations and developer experiences. The implementations here are a starting point. What matters is supporting the "real world hello world" use cases — a developer should be able to go from zero to a working A2UI surface (fixed, streaming, or dynamic) by following the patterns in the dojo and docs.
Test plan
pnpm test)🤖 Generated with Claude Code