feat(server-utils): Capture usage and output on streamed Vercel AI v7 spans#22000
Draft
logaretm wants to merge 5 commits into
Draft
feat(server-utils): Capture usage and output on streamed Vercel AI v7 spans#22000logaretm wants to merge 5 commits into
logaretm wants to merge 5 commits into
Conversation
Contributor
size-limit report 📦
|
mydea
reviewed
Jul 7, 2026
| // channel already resolved the model call. The native-channel (v7) tap and the v6 OTel | ||
| // spans both carry them; the v6 orchestrion adapter rebuilds spans from v6's synchronous | ||
| // result shapes, which never include the drained stream, so scope these to the rest. | ||
| if (!(version === '6' && isOrchestrionEnabled())) { |
Member
There was a problem hiding this comment.
can we fix this for v6 orchestrion too? 🤔
Member
Author
There was a problem hiding this comment.
I wasn't sure about the orchistrion payload so never tried, lmk see if I can
Member
Author
mydea
reviewed
Jul 7, 2026
| // See the Node-18 note on `scenario-stream-text.mjs` above. The v6 orchestrion adapter rebuilds | ||
| // spans from v6's synchronous result shapes (no drained stream), so this stream-enrichment test | ||
| // runs against the native-channel (v7) and OTel (v6) paths only. | ||
| test.skipIf((version === '7' && nodeVersion === 18) || (version === '6' && isOrchestrionEnabled()))( |
Member
Author
There was a problem hiding this comment.
Cool, I will wait for it to be merged then!
mydea
reviewed
Jul 7, 2026
| // See the Node-18 note on `scenario-stream-text.mjs` above. The v6 orchestrion adapter rebuilds | ||
| // spans from v6's synchronous result shapes (no drained stream), so this stream-enrichment test | ||
| // runs against the native-channel (v7) and OTel (v6) paths only. | ||
| test.skipIf((version === '7' && nodeVersion === 18) || (version === '6' && isOrchestrionEnabled()))( |
… spans
The `ai:telemetry` channel resolves a `streamText` model call as soon as it
hands back the unconsumed stream, so usage, finish reason and output text
(which only arrive as the stream drains) never reached the gen_ai spans.
Tap the streamed model call's ReadableStream: forward every chunk to the SDK
untouched, aggregate the final usage/finish/output, then enrich and end the
model-call span out of band via the tracing-channel `deferSpanEnd` hook. Sum
the usage across the operation's model calls and propagate it, plus the output
messages, onto the parent invoke_agent span, whose own channel result is always
undefined for a stream.
This is v7 (native channel) only. Structured-output streaming rides the same
path since `streamObject` is deprecated in v7 in favour of
`streamText({ experimental_output })`.
…-utils Move the state-free type-coercion helpers (asString, asNumber, sum, isRecord, safeStringify) into a shared module and drop the duplicated copies in the orchestrion subscriber and the inline checks in the stream tap.
…til.ts Fold the streamed model-call tap and the pure helpers into a single `vercel-ai/util.ts`. Core's `isPlainObject`/`isString` aren't drop-ins (the former rejects tag-bearing objects like `ReadableStream`, the latter is a guard not a coercion) and core has no single-value safe stringify, so the loose local helpers stay.
…apter Reuse the stream tap in the orchestrion `doStream` patch so v6 (orchestrion/CJS) streamed model-call spans get usage, finish reason and output, matching v7 and the v6 OTel path. The parent invoke_agent span can't be enriched here: v6 `streamText` ends its operation span synchronously, before the stream drains, so that aggregate stays unset in orchestrion (tests gate it on `enrichesInvokeAgentForStream`).
Defer the orchestrion streamText operation span (via the deferSpanEnd hook the binding already supports) and await the StreamTextResult's completion promises (totalUsage/text/finishReason/toolCalls, resolved on drain) to enrich the invoke_agent span. This closes the earlier gap where that span ended synchronously before the stream drained, so v6 orchestrion now matches v7 and the v6 OTel path for streamed usage and output.
logaretm
force-pushed
the
awad/vercelai-channel-stream-support
branch
from
fc1b724 to
8dae3f0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Streamed
streamTextcalls weren't getting usage tokens, finish reason, or output text on their spans. Theai:telemetrychannel resolves the model call as soon as it hands back the stream, so that data only appears later as the stream drains and never reached us.This taps the streamed model call's
ReadableStream, forwards every chunk to the SDK untouched, and aggregates the final usage/finish/output so the span can be enriched and closed once the stream settles. Usage is summed across the operation's model calls and, along with the output, propagated onto the parentinvoke_agentspan, whose own channel result is always undefined for a stream.This is v7 (native channel) only. Structured output rides the same path since
streamObjectis deprecated in v7 in favor ofstreamText({ experimental_output }).