feat(deepgram): add STT/TTS with_cloudflare() for Cloudflare AI Gateway by mcdgavin · Pull Request #6133 · livekit/agents

mcdgavin · 2026-06-17T03:44:50Z

Summary

Adds Cloudflare AI Gateway support to the Deepgram plugin — streaming STT (Nova-3 / Flux)
and TTS (Aura) — via deepgram.STT.with_cloudflare(...) and deepgram.TTS.with_cloudflare(...).

Addresses the STT/TTS portion of #3163. (The LLM portion shipped separately in #6131 as
openai.LLM.with_cloudflare.)

from livekit.plugins import deepgram

# account_id from CLOUDFLARE_ACCOUNT_ID, token from CLOUDFLARE_AI_GATEWAY_TOKEN
stt = deepgram.STT.with_cloudflare(model="@cf/deepgram/nova-3")
tts = deepgram.TTS.with_cloudflare(model="@cf/deepgram/aura-1")

Design notes

Rides on the Deepgram plugin. Cloudflare's AI Gateway proxies Deepgram's streaming
WebSocket protocol verbatim (channel.alternatives[0].transcript + is_final for STT;
Speak/Flush/Flushed for TTS), so these reuse the existing Deepgram STT/TTS
streaming implementations rather than duplicating them. This mirrors the LLM approach, where
Cloudflare-LLM rode on the OpenAI plugin because it's OpenAI-compatible — each Cloudflare
modality lives on the plugin whose protocol it matches.
Backward-compatible auth seam. The connection auth headers (previously hardcoded
Authorization: Token <key>) become an overridable _connect_headers, defaulting to the
exact prior value. Existing Deepgram usage is unchanged; with_cloudflare injects
cf-aig-authorization (the raw Cloudflare token — no Deepgram key required).

Testing

tests/test_deepgram_with_cloudflare.py — 12 hermetic --unit tests covering URL building,
CLOUDFLARE_ACCOUNT_ID env fallback, base_url override, the cf-aig-authorization header,
default models, streaming capability, and the ValueError paths.
A regression check confirms default (non-Cloudflare) Deepgram STT/TTS auth is byte-identical.
make check passes (ruff format + lint, strict mypy).

…ivekit#3163)

devin-ai-integration

Devin Review found 0 potential issues.

chenghao-mou · 2026-06-17T10:47:05Z

+    @staticmethod
+    def with_cloudflare(
+        *,
+        model: str = "@cf/deepgram/nova-3",


is it possible to construct "@cf/deepgram" part internally? Then we can reuse the same DeepgramModels | str type

Thank you for the feedback @chenghao-mou, this is done.

with_cloudflare now takes a bare Deepgram model name typed DeepgramModels | str (default nova-3 for STT, aura-1 for TTS) and prepends @cf/deepgram/ internally.
A value already prefixed with @cf/ is passed through unchanged, so the full form still works.

chenghao-mou · 2026-06-17T10:53:28Z

        numerals: bool = False,
        mip_opt_out: bool = False,
        vad_events: bool = True,
+        _connect_headers: NotGivenOr[dict[str, str]] = NOT_GIVEN,


We can just call it extra_headers: NotGivenOr[dict[str, str]] = NOT_GIVEN. We use similar parameters in other plugins as well, it could also be useful for self-hosted DG instances too.

Switched _connect_headers to a public extra_headers, additive, matching the other plugins and supporting self hosting.

One note, a purely additive extra_headers merges on top of the default Authorization: Token <key>, so the Cloudflare path would send both Authorization and cf-aig-authorization which requires a Deepgram key it doesn't need and risks the gateway rejecting the stray header. To keep it additive without that, the default Authorization: Token is now built only when a Deepgram key is present, and at least one auth source (key or extra_headers) is required:

headers = ({"Authorization": f"Token {key}"} if key else {}) | extra_headers

Behavior:

Normal Deepgram (key set): Authorization: Token + extras — unchanged.

Self-hosted DG with custom auth (no key): just extra_headers.

Cloudflare via with_cloudflare (no key): just cf-aig-authorization.

chenghao-mou · 2026-06-17T10:56:59Z

+            sample_rate=sample_rate,
+            base_url=base_url,
+            http_session=http_session,
+            _connect_headers={"cf-aig-authorization": cf_aig_token},


double checking here: the code in their doc says "cf-aig-authorization": "Bearer {CF_AIG_TOKEN}": https://developers.cloudflare.com/ai-gateway/usage/providers/deepgram/

I double-checked against the realtime WebSockets page. The Bearer form is documented on the /deepgram provider-proxy endpoint, but this plugin targets the /workers-ai endpoint, and its Deepgram (Workers AI) examples (both @cf/deepgram/nova-3 and @cf/deepgram/aura-1) use the raw cf-aig-authorization token:

"wss://gateway.ai.cloudflare.com/v1/<account_id>/<gateway>/workers-ai?model=@cf/deepgram/nova-3..." headers: { "cf-aig-authorization": process.env.CLOUDFLARE_API_KEY }

Since it flows through extra_headers, it's still overridable either way.

- public extra_headers (additive, merged over default Token auth; sole auth when no key) replacing the private _connect_headers seam - with_cloudflare accepts bare model names (nova-3/aura-1), prefixes @cf/deepgram/ - cf-aig-authorization sent as 'Bearer <token>' per Cloudflare's Deepgram docs

devin-ai-integration

Devin Review found 0 new potential issues.

- public extra_headers (additive, merged over the default Token auth; sole auth when no key is set) replacing the private _connect_headers seam - with_cloudflare accepts bare model names (nova-3 / aura-1) and prepends @cf/deepgram/ internally - cf-aig-authorization sent as the raw token, matching the /workers-ai endpoint examples (the Bearer form documented on Cloudflare's /deepgram provider proxy is a different endpoint)

…on for CF models - with_cloudflare passes api_key="" so a DEEPGRAM_API_KEY in the env is no longer picked up and sent to the gateway as a stray Authorization: Token header - TTS only falls back to the env var when api_key is None (explicit "" means no key) - _validate_model/_validate_keyterm strip the @cf/deepgram/ routing prefix so keyterm is correctly allowed for @cf/deepgram/nova-3

_validate_model dropped the routing prefix when downgrading an en-only model to nova-2-general for a non-English language, yielding an invalid gateway model.

devin-ai-integration

Devin Review found 1 new potential issue.

devin-ai-integration · 2026-06-17T15:51:39Z

🚩 stt_v2.py still uses the old _api_key pattern and was not updated

The file livekit-plugins/livekit-plugins-deepgram/livekit/plugins/deepgram/stt_v2.py still uses the old self._api_key / Authorization: Token {self._api_key} pattern (lines 119, 190, 288, 487). It was not modified in this PR. If Cloudflare AI Gateway support is expected to work with stt_v2 as well, it would need parallel changes. This may be intentional if stt_v2 is legacy/deprecated.

Was this helpful? React with 👍 or 👎 to provide feedback.

Intentional. STTv2 is the Deepgram v2 / Flux class (different protocol, /v2/listen). This PR scopes Cloudflare support to the stable STT (nova-3) and TTS (Aura).

A STTv2.with_cloudflare for Flux is a sensible follow-up rather than part of this change.

devin-ai-integration

Devin Review found 0 new potential issues.

mcdgavin added 5 commits June 16, 2026 22:58

refactor(deepgram): make STT connection auth headers overridable

94e749b

feat(deepgram): add STT.with_cloudflare() for Cloudflare AI Gateway (l…

60c5ef8

…ivekit#3163)

refactor(deepgram): make TTS connection auth headers overridable

dc11d89

feat(deepgram): add TTS.with_cloudflare() for Cloudflare AI Gateway (l…

6305f0e

…ivekit#3163)

refactor(deepgram): drop api_key fields made unused by the auth seam

799be2b

mcdgavin requested a review from a team as a code owner June 17, 2026 03:44

mcdgavin mentioned this pull request Jun 17, 2026

Request: Cloudflare TTS, LLM, STT plugins #3163

Open

devin-ai-integration Bot reviewed Jun 17, 2026

View reviewed changes

chenghao-mou reviewed Jun 17, 2026

View reviewed changes

devin-ai-integration Bot reviewed Jun 17, 2026

View reviewed changes

mcdgavin force-pushed the feat/deepgram-with-cloudflare branch from 55c1545 to 21f1d68 Compare June 17, 2026 14:23

This comment was marked as resolved.

Sign in to view

fix(deepgram): preserve @cf/deepgram/ prefix in en-only model fallback

7014597

_validate_model dropped the routing prefix when downgrading an en-only model to nova-2-general for a non-English language, yielding an invalid gateway model.

devin-ai-integration Bot reviewed Jun 17, 2026

View reviewed changes

test(deepgram): use a v1 model in the prefix test (flux is v2/STTv2)

8cf03bd

devin-ai-integration Bot reviewed Jun 17, 2026

View reviewed changes

Conversation

mcdgavin commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Design notes

Testing

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

chenghao-mou Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

mcdgavin Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chenghao-mou Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

mcdgavin Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chenghao-mou Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

mcdgavin Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

mcdgavin Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mcdgavin commented Jun 17, 2026 •

edited

Loading

mcdgavin Jun 17, 2026 •

edited

Loading

mcdgavin Jun 17, 2026 •

edited

Loading

mcdgavin Jun 17, 2026 •

edited

Loading

mcdgavin Jun 17, 2026 •

edited

Loading