Request: Cloudflare TTS, LLM, STT plugins

[leemuljadi]

Hi would like to request plugins for Cloudflare TTS, LLM, STT

[davidzhao]

please consider contributing one! it's pretty straight forward to build plugins

[beingPro007]

@davidzhao, currently, Cloudflare won't provide STT over streaming; instead, they are doing batch processing. Is there any way to do it, then suggest to me that I am going to build it?.

[davidzhao]

sure thing, please go for it. OpenAI STT is also non-streaming, you can use it as a reference implementation

[leemuljadi]

@davidzhao cool, how about for the Cloudflare meloTTS as it's using batch process as well? any example implementation of batch processing mp3?
Will the Cloudflare stack be a concern for latency since they are not providing streaming for STT and TTS?
And will it be possible to deploy Livekit agents in Clouldflare?

[leemuljadi]

@beingPro007 thanks for collaborating!

[mcdgavin]

Opened #6133 covering the STT and TTS parts of this request, and the LLM part shipped separately in #6131. Could a maintainer reopen this to track them? 🙏

What's changed since this issue was filed:

• Streaming is now available. Earlier discussion here assumed Cloudflare was batch-only request/response. That's no longer true: Cloudflare now hosts Deepgram partner models — Nova-3 / Flux (STT) and Aura (TTS) — exposed over a real-time WebSocket through the AI Gateway, allowing for true streaming plugins rather than batch-wrapped.
• The wire protocol is Deepgram's. The gateway proxies Deepgram's streaming protocol verbatim, so feat(deepgram): add STT/TTS with_cloudflare() for Cloudflare AI Gateway #6133 adds deepgram.STT.with_cloudflare(...) / deepgram.TTS.with_cloudflare(...) that reuse the existing Deepgram streaming implementation — auth via the cf-aig-authorization header, no Deepgram key required.

from livekit.plugins import deepgram, openai

stt = deepgram.STT.with_cloudflare(model="@cf/deepgram/nova-3")
tts = deepgram.TTS.with_cloudflare(model="@cf/deepgram/aura-1")
llm = openai.LLM.with_cloudflare(model="openai/gpt-4o")  # #6131