feat(openai): add LLM.with_cloudflare() for Cloudflare AI Gateway#6131
Open
mcdgavin wants to merge 1 commit into
Open
feat(openai): add LLM.with_cloudflare() for Cloudflare AI Gateway#6131mcdgavin wants to merge 1 commit into
mcdgavin wants to merge 1 commit into
Conversation
Adds a with_cloudflare() convenience constructor that routes LLM traffic through the Cloudflare AI Gateway unified OpenAI-compatible endpoint, mirroring with_openrouter / with_azure. - Builds the gateway URL from account_id (env CLOUDFLARE_ACCOUNT_ID) and gateway_id (default "default"), with a base_url override, mirroring with_azure. - Dual auth: cf_aig_token -> cf-aig-authorization, api_key -> Authorization (BYOK); requires at least one. Placeholder key for gateway-stored-keys mode (with_ollama precedent). - gateway_options (CloudflareGatewayOptions TypedDict) maps caching/retry/timeout/metadata/ custom-cost to cf-aig-* request headers. - Lists Cloudflare in the plugin's provider docstring; hermetic unit tests for URL building, auth modes, and the gateway-options header mapping. Closes livekit#6101
This was referenced Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #6101. Adds
openai.LLM.with_cloudflare(...), a convenience constructor that routesLLM traffic through the Cloudflare AI Gateway
unified OpenAI-compatible endpoint — mirroring the existing
with_openrouter/with_azurehelpers.
Design notes
cf-aig-authorization(gateway token) andAuthorization(BYO downstream provider key); at least one is required. In gateway-stored-keysmode the OpenAI SDK still needs a non-empty
Authorization, so a"cloudflare"placeholder isused — the same pattern as
with_ollama.Cloudflare's controls are
cf-aig-*HTTP headers.gateway_optionsis aCloudflareGatewayOptionsTypedDict covering caching, retries, timeout, metadata, and customcost; set fields map to the corresponding headers (
metadata/custom_costJSON-encoded). Agrouped TypedDict (over loose kwargs) keeps the signature to one extra param while typing the
structured values. Validation is type-level only; numeric ranges pass through to Cloudflare.
account_id(envCLOUDFLARE_ACCOUNT_ID) andgateway_id(default"default"), with abase_urloverride.Testing
tests/test_openai_with_cloudflare.py(hermetic,--unit): URL building fromaccount_id/gateway_id,
CLOUDFLARE_ACCOUNT_IDenv fallback,base_urloverride, BYOK vsgateway-stored-keys auth, the full
gateway_options → cf-aig-*header mapping (incl. JSONencoding and the
skip_cache-only-when-true rule), and allValueErrorpaths.make checkpasses (ruff format + lint, strict mypy).
Out of scope
STT/TTS via Cloudflare Workers AI (#3163), tracked separately.