feat(flux): support flux-general-* on /v2/listen - #14
Merged
Conversation
Flux is a different endpoint, not another model on /v1/listen, and the two disagree about what a parameter is. v1 drops query params it does not know and transcribes anyway; v2 refuses the whole handshake for a single unknown one and returns a body that reads only "Unexpected error when initializing websocket connection" — no parameter, no reason. The params panel sends a pile of v1 fields on every request, so without gating a Flux stream simply never starts. Selecting a flux-* model now switches the endpoint, the URL bar, the params panel and the response parsing together. - stt/options.py grows an ALLOWLIST for Flux (the mode denylist is a v1 idea) plus flux_validation_error, which exists purely for the message: Deepgram enforces the same rules and refuses to say which knob is wrong. - app.py routes on the model, and parses both endpoints through one _transcript_event so a turn maps onto is_final the same way everywhere. - The panel hides every control v2 would reject, collapses sections left empty, and shows /v2/listen in the URL bar so a copied curl works. Every parameter, bound and behaviour here was handshaked against production rather than read off a page, because the two available sources are both wrong: the docs list `version` as supported (400) and cap eot_timeout_ms at 10000 (the real ceiling is 60000), while the SDK names `language_hint` unconditionally though it is a 400 on flux-general-en. Three traps worth naming, all measured: - Every v2 message arrives as a plain dict. V2SocketClientResponse is a union containing a bare typing.Any, which construct_type matches first, so isinstance(msg, ListenV2TurnInfo) is never true. The v1 union has no Any and does yield models, so a handler copied from the v1 one drops every turn in silence. Both halves of that asymmetry are now pinned by tests. - Flux has no KeepAlive control message and no send_keep_alive on the socket client, so the v1 keep-alive loop would have killed every mic stream with an AttributeError after eight seconds. - A turn ends on silence in the AUDIO, and CloseStream flushes turns that have already ended rather than forcing the open one shut. Audio cut tight to the last syllable loses its final turn: same 19-word clip, 8 runs each, 8/8 complete with a second of trailing silence and 0/8 without. Rather than inject silence — which cannot work for containerised WebM and would make the tool transcribe audio the user never supplied — the app detects the open turn and explains it, including the text that did not make the transcript. Also fixed, found while building this: - _clean_error leaked the Deepgram API key. Its header stripping was only a side effect of extracting the "status_code: N, body: ..." tail, so any SDK error without that tail fell through to `return str(e)` with the Authorization header intact — and v2 handshake failures are exactly that shape. Redaction is unconditional now. - The browser kept its own copies of the mode lists, in two functions, and they had drifted: the JS was missing channels, encoding, sample_rate, filler_words, measurements, utt_split and detect_language, so the URL bar advertised a request that was not the one being sent. They now come from /api/param-gating, which serves stt/options.py, and a test fails if the two ever disagree. The URL bar also applies the keyterms/tags aliases and the zero-means-unset rule, so endpointing=0 stops vanishing from the preview. - The panel offered a `callback` field the server has always refused to forward (it is a data-exfil primitive). A control that cannot affect the request reads as evidence the feature was tested; it is hidden now, along with anything else the gate would strip. - streaming_task and file_streaming_task were two copies of the same lifecycle. Adding an endpoint to both would have doubled the divergence, so they are one _run_stream with the driving half passed in. - probe_ui.py never collected .toggle-label, so --expect-absent on any switch passed without checking anything. It also gains --model, --expect-url-contains and --expect-url-absent, since Flux gating is model-driven and the URL bar is part of the contract.
- Keep-alive gated on the ENDPOINT, not on hasattr(ws, "send_keep_alive"). The constraint is that /v2/listen has only Configure and CloseStream, not that this SDK build happens not to expose the method — a later regeneration could add it and the hasattr form would silently start sending something Flux never agreed to accept. Tested with a ws that does expose it. - The TTS round trip reported no truncation. It is the sweep path that writes CSVs, so a silently short Flux transcript there becomes a data point someone later cites as a model result. _TurnTracker is now shared between it and the socket path, and the round trip returns the unfinalized text alongside a notice rather than a quietly short transcript. - Flux in-band errors were rendered by guessing at field names. FatalError actually carries `code` and `description`; ConfigureFailure carries NEITHER, so the JSON fallback fired and dumped a three-field object that explains nothing. Each now says what it can, and ConfigureFailure says plainly that Deepgram gave no reason.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Flux is a different endpoint, not another model on
/v1/listen, and the two disagree about what a parameter is. Selectingflux-general-enorflux-general-multinow switches the endpoint, the URL bar, the params panel and the response parsing together.Why this needed real work
v1 drops query params it does not recognise and transcribes anyway. v2 refuses the entire handshake for a single unknown one, and the whole response body is:
No parameter name, no value, no reason. The params panel sends a pile of v1 fields on every request, so without gating a Flux stream simply never starts — and the user is told nothing about why. That is why Flux gating is an allowlist (
FLUX_PARAMS) rather than the mode denylist v1 uses, and whyflux_validation_errorduplicates checks Deepgram already performs: purely to produce a message that names the offending knob.Everything here was measured, because both sources are wrong
versionsupportedeot_timeout_msmax 10000/docs/flux/configurationlanguage_hintalways availableconnect()signatureflux-general-enredactany valuenumbers,aggressive_numbersBounds bisected live and inclusive:
eot_threshold0.5–0.9,eager_eot_threshold0.3–0.9,eot_timeout_ms500–60000,eager <= eot.Three traps worth naming
Every v2 message arrives as a plain
dict.V2SocketClientResponseis a union containing a baretyping.Any, whichconstruct_typematches first, soisinstance(msg, ListenV2TurnInfo)is never true. The v1 union has noAnyand does yield models. A handler copied from the v1 one drops every turn in silence. Both halves of that asymmetry are pinned by tests, so a future SDK that tightens the union fails loudly instead of quietly changing behaviour.Flux has no
KeepAlive— onlyConfigureandCloseStream, and nosend_keep_aliveon the v2 socket client. The v1 keep-alive loop would have killed every Flux mic stream withAttributeErrorafter eight seconds. Gated on the endpoint, not onhasattr, so a later SDK adding the method cannot make us start sending one.A turn ends on silence in the AUDIO, and
CloseStreamdoes not force the open turn shut. Audio cut tight to the last syllable loses its final turn. Same 19-word clip, 8 runs each: 8/8 complete with a second of trailing silence, 0/8 without — the failures returned only the 8 words of the first completed turn. Rather than inject silence (impossible for containerised WebM, and it would make the tool transcribe audio the user never supplied), the app detects the open turn and explains it, including the text that did not make the transcript.Also fixed, found while building this
_clean_errorleaked the Deepgram API key. Its header stripping was only a side effect of extracting thestatus_code: N, body: ...tail, so any SDK error without that tail fell through toreturn str(e)with theAuthorizationheader intact — and v2 handshake failures are exactly that shape. Redaction is unconditional now, with a test.channels,encoding,sample_rate,filler_words,measurements,utt_split,detect_language. The URL bar advertised a request that was not the one being sent — the worst possible bug in an app whose headline feature is "copy this URL into curl". Rules now come fromGET /api/param-gating, and a test fails if it stops matchingstt/options.py.callbackfield the server has always refused to forward (data-exfil primitive). A control that cannot affect the request reads as evidence the feature was tested. Hidden, along with anything else the gate strips.streaming_taskandfile_streaming_taskwere two copies of one lifecycle. Now one_run_streamwith the driving half passed in.probe_ui.pynever collected.toggle-label, so--expect-absenton any switch passed without checking anything. Also gains--model,--expect-url-contains,--expect-url-absent.Live verification
Against production Deepgram, through the app's own SocketIO layer (not the SDK directly):
Truncation notice, audio deliberately chopped mid-sentence:
A/B, same audio through the SDK and through the app, 3/3 identical — confirming the app is faithful to the endpoint rather than adding its own truncation.
Visual artifacts
scripts/probe_ui.py, zero console errors, all DOM + URL-bar expectations met:flux-general-enapi.deepgram.com/v2/listen?model=flux-general-enflux-general-multi/v2/listen+language_hintvisiblenova-3streaming/v1/listen?...&interim_results=true&vad_events=truenova-3batch/v1/listen?...no streaming-only paramsOn Flux the panel renders exactly the controls v2 accepts and nothing else;
Features,IntelligenceandStreamingcollapse entirely rather than showing headers over empty bodies.163 passed, 1 skipped.