diff --git a/.CLAUDE/context/terminology.md b/.CLAUDE/context/terminology.md index 78b8a970..81c1c931 100644 --- a/.CLAUDE/context/terminology.md +++ b/.CLAUDE/context/terminology.md @@ -89,6 +89,7 @@ Speechmatics does not sell a "voice agent." A voice agent is a full conversation | SDK | sdk | All-caps. | | job | Job | Lowercase common noun. | | transcript | Transcript | Lowercase unless starting a sentence. | +| Mandarin | Chinese Mandarin, Mandarin Chinese | Language name, matching the canonical row on the languages page (`cmn`). Locale names are "Simplified Mandarin" (`cmn-Hans`) and "Traditional Mandarin" (`cmn-Hant`). | --- diff --git a/docs/private/voice-agent-api.mdx b/docs/private/voice-agent-api.mdx index 0ec355a7..2926052c 100644 --- a/docs/private/voice-agent-api.mdx +++ b/docs/private/voice-agent-api.mdx @@ -41,7 +41,7 @@ Adapts to each speaker's pace over the course of a conversation. It adjusts the **Best for:** General conversational voice agents. -**Languages:** All supported languages. Disfluency detection is English-only — other languages fall back to speech-rate adaptation. +**Languages:** All supported languages. Disfluency detection is available for the [supported languages](https://docs.speechmatics.com/speech-to-text/formatting#supported-languages-for-disfluencies) — other languages fall back to speech-rate adaptation. **Trade-off:** Latency varies by speaker. diff --git a/docs/speech-to-text/formatting.mdx b/docs/speech-to-text/formatting.mdx index a585e9ce..59bea9e1 100644 --- a/docs/speech-to-text/formatting.mdx +++ b/docs/speech-to-text/formatting.mdx @@ -37,7 +37,7 @@ Available English locales: - US English (`en-US`) - Australian English (`en-AU`) -Available Chinese Mandarin locales: +Available Mandarin locales: - Simplified Mandarin (`cmn-Hans`, default) - Traditional Mandarin (`cmn-Hant`) @@ -78,7 +78,7 @@ For other languages, consider using [word replacement](#word-replacement) to ide ## Disfluencies -Disfluencies are hesitation sounds like "um", "uh", and "hmm". In English, these are automatically tagged with `disfluency`: +Disfluencies are hesitation sounds like "um", "uh", and "hmm". Speechmatics automatically tags them with `disfluency` in the transcript output: ```json "results": [ @@ -101,7 +101,7 @@ Disfluencies are hesitation sounds like "um", "uh", and "hmm". In English, these ```
- Full list of tagged disfluencies + Full list of tagged English disfluencies {`huh aha @@ -147,6 +147,33 @@ uum`}
+### Supported languages for disfluencies + +Disfluency tagging and removal are available for the following languages. Each language has its own set of hesitation sounds; the expandable list at the start of this section covers English. + +- Arabic (`ar`) +- Danish (`da`) +- Dutch (`nl`) +- English (`en`) +- Finnish (`fi`) +- French (`fr`) +- German (`de`) +- Greek (`el`) +- Hebrew (`he`) +- Hindi (`hi`) +- Hungarian (`hu`) +- Italian (`it`) +- Japanese (`ja`) +- Mandarin (`cmn`) +- Norwegian (`no`) +- Polish (`pl`) +- Portuguese (`pt`) +- Russian (`ru`) +- Spanish (`es`) +- Swedish (`sv`) + +Coverage of hesitation sounds varies by language. If you rely on disfluency removal for a specific language, test it with representative audio rather than assuming full coverage. + ### Removing disfluencies You can automatically remove disfluencies from your transcript: @@ -169,7 +196,7 @@ Without disfluency removal: With disfluency removal: > What would you like? -This feature is available for English only. The default setting is `"remove_disfluencies": false`. +This feature is available for the [supported languages](#supported-languages-for-disfluencies). The default setting is `"remove_disfluencies": false`. ## Word replacement @@ -341,7 +368,7 @@ The system applies different formatting rules based on the type of entity detect The system chooses entity classes based on context, so occasionally a value might be classified differently than expected. For example, "2001" could be a "cardinal" number or a "date". ::: -### Languages +### Languages for smart formatting Each language follows its own conventions for: @@ -357,7 +384,6 @@ Examples: Smart formatting has had dedicated work for consistent results in these languages: - Cantonese -- Chinese Mandarin (Simplified and Traditional) - Dutch - English - French @@ -365,6 +391,7 @@ Smart formatting has had dedicated work for consistent results in these language - Hindi - Italian - Japanese +- Mandarin (Simplified and Traditional) - Mandarin & English (bilingual) - Mandarin Malay Tamil & English (multilingual) - Norwegian diff --git a/spec/realtime.yaml b/spec/realtime.yaml index db9490c8..53bd0887 100644 --- a/spec/realtime.yaml +++ b/spec/realtime.yaml @@ -1127,7 +1127,7 @@ components: tags: type: array description: | - This is a set list of profanities and disfluencies respectively that cannot be altered by the end user. `[disfluency]` is only present in English, and `[profanity]` is present in English, Spanish, and Italian + This is a set list of profanities and disfluencies respectively that cannot be altered by the end user. `[disfluency]` is present in the [supported languages](https://docs.speechmatics.com/speech-to-text/formatting#supported-languages-for-disfluencies), and `[profanity]` is present in English, Spanish, and Italian items: $ref: "#/components/schemas/RecognitionAlternativeTagsEnum" required: