A production-ready, multi-agent voice assistant built from scratch using LiveKit Agents SDK. Features consent collection, manager escalation with a different Cartesia voice, semantic turn detection, and multi-model fallback.
🔗 Based on the workshop: Building Production-Ready Voice Agents with LiveKit
- 🎤 Real-time voice conversation via WebRTC (LiveKit)
- 🧠 Multi-model LLM fallback — OpenAI GPT-4.1 Mini → Google Gemini 2.5 Flash
- 🗣️ Multi-model STT fallback — AssemblyAI Universal Streaming → Deepgram Nova-3
- 🔊 Multi-model TTS fallback — Cartesia Sonic-3 → Inworld TTS-1
- 🔇 Background noise cancellation via LiveKit BVC
- 🛑 Semantic Turn Detection — no awkward mid-sentence interruptions (
MultilingualModel) - ⚡ Preemptive generation for ultra-low latency responses
- ✅ Consent Collection Task — legally compliant recording consent before call starts
- 👨💼 Manager Escalation — seamless handoff to
ManagerAgentwith a different Cartesia voice - 🗂️ Full conversation history preserved across all agent handoffs
- 🐳 Docker support for containerized deployment
- ☁️ LiveKit Cloud deployment ready via
lkCLI
🖥️ Local Console Testing Demo — click to expand
☁️ LiveKit Cloud Deployment Demo — click to expand
| Category | Provider | Model / Details |
|---|---|---|
| LLM (Primary) | OpenAI | gpt-4.1-mini |
| LLM (Fallback) | gemini-2.5-flash |
|
| STT (Primary) | AssemblyAI | universal-streaming:en |
| STT (Fallback) | Deepgram | nova-3 |
| TTS — Assistant | Cartesia | sonic-3 · voice 9626c31c-bec5-4cca-baa8-f8ba9e84c8bc |
| TTS — Manager | Cartesia | sonic-3 · voice 6f84f4b8-58a2-430c-8c79-688dad597532 |
| TTS (Fallback) | Inworld | inworld-tts-1 |
| VAD | Silero | — |
| Turn Detection | LiveKit | MultilingualModel (semantic) |
| Noise Cancellation | LiveKit | BVC |
| Infrastructure | LiveKit Cloud | WebRTC |
User joins room
│
▼
┌──────────────────────┐
│ CollectConsent Task │ ◄─ Asks for recording permission (Yes / No)
└────────┬─────────────┘
│
Yes ─┤─ No
│ └─► Proceed without recording
▼
┌──────────────────────┐
│ Assistant Agent │ ◄─ Friendly CSR · Cartesia Voice 1
│ │ Handles general queries
└────────┬─────────────┘
│
"I want a manager"
│
▼
┌──────────────────────┐
│ Manager Agent │ ◄─ Empathetic Manager · Cartesia Voice 2
│ │ Full chat history preserved ✅
└──────────────────────┘
WORKSHOP-DEMO/
└── livekit-voice-agent/
├── agent.py # Main voice agent — all agent classes & entrypoint
├── .env # API keys (not committed to git)
├── .env.example # Environment variable template
├── pyproject.toml # uv project config & dependencies
├── uv.lock # Locked dependency versions
├── Dockerfile # Docker container config
├── .dockerignore # Docker ignore rules
├── livekit.toml # LiveKit Cloud deployment config
└── README.md # This file
- Python 3.11+
uvpackage manager- LiveKit Cloud account → cloud.livekit.io
- LiveKit CLI (
lk) → Install guide
git clone https://github.com/suyashsahu00/WORKSHOP-DEMO.git
cd WORKSHOP-DEMO/livekit-voice-agentuv synccp .env.example .envOpen .env and fill in your LiveKit credentials:
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your_api_key_here
LIVEKIT_API_SECRET=your_api_secret_here🔑 Get your API keys here: LiveKit Cloud API Keys
Note: All model inference (OpenAI, AssemblyAI, Cartesia, Deepgram, Inworld) runs via LiveKit Cloud Inference — no separate API keys needed!
uv run agent.py consoleThen open LiveKit Agents Playground, connect to your room, and start talking!
uv run agent.py devdocker build -t livekit-voice-agent .docker run --env-file .env livekit-voice-agentlk cloud authlk agent deployLiveKit Cloud Dashboard → Agents → status should be Running ✅
| Commit | Message |
|---|---|
8b6f5ba |
feat: replace tech support persona with Dr. Sydney health assistant |
2f37fce |
feat: initialize project with uv dependency management and configuration files |
7c8876e |
feat: change Sydney persona from health assistant to weather girl |
febb85d |
feat: connecting voice agent to external services with MCP |
ef7aaf7 |
feat: implement production-ready LiveKit voice agent with semantic turn detection, fallback models, and manager escalation |
latest |
feat: add multi-agent voice system with consent workflow, manager handoff, and Cartesia TTS override |
- ImportError: Ensure you have run
uv syncto install all plugins listed inpyproject.toml. - API Key Errors: Double-check your
.envfile. EnsureGROQ_API_KEY,DEEPGRAM_API_KEY, andMURF_API_KEYare correctly set alongside your LiveKit credentials. - Voice Not Working: Some voices like "Tanushree" are premium on Murf AI. Ensure your API key has access to the requested voice and model (FALCON/GEN2).
- Network Issues: If running locally, ensure you have a stable internet connection as the STT, LLM, and TTS all communicate with cloud providers.
| Resource | Link |
|---|---|
| 🔗 Workshop Tutorial | Building Production-Ready Voice Agents with LiveKit |
| 🔑 LiveKit API Keys | cloud.livekit.io → API Keys |
| 📖 LiveKit Agents Docs | docs.livekit.io/agents |
| 🧪 Agents Playground | agents-playground.livekit.io |
| 💬 LiveKit Community | LiveKit Slack |
| ☁️ LiveKit Cloud | cloud.livekit.io |
MIT License © 2026 Suyash Sahu
This branch introduces several high-performance updates to the tech stack:
- 🧠 High-Performance LLM — Groq Llama-3.3-70b-versatile
- 🗣️ Ultra-fast STT — Deepgram Nova-2
- 🔊 Premium TTS — Murf AI (Tanushree Voice · FALCON Model)
| Category | Provider | Model / Details |
|---|---|---|
| LLM | Groq | llama-3.3-70b-versatile |
| STT | Deepgram | nova-2 |
| TTS | Murf AI | Tanushree (FALCON model) |
To use the features from this branch, get your keys from the following consoles and add them to your .env file:
- Groq: console.groq.com
- Deepgram: console.deepgram.com
- Murf AI: murf.ai/api
GROQ_API_KEY=your_groq_api_key_here
DEEPGRAM_API_KEY=your_deepgram_api_key_here
MURF_API_KEY=your_murf_api_key_here