docs(llm): document LM Studio / remote GPU box as an LLM backend

[BrettKinny]

Why

A recurring support question from new users: "Can I run the LLM in LM Studio on my Windows PC that has the GPU?"

The answer is yes, with no code changes — OpenAICompat already speaks that wire format — but the docs never said so concretely. docs/llm-backends.md mentioned LM Studio only in a passing list, and every worked example was OpenRouter / Anthropic / llama-swap. Worse, the CUDA block in docker-compose.yml.template reads as though a GPU on the Docker host were a prerequisite for the LLM, when it only ever governed WhisperLocal ASR. Users were also conflating compose.local.override.yml (Ollama on the Docker host's own GPU) with serving models off a second machine.

What changed

Docs only — no behaviour change, no code touched.

docs/llm-backends.md

• New "LM Studio (or any inference box on another machine)" subsection under the OpenAI-compatible backend, with a worked .config.yaml and the three failure modes behind most "it won't connect" reports:
  • loopback vs LAN IP (the config is read from inside a container on a different host)
  • LM Studio's localhost-only default binding — "Serve on Local Network" must be enabled
  • host firewall blocking the inbound port
• Added curl http://<GPU_BOX_IP>:1234/v1/models as the pre-flight check; it doubles as the way to get the exact model id. Noted that make doctor does not validate that value, so a typo surfaces at runtime as "My brain returned an error" from the robot.
• Stated explicitly that the Docker host needs no GPU for this, and distinguished compose.local.override.yml from the remote-inference case.
• Noted that switching off PiVoiceLLM makes the docker.sock + docker CLI mounts unnecessary, and that they should be removed — the socket mount is root-equivalent on the Docker host, so dropping it when nothing uses it is a real reduction in blast radius.
• Comparison table: "Runs where" / "Privacy" rows no longer imply the OpenAI-compatible backend is cloud-only.

docker-compose.yml.template

• Marked the CUDA block ASR-only, with a pointer to docs/llm-backends.md.

Notes

• Last verified: at the foot of llm-backends.md is deliberately unchanged — only the OpenAICompat material was re-checked against the code this pass, not the llama-swap or PiVoiceLLM sections.
• Config snippets were checked against custom-providers/openai_compat/openai_compat.py and .config.yaml.template; the mount advice against compose.all-in-one.yml.
• YAML validity of the template re-verified after the comment addition.

AI assistance

Drafted with Claude Code in response to a user support question; reviewed by @BrettKinny before merge. See AI_TRANSPARENCY.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pqs9dxTt13WHG1GynHTXax