Rapid-MLX: Fast local OpenAI-compatible LLM provider for LlamaIndex on Mac

[raullenchai]

Sharing Rapid-MLX as a fast local LLM provider for LlamaIndex on Apple Silicon.

Setup with LlamaIndex:

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="default",
    api_base="http://localhost:8000/v1",
    api_key="not-needed",
    is_chat_model=True
)

Start the server: rapid-mlx serve qwen3.5-9b

Why Rapid-MLX?

• 2-4x faster than Ollama on Apple Silicon (M1-M4)
• OpenAI API compatible — works with LlamaIndex OpenAI client
• Tool calling, structured output, streaming all supported
• 22 models tested and auto-configured

Benchmarks:

• Qwen3.5-9B: 79 tok/s (vs Ollama 33 tok/s)
• Qwen3.5-4B on 16GB Air: 168 tok/s

Install: brew install raullenchai/rapid-mlx/rapid-mlx or pip install rapid-mlx