llama.cpp OpenAI-compatible server on Vulkan for AMD Strix Halo (gfx1151), GGUF weights pinned to GTT not VRAM. Stock-image Vulkan stack plus an opt-in ROCmFP4 + MTP stack: Ubuntu 26.04 + TheRock ROCm 7.13, built for plunderstruck's Qwen3.6 quants. Docker Compose, with real measured benchmarks.
docker docker-compose amd vulkan mtp rocm gtt therock llama-cpp local-llm llm-inference qwen gguf speculative-decoding ryzen-ai openai-compatible strix-halo gfx1151 radv rocmfp4
-
Updated
Jul 9, 2026 - Python