Skip to content

docs: clarify Windows CUDA GPU architecture support #3291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
LauraGPT merged 1 commit into from
Jul 19, 2026
Merged

docs: clarify Windows CUDA GPU architecture support #3291

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,11 @@ hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav --backend cuda
```

The current Windows CUDA package targets CUDA architecture 86. RTX 50 / Blackwell
GPUs report compute capability 12.0 (`sm_120`) and should use the CPU package or
build from source with `-DCMAKE_CUDA_ARCHITECTURES=120` until a dedicated CUDA
asset is published.
Comment on lines +267 to +270

@gemini-code-assist gemini-code-assist Bot Jul 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The Blackwell architecture (RTX 50 series) has a compute capability of 10.0 (sm_100), not 12.0 (sm_120). Using -DCMAKE_CUDA_ARCHITECTURES=120 will result in a compilation error because sm_120 is not a valid CUDA architecture. It should be updated to 10.0 / sm_100 / -DCMAKE_CUDA_ARCHITECTURES=100.

Suggested change:

The current Windows CUDA package targets CUDA architecture 86. RTX 50 / Blackwell
GPUs report compute capability 10.0 (`sm_100`) and should use the CPU package or
build from source with `-DCMAKE_CUDA_ARCHITECTURES=100` until a dedicated CUDA
asset is published.


**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.1.7](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.7) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.7/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/)

[OpenAI API example →](./examples/openai_api/) · [Gradio demo →](./examples/openai_api/GRADIO.md) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript recipes →](./examples/openai_api/JAVASCRIPT.md) · [Kubernetes template →](./examples/openai_api/kubernetes/) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Postman collection →](./examples/openai_api/POSTMAN.md) · [OpenAPI spec →](./examples/openai_api/OPENAPI.md) · [Security guide →](./examples/openai_api/SECURITY.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)
Expand Down
4 changes: 4 additions & 0 deletions README_zh.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,10 @@ hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav --backend cuda
```

当前 Windows CUDA 包面向 CUDA architecture 86。RTX 50 / Blackwell GPU 会报告
compute capability 12.0（`sm_120`），在专用 CUDA 产物发布前，请使用 CPU 包，或从
源码构建并设置 `-DCMAKE_CUDA_ARCHITECTURES=120`。
Comment on lines +306 to +308

@gemini-code-assist gemini-code-assist Bot Jul 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The Blackwell architecture (RTX 50 series) has a compute capability of 10.0 (sm_100), not 12.0 (sm_120). Using -DCMAKE_CUDA_ARCHITECTURES=120 will result in a compilation error because sm_120 is not a valid CUDA architecture. It should be updated to 10.0 / sm_100 / -DCMAKE_CUDA_ARCHITECTURES=100.

Suggested change:

当前 Windows CUDA 包面向 CUDA architecture 86。RTX 50 / Blackwell GPU 会报告
compute capability 10.0（`sm_100`），在专用 CUDA 产物发布前，请使用 CPU 包，或从
源码构建并设置 `-DCMAKE_CUDA_ARCHITECTURES=100`。


**预编译二进制：** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.1.7](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.7) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.7/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始：** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF 模型：** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测：** [runtime/llama.cpp/](./runtime/llama.cpp/)

[OpenAI API 示例 →](./examples/openai_api/README_zh.md) · [Gradio Demo →](./examples/openai_api/GRADIO_zh.md) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript 配方 →](./examples/openai_api/JAVASCRIPT_zh.md) · [Kubernetes 模板 →](./examples/openai_api/kubernetes/README_zh.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS_zh.md) · [Postman 集合 →](./examples/openai_api/POSTMAN_zh.md) · [OpenAPI 规范 →](./examples/openai_api/OPENAPI_zh.md) · [安全指南 →](./examples/openai_api/SECURITY_zh.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)
Expand Down
12 changes: 9 additions & 3 deletions runtime/llama.cpp/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ cmake -B build -DCMAKE_BUILD_TYPE=Release # fetches pinned llama.cpp; stati
cmake --build build -j # -> build/bin/llama-funasr-* (all tools)
```

### Optional CUDA backend for SenseVoiceSmall
### Optional Windows CUDA backend for SenseVoiceSmall

The CPU release ZIPs are portable packages. Tagged releases also publish
`funasr-llamacpp-windows-x64-cuda.zip` for SenseVoiceSmall graph execution on
Expand All @@ -73,15 +73,21 @@ then select the backend at runtime:
-m sensevoice-small-q8.gguf --vad fsmn-vad.gguf -a sample.wav --backend cuda
```

Build from source to target a different GPU architecture:
Build from source to target other GPU architectures:

```bash
cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \
-DCMAKE_CUDA_ARCHITECTURES=120
Comment on lines +79 to +80

@gemini-code-assist gemini-code-assist Bot Jul 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The Blackwell architecture (RTX 50 series) uses compute capability 10.0 (sm_100). Specifying -DCMAKE_CUDA_ARCHITECTURES=120 will cause nvcc to fail with an unsupported GPU architecture error. It should be -DCMAKE_CUDA_ARCHITECTURES=100.

Suggested change:

cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \
  -DCMAKE_CUDA_ARCHITECTURES=100

cmake --build build-cuda -j --target llama-funasr-sensevoice
./build-cuda/bin/llama-funasr-sensevoice \
-m sensevoice-small-f16.gguf -a sample.wav --backend cuda
```

Use the matching `CMAKE_CUDA_ARCHITECTURES` value for your GPU. RTX 50 /
Blackwell cards report compute capability 12.0 (`sm_120`), so the current
`windows-x64-cuda` prebuilt package for architecture 86 will not cover those
cards.
Comment on lines +86 to +89

@gemini-code-assist gemini-code-assist Bot Jul 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The Blackwell architecture (RTX 50 series) reports compute capability 10.0 (sm_100), not 12.0 (sm_120).

Suggested change:

Use the matching `CMAKE_CUDA_ARCHITECTURES` value for your GPU. RTX 50 /
Blackwell cards report compute capability 10.0 (`sm_100`), so the current
`windows-x64-cuda` prebuilt package for architecture 86 will not cover those
cards.


`--backend cpu` remains the default and is what the portable cross-platform
prebuilt binaries use. The CUDA package requires an NVIDIA driver compatible
with the CUDA Toolkit version configured by the release workflow. A binary built
Expand Down
2 changes: 2 additions & 0 deletions runtime/llama.cpp/tests/test_release_workflow.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,5 @@ def test_release_notes_explain_cpu_and_cuda_windows_assets():
assert "CUDA architecture 86" in readme
assert "Build from source" in readme
assert "other GPU architectures" in readme
assert "CMAKE_CUDA_ARCHITECTURES=120" in readme
assert "sm_120" in readme
Comment on lines +42 to +43

@gemini-code-assist gemini-code-assist Bot Jul 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Update the test assertions to match the correct Blackwell compute capability of 10.0 (sm_100) and the corresponding CMake flag -DCMAKE_CUDA_ARCHITECTURES=100.

Suggested change
assert "CMAKE_CUDA_ARCHITECTURES=120" in readme
assert "sm_120" in readme
assert "CMAKE_CUDA_ARCHITECTURES=100" in readme
assert "sm_100" in readme

Loading