diff --git a/README.md b/README.md index 95198c8b8..dd2cd7a63 100644 --- a/README.md +++ b/README.md @@ -264,6 +264,11 @@ hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf .\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav --backend cuda ``` +The current Windows CUDA package targets CUDA architecture 86. RTX 50 / Blackwell +GPUs report compute capability 12.0 (`sm_120`) and should use the CPU package or +build from source with `-DCMAKE_CUDA_ARCHITECTURES=120` until a dedicated CUDA +asset is published. + **Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.1.7](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.7) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.7/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/) [OpenAI API example →](./examples/openai_api/) · [Gradio demo →](./examples/openai_api/GRADIO.md) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript recipes →](./examples/openai_api/JAVASCRIPT.md) · [Kubernetes template →](./examples/openai_api/kubernetes/) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Postman collection →](./examples/openai_api/POSTMAN.md) · [OpenAPI spec →](./examples/openai_api/OPENAPI.md) · [Security guide →](./examples/openai_api/SECURITY.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html) diff --git a/README_zh.md b/README_zh.md index 0860c62db..601c4d682 100644 --- a/README_zh.md +++ b/README_zh.md @@ -303,6 +303,10 @@ hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf .\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav --backend cuda ``` +当前 Windows CUDA 包面向 CUDA architecture 86。RTX 50 / Blackwell GPU 会报告 +compute capability 12.0(`sm_120`),在专用 CUDA 产物发布前,请使用 CPU 包,或从 +源码构建并设置 `-DCMAKE_CUDA_ARCHITECTURES=120`。 + **预编译二进制:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.1.7](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.7) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.7/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF 模型:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测:** [runtime/llama.cpp/](./runtime/llama.cpp/) [OpenAI API 示例 →](./examples/openai_api/README_zh.md) · [Gradio Demo →](./examples/openai_api/GRADIO_zh.md) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript 配方 →](./examples/openai_api/JAVASCRIPT_zh.md) · [Kubernetes 模板 →](./examples/openai_api/kubernetes/README_zh.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS_zh.md) · [Postman 集合 →](./examples/openai_api/POSTMAN_zh.md) · [OpenAPI 规范 →](./examples/openai_api/OPENAPI_zh.md) · [安全指南 →](./examples/openai_api/SECURITY_zh.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html) diff --git a/runtime/llama.cpp/README.md b/runtime/llama.cpp/README.md index 2439551fd..f38460507 100644 --- a/runtime/llama.cpp/README.md +++ b/runtime/llama.cpp/README.md @@ -59,7 +59,7 @@ cmake -B build -DCMAKE_BUILD_TYPE=Release # fetches pinned llama.cpp; stati cmake --build build -j # -> build/bin/llama-funasr-* (all tools) ``` -### Optional CUDA backend for SenseVoiceSmall +### Optional Windows CUDA backend for SenseVoiceSmall The CPU release ZIPs are portable packages. Tagged releases also publish `funasr-llamacpp-windows-x64-cuda.zip` for SenseVoiceSmall graph execution on @@ -73,15 +73,21 @@ then select the backend at runtime: -m sensevoice-small-q8.gguf --vad fsmn-vad.gguf -a sample.wav --backend cuda ``` -Build from source to target a different GPU architecture: +Build from source to target other GPU architectures: ```bash -cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON +cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \ + -DCMAKE_CUDA_ARCHITECTURES=120 cmake --build build-cuda -j --target llama-funasr-sensevoice ./build-cuda/bin/llama-funasr-sensevoice \ -m sensevoice-small-f16.gguf -a sample.wav --backend cuda ``` +Use the matching `CMAKE_CUDA_ARCHITECTURES` value for your GPU. RTX 50 / +Blackwell cards report compute capability 12.0 (`sm_120`), so the current +`windows-x64-cuda` prebuilt package for architecture 86 will not cover those +cards. + `--backend cpu` remains the default and is what the portable cross-platform prebuilt binaries use. The CUDA package requires an NVIDIA driver compatible with the CUDA Toolkit version configured by the release workflow. A binary built diff --git a/runtime/llama.cpp/tests/test_release_workflow.py b/runtime/llama.cpp/tests/test_release_workflow.py index 18f749952..201c97698 100644 --- a/runtime/llama.cpp/tests/test_release_workflow.py +++ b/runtime/llama.cpp/tests/test_release_workflow.py @@ -39,3 +39,5 @@ def test_release_notes_explain_cpu_and_cuda_windows_assets(): assert "CUDA architecture 86" in readme assert "Build from source" in readme assert "other GPU architectures" in readme + assert "CMAKE_CUDA_ARCHITECTURES=120" in readme + assert "sm_120" in readme