support qwen3 on nvidia

[icyxp]

Support Qwen3 on Nvidia

[NikiBase]

wrong url https://huggingface.co/collections/Qwen/qwen3-67c6c6f89c4f76621268bb6d
I think it must be this one https://huggingface.co/collections/Qwen/qwen3-67dd247413f0e2e4f653967f

[NikiBase]

Hi @icyxp, I have tried your pull request on local using the image with llamacpp backend and it does not work
https://huggingface.co/docs/text-generation-inference/main/en/backends/llamacpp#build-docker-image
I had updated the llamacpp version on the Dockerfile_llamacpp to the latest version but still not working for Qwen3 model, maybe I should change something else but not sure. Could you please explain how you run your pull request.
Thank you in advance.

[icyxp]

@NikiBase Pls use Dockerfile,Not tested on llamacpp. maybe you can check out this: https://qwen.readthedocs.io/en/latest/run_locally/llama.cpp.html

[NikiBase]

I am trying to run the docker build from the base dockerfile and I got the following error on this stage

ERROR [base 10/26] RUN cd server &&  uv sync --frozen --extra gen --extra bnb --extra accelerate --extra compressed-tensors --extra quantize --extra peft --extra outlines --extra t

...

583.8 requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: https://huggingface.co/api/resolve-cache/models/kernels-community/moe/e3efab933893cde20c5417ba185fa3b7cc811b24/build%2Ftorch27-cxx11-cu128-x86_64-linux%2Fmoe%2Fconfigs%2FE%3D8%2CN%3D8192%2Cdevice_name%3DAMD_Instinct_MI325X%2Cdtype%3Dfp8_w8a8.json

did you experienced this issue during the build phase?

I am running it again to see if it is a temporal connection issue, will tell you if it resolves this way

Thank you!