[feature request] nvfp4 .saftensors support for Klein 4B/9B

[VladiCz]

I would like to ask If you plan supporting black-forest-labs/FLUX.2-klein-4b-nvfp4 and possibly black-forest-labs/FLUX.2-klein-9b-nvfp4 for RTX Blackwell native?

That would probably allow for more real-time steps / higher-res / lower mem Blackwell cards to be supported or significant speed gain.

[tensorforger]

Hi, I have tried this model (the first one), but its loading is tricky.
These quantized models are better when you need to reduce vram usage, not latency. Latency is usually almost same.
I have already added support of int8 quantization and 20-24 GB cards.
Running this on < 20 GB cards becomes not practical as they are usually also much slower.

You can try adding this by your own. If there would be some meaningful speedup or vram usage decrease compared to current int8 approach, it would be added.

[VladiCz]

Hi, I have tried this model (the first one), but its loading is tricky.
These quantized models are better when you need to reduce vram usage, not latency. Latency is usually almost same.
I have already added support of int8 quantization and 20-24 GB cards.
Running this on < 20 GB cards becomes not practical as they are usually also much slower.

You can try adding this by your own. If there would be some meaningful speedup or vram usage decrease compared to current int8 approach, it would be added.

You are right, it makes sense.