Missing MXFP4 quantized models for Qwen3-235B-A22B, MiniMax-M2.5, and GLM-5

[sunway513]

Summary

Several high-priority frontier MoE models are missing from the AMD Quark MXFP4 quantized models collection on HuggingFace (amd/ org). NVIDIA has already published NVFP4 versions for all of these, creating a benchmark gap.

Missing Models

Model	HuggingFace ID	Params	Active	NVIDIA NVFP4 Available?
Qwen3-235B-A22B	Qwen/Qwen3-235B-A22B	235B	22B	✅ nvidia/Qwen3-235B-A22B-NVFP4
MiniMax-M2.5	MiniMaxAI/MiniMax-M2.5	229B	10B	✅ nvidia/MiniMax-M2.5-NVFP4
GLM-5	zai-org/GLM-5	745B	44B	✅ nvidia/GLM-5-NVFP4

Existing AMD MXFP4 Models (for reference)

The following models already have MXFP4 versions:

• amd/DeepSeek-R1-0528-MXFP4 ✅
• amd/Kimi-K2.5-MXFP4 ✅
• amd/GLM-5-MXFP4 ✅ (exists but has weight shape issues in ATOM — see below)
• amd/Qwen3.5-397B-A17B-MXFP4 ✅
• amd/Qwen3-Coder-Next-MXFP4 ✅

Why This Matters

1. Benchmark parity: Without MXFP4 versions, MI355X benchmarks must use FP8 (2x larger, 2x less compute density), making MI355X vs B300 comparisons unfair
2. InferenceMAX/InferenceX: SemiAnalysis benchmarks use NVFP4 on NVIDIA side — AMD needs equivalent MXFP4 for apples-to-apples comparison
3. Customer readiness: These are top-5 deployed open MoE models as of March 2026

Additional Issue: GLM-5 MXFP4 Weight Shape Bug

amd/GLM-5-MXFP4 exists on HuggingFace but fails to load in ATOM with:

RuntimeError: The size of tensor a (3072) must match the size of tensor b (12288) at non-singleton dimension 1

This occurs in the weight loader during MXFP4 unpacking with TP=8. The GLM-5 hidden_size=6144 divided by TP=8 gives 768, but the MXFP4 packed tensor dimension is 12288 (expecting undivided size). Please verify the quantization output format.

Request

1. Publish amd/Qwen3-235B-A22B-MXFP4 (or Instruct variant)
2. Publish amd/MiniMax-M2.5-MXFP4
3. Fix and re-publish amd/GLM-5-MXFP4 (weight shape issue)

cc @quark-team

[sunway513]

@thiagocrepaldi — Could you take a look at this? We're running MI355X vs B300 inference benchmarks for frontier MoE models and the missing MXFP4 weights are blocking fair comparisons. NVIDIA already has NVFP4 versions for all of these on HuggingFace.

Priority order:

1. Qwen3-235B-A22B — highest enterprise demand, no MXFP4 safetensors exist anywhere
2. MiniMax-M2.5 — needed for competitive benchmarks
3. GLM-5 MXFP4 fix — amd/GLM-5-MXFP4 exists but has weight shape mismatch in ATOM

Any ETA or the right team to route this to?

[niuxjamd]

Hi, @sunway513,

These 3 models are available on Huggingface:
https://huggingface.co/amd/Qwen3-235B-A22B-Instruct-2507-MXFP4
https://huggingface.co/amd/MiniMax-M2.5-MXFP4
https://huggingface.co/amd/GLM-5-MXFP4

Will put resource in my team to validate these models in ATOM.

[djdeniro]

can you did also Qwen3.5-397B-A17B-MXFP4 with no mixed quants (mxfp4/mxfp6)? it's hard to launch in 8xR9700, got OOM, but worked with --enforce-eager mode at 10 t/s

[djdeniro]

@niuxjamd

[niuxjamd]

@djdeniro Is this what you need: https://huggingface.co/amd/Qwen3.5-397B-A17B-MXFP4?

[djdeniro]

@niuxjamd with this model i got OOM, and warnings

when MTP on:

RuntimeError: The size of tensor a (2048) must match the size of tensor b (4096) at non-singleton dimension 1
args to launch:

non-default args: {'model_tag': '/app/models/models/vllm/Qwen3.5-397B-A17B-MXFP4', 'chat_template': '/app/models/models/vllm/chat_template.jinja', 'enable_auto_tool_choice': True, 'tool_call_parser': 'qwen3_coder', 'host': '0.0.0.0', 'model': '/app/models/models/vllm/Qwen3.5-397B-A17B-MXFP4', 'trust_remote_code': True, 'max_model_len': 155648, 'served_model_name': ['model'], 'reasoning_parser': 'qwen3', 'tensor_parallel_size': 8, 'gpu_memory_utilization': 0.967, 'enable_prefix_caching': True, 'max_num_seqs': 16, 'speculative_config': {'method': 'qwen3_next_mtp', 'num_speculative_tokens': 2}, 'compilation_config': {'mode': None, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': [], 'splitting_ops': None, 'compile_mm_encoder': False, 'compile_sizes': None, 'compile_ranges_endpoints': None, 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': None, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 32, 64, 128], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': None, 'pass_config': {}, 'max_cudagraph_capture_size': 128, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': None, 'static_all_moe_layers': []}}

warrnings

(APIServer pid=1) WARNING 04-07 20:03:50 [quark_ocp_mx.py:168] AITER is not found or QuarkOCP_MX is not supported on the current platform. QuarkOCP_MX quantization will not be available.
(APIServer pid=1) INFO 04-07 20:03:51 [scheduler.py:231] Chunked prefill is enabled with max_num_batched_tokens=2048.

(APIServer pid=1) WARNING 04-07 20:03:51 [config.py:379] Mamba cache mode is set to 'align' for Qwen3_5MoeForConditionalGeneration by default when prefix caching is enabled
WARNING 04-07 20:04:03 [quark_ocp_mx.py:168] AITER is not found or QuarkOCP_MX is not supported on the current platform. QuarkOCP_MX quantization will not be available.

i think main error from this warning

(Worker_TP0 pid=552) WARNING 04-07 20:05:36 [quark_moe.py:765] The current mode (supports_mx=False, use_mxfp4_aiter_moe=None, ocp_mx_scheme=OCP_MX_Scheme.w_mxfp4_a_mxfp4) does not support native MXFP4/MXFP6 computation. Simulated weight dequantization and activation QDQ (quantize and dequantize) will be used, with the linear layers computed in high precision.

This model works with --enforce-eager flag only, without it i got OOM always.

[djdeniro]

This model works very good: https://huggingface.co/olka-fi/Qwen3.5-122B-A10B-MXFP4, but it looks different quantization method.

with GPTQ-Int4 from Qwen, i have always exclamation mark as answer, because --dtype half.

[BowenBao]

Hi @djdeniro, mxfp4 dtype is not natively supported on the device you are running (R9700). mxfp4 model is designed for MI355x devices. In your case, please try FP8 https://huggingface.co/Qwen/Qwen3.5-397B-A17B-FP8

[djdeniro]

@BowenBao i create my own mxfp4 quantization, and it's works well. i made power limit for 210W per GPU and got 33t/s

https://huggingface.co/djdeniro/Qwen3.5-397B-A17B-MXFP4