Skip to content

Amarshar/weight free decode moe ep - #1470

Open
amarquic wants to merge 7 commits into
qualcomm:mainfrom
amarquic:amarshar/weight-free-decode-moe-ep
Open

amarquic wants to merge 7 commits into
qualcomm:mainfrom
amarquic:amarshar/weight-free-decode-moe-ep

Conversation

@amarquic

@amarquic amarquic commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Title

Add weight-free MoE expert-parallel and decode-BMM coverage

Summary

This PR adds weight-free QAIC tests for MoE prefill and decode configurations, makes QAIC test core selection device-aware, and configures Jenkins to store prepared weight-free checkpoints in the test-owned cache.

Changes

  • Cover MoE prefill with expert parallelism and decode with decode_bmm.

  • Add expert-parallel decode coverage for selected VLM and continuous-batching tests.

  • Check the configured MoE flavor and expert-parallel chunk size through model hash parameters.

  • Compare QAIC-generated tokens with Hugging Face output for the additional expert-parallel decode paths.

  • Replace hardcoded QAIC core counts with DEFAULT_AIC_NUM_CORES.

  • Add the use_onnx_subfunctions guard to the Qwen3-MoE parity test.

  • Keep generated weight-free checkpoints under QEFF_HOME for test cleanup.

  • Set QEFF_WF_HOME in Jenkins to:

    /efficient-transformers/weight_free_qaic/weight_free_checkpoints
    
  • Preserve the Gemma4 vision overflow fix by folding standardization scales before multiplying by the root hidden size.

Test coverage and configurations

Test (test_file_path::test_name) Model / parameter Configuration covered
tests/weight_free/test_disaggregated.py::test_weight_free_disaggregated_prefill_and_decode[qwen3-moe] tiny-random/qwen3-moe Disaggregated expert-parallel prefill; decode_bmm decode; mxint8_kv_cache=False
tests/weight_free/test_disaggregated.py::test_weight_free_disaggregated_prefill_and_decode[gpt-oss] tiny-random/gpt-oss Disaggregated expert-parallel prefill; decode_bmm decode; mxint8_kv_cache=False
tests/weight_free/test_disaggregated.py::test_weight_free_vlm_disaggregated_prefill_and_decode[qwen3-vl-moe] tiny-random/qwen3-vl-moe Vision and expert-parallel prefill; separate decode_bmm and expert-parallel decode paths; HF token parity; mxint8_kv_cache=False
tests/weight_free/test_disaggregated.py::test_weight_free_vlm_disaggregated_prefill_and_decode[gemma-4-moe] tiny-random/gemma-4-moe Vision and expert-parallel prefill; separate decode_bmm and expert-parallel decode paths; HF token parity; mxint8_kv_cache=False
tests/weight_free/test_disaggregated.py::test_weight_free_qwen3_moe_disagg_cb_kv_handoff_and_hf_parity[local_qwen3_moe_prefill2_decode1_stages2] Local tiny Qwen3-MoE FP32 checkpoint Continuous batching, disaggregated KV handoff, expert-parallel decode, and HF token parity; mxint8_kv_cache=False
tests/weight_free/test_disaggregated.py::test_weight_free_gpt_oss_disagg_cb_kv_handoff_and_hf_parity[tiny_model_prefill2_decode1_stages2] tiny-random/gpt-oss Continuous batching, disaggregated KV handoff, expert-parallel decode, and HF token parity; mxint8_kv_cache=False
tests/weight_free/test_on_qaic_vlm.py::test_weight_free_vlm_multi_specialization_hf_qaic_parity[qwen3.5-moe] tiny-random/qwen3.5-moe Two vision resolutions; expert-parallel prefill; separate decode_bmm and expert-parallel decode paths; HF token parity. MXINT8 KV cache is omitted and remains disabled by default.

Signed-off-by: Hem Agnihotri <hemagnih@qti.qualcomm.com>

@quic-amitraj Amit Raj (quic-amitraj) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

amarquic lets fix the DCO issue and correct the PR title. Other changes LGTM.

Signed-off-by: amarshar <amarshar@qti.qualcomm.com>
Signed-off-by: amarshar <amarshar@qti.qualcomm.com>
Signed-off-by: amarshar <amarshar@qti.qualcomm.com>
Signed-off-by: amarshar <amarshar@qti.qualcomm.com>
Signed-off-by: amarshar <amarshar@qti.qualcomm.com>
Signed-off-by: amarshar <amarshar@qti.qualcomm.com>
@amarquic
amarquic force-pushed the amarshar/weight-free-decode-moe-ep branch from d420e52 to 0db5edd Compare October 30, 2026 06:49

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test coverage table is descriptive and good. It would be helpful to include the test file path also in the table for easy lookup. test_file_path::test_name

@amarquic

Copy link
Copy Markdown
Contributor Author

Test coverage table is descriptive and good. It would be helpful to include the test file path also in the table for easy lookup. test_file_path::test_name

Test coverage table is descriptive and good. It would be helpful to include the test file path also in the table for easy lookup. test_file_path::test_name

Updated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants