Skip to content

feat(npu): route gated norm to aclnn layer norm fwd#1919

Draft
Fengfengst123 wants to merge 1 commit into
xLLM-AI:mainfrom
Fengfengst123:codex-aclnn-layernorm-fwd-runtime
Draft

feat(npu): route gated norm to aclnn layer norm fwd#1919
Fengfengst123 wants to merge 1 commit into
xLLM-AI:mainfrom
Fengfengst123:codex-aclnn-layernorm-fwd-runtime

Conversation

@Fengfengst123

Copy link
Copy Markdown

Summary

  • Route the NPU gated normalization path to aclnnLayerNormFwd.
  • Enable the AscendC implementation by default.
  • Support fallback to the original Triton implementation by setting XLLM_USE_ACLNN_LAYER_NORM_FWD=0.

Performance

  • Single-op performance improves by approximately 1.45x compared with the original Triton implementation.
  • Qwen3.5 TP1/DP1 TTFT improves by approximately 4.34%, while TPOT remains roughly unchanged.

Validation

  • ATK single-op accuracy validation passed.
  • CPU golden comparison passed for the covered BF16, FP16, and FP32 LayerNorm/RMSNorm cases.
  • Qwen3.5 TP1/DP1 end-to-end CEval validation passed with the AscendC operator enabled.
  • xllm-ops runtime integration harness passed all 18 checks

return dtype == torch::kBFloat16 || dtype == torch::kFloat32;
}

bool env_flag_enabled(const char* name, bool default_value) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why add ?

@Fengfengst123 Fengfengst123 marked this pull request as draft July 10, 2026 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants