Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,4 @@ sources: []
- [数值精确性与平台可移植性规则](rules-numeric-exactness.md)
- [Model Executor — NPU KV cache 布局规则](rules-npu-kv-layout.md)
- [Model Executor — Dynamo/cudagraph compile guard rules](rules-compile-guard.md)
- [Model Executor — INC/AutoRound 量化配置规则](rules-inc-autoround.md)
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: "Model Executor — INC/AutoRound 量化配置规则"
created: 2026-10-13
updated: 2026-10-13
type: rule
tags: [vllm-omni, components, model-executor]
sources: ["PR #8870"]
---

# Model Executor — INC/AutoRound 量化配置规则

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add the required related-page links

This new deposited page contains no Markdown links, although doc/knowledge/SCHEMA.md requires every deposited page to link at least two related pages. Add outgoing links to the existing AutoRound dispatch rule and the relevant Qwen-Omni owner page so the page satisfies the knowledge contract and remains navigable; the current validators do not catch this requirement.

AGENTS.md reference: AGENTS.md:L7-L11

Useful? React with 👍 / 👎.


## VLLM-OMNI-PR8870-INC-PARSER — OmniINCConfig 升级后必须重建 config_parser,使阶段前缀重映射对层解析可见

- 触发:改动 `vllm_omni/quantization/inc_config.py` 中 `OmniINCConfig.from_inc_config` / `maybe_upgrade` 的属性复制或 `config_parser` 接线,或改动 `apply_vllm_mapper` 之后的层名解析路径;也覆盖 Qwen2.5-Omni AutoRound W4A16 的加载与 `tests/e2e/offline_inference/test_qwen2_5_omni_autoround_w4a16_expansion.py` 中的 e2e 测试。
- 强制:`from_inc_config` 以 `omni.__dict__.update(inc.__dict__)` 复制原 INCConfig 属性后,必须用升级后的实例重建 parser(`omni.config_parser = type(inc.config_parser)(omni)`)。否则被复制的 parser 仍指向原 vanilla `INCConfig`,`apply_vllm_mapper` 对升级配置上 `block_name_to_quantize` 的前缀重映射(`thinker.model.layers` → `thinker.language_model.model.layers`)在层解析时不可见,受影响 thinker 层退化为未量化方法,加载 checkpoint 打包 `qweight` 张量时报 `There is no module or parameter named 'layers.0.mlp.down_proj.qweight'`(issue #5652)。升级与重映射行为由 `tests/diffusion/quantization/test_component_routing.py::test_upgraded_config_resolves_remapped_blocks` 回归守护:重映射后的 thinker 层解析为 (4, 128, True)、量化块之外的层保持未量化 (16, -1, True)、原始 vanilla 配置的 `block_name_to_quantize` 不被改写。issue #5652 专属的 e2e skip 已随本修复移除,Qwen2.5-Omni AutoRound W4A16 五条 e2e 测试须保持启用;Qwen3-Omni 同路径测试因独立的部署后端限制(默认 `moe_backend: triton` 被 vLLM 0.31.0 对该 AutoRound MoE checkpoint 拒绝)仍保持跳过,不得以本规则要求启用。
- 禁止:令升级后的 `OmniINCConfig` 沿用指向原 `INCConfig` 的 `config_parser`,或以其他方式使 `get_layer_config` 读不到重映射后的块名;删除或绕过 `test_upgraded_config_resolves_remapped_blocks`;以本次修复已解决为由给 Qwen2.5-Omni AutoRound e2e 重新加回 issue-5652 skip,或把 Qwen3-Omni 的跳过误判为本规则违反。
- 验收:`from_inc_config` 在复制 `__dict__` 后重建 `config_parser` 指向新实例;`pytest tests/diffusion/quantization/test_component_routing.py` 通过;`test_qwen2_5_omni_autoround_w4a16_expansion.py` 不再含 `_SKIP_ISSUE_5652` 且五个测试均无该 skip 装饰。^[PR #8870]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Wait for PR #8870 to merge before activating this rule

While upstream PR #8870 remains open, this active rule's ^[PR #8870] citation makes the change unmergeable: src/infermatrix_copilot/knowledge_service/facts.py treats every PR citation in an active rule as a claim that the PR is merged and rejects the gate when that claim is false. The page also describes the fix and removed skips as current behavior even though they exist only on the open PR head, so defer this rule until that upstream change is merged.

Useful? React with 👍 / 👎.


<!-- kb:rule status=active since=v0.30.0 -->
Loading