Repository navigation
knowledge(vllm-omni): 1 change(s) from intake #363
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| --- | ||
| title: "Model Executor — INC/AutoRound 量化配置规则" | ||
| created: 2026-10-13 | ||
| updated: 2026-10-13 | ||
| type: rule | ||
| tags: [vllm-omni, components, model-executor] | ||
| sources: ["PR #8870"] | ||
| --- | ||
|
|
||
| # Model Executor — INC/AutoRound 量化配置规则 | ||
|
|
||
| ## VLLM-OMNI-PR8870-INC-PARSER — OmniINCConfig 升级后必须重建 config_parser,使阶段前缀重映射对层解析可见 | ||
|
|
||
| - 触发:改动 `vllm_omni/quantization/inc_config.py` 中 `OmniINCConfig.from_inc_config` / `maybe_upgrade` 的属性复制或 `config_parser` 接线,或改动 `apply_vllm_mapper` 之后的层名解析路径;也覆盖 Qwen2.5-Omni AutoRound W4A16 的加载与 `tests/e2e/offline_inference/test_qwen2_5_omni_autoround_w4a16_expansion.py` 中的 e2e 测试。 | ||
| - 强制:`from_inc_config` 以 `omni.__dict__.update(inc.__dict__)` 复制原 INCConfig 属性后,必须用升级后的实例重建 parser(`omni.config_parser = type(inc.config_parser)(omni)`)。否则被复制的 parser 仍指向原 vanilla `INCConfig`,`apply_vllm_mapper` 对升级配置上 `block_name_to_quantize` 的前缀重映射(`thinker.model.layers` → `thinker.language_model.model.layers`)在层解析时不可见,受影响 thinker 层退化为未量化方法,加载 checkpoint 打包 `qweight` 张量时报 `There is no module or parameter named 'layers.0.mlp.down_proj.qweight'`(issue #5652)。升级与重映射行为由 `tests/diffusion/quantization/test_component_routing.py::test_upgraded_config_resolves_remapped_blocks` 回归守护:重映射后的 thinker 层解析为 (4, 128, True)、量化块之外的层保持未量化 (16, -1, True)、原始 vanilla 配置的 `block_name_to_quantize` 不被改写。issue #5652 专属的 e2e skip 已随本修复移除,Qwen2.5-Omni AutoRound W4A16 五条 e2e 测试须保持启用;Qwen3-Omni 同路径测试因独立的部署后端限制(默认 `moe_backend: triton` 被 vLLM 0.31.0 对该 AutoRound MoE checkpoint 拒绝)仍保持跳过,不得以本规则要求启用。 | ||
| - 禁止:令升级后的 `OmniINCConfig` 沿用指向原 `INCConfig` 的 `config_parser`,或以其他方式使 `get_layer_config` 读不到重映射后的块名;删除或绕过 `test_upgraded_config_resolves_remapped_blocks`;以本次修复已解决为由给 Qwen2.5-Omni AutoRound e2e 重新加回 issue-5652 skip,或把 Qwen3-Omni 的跳过误判为本规则违反。 | ||
| - 验收:`from_inc_config` 在复制 `__dict__` 后重建 `config_parser` 指向新实例;`pytest tests/diffusion/quantization/test_component_routing.py` 通过;`test_qwen2_5_omni_autoround_w4a16_expansion.py` 不再含 `_SKIP_ISSUE_5652` 且五个测试均无该 skip 装饰。^[PR #8870] | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
While upstream PR #8870 remains open, this active rule's Useful? React with 👍 / 👎. |
||
|
|
||
| <!-- kb:rule status=active since=v0.30.0 --> | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This new deposited page contains no Markdown links, although
doc/knowledge/SCHEMA.mdrequires every deposited page to link at least two related pages. Add outgoing links to the existing AutoRound dispatch rule and the relevant Qwen-Omni owner page so the page satisfies the knowledge contract and remains navigable; the current validators do not catch this requirement.AGENTS.md reference: AGENTS.md:L7-L11
Useful? React with 👍 / 👎.