Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,4 @@ sources: []

- [AutoRound NVFP4 method dispatch 合同](rules-autoround-dispatch.md)。
- [数值精确性与平台可移植性规则](rules-numeric-exactness.md)
- [Model Executor — NPU KV cache 布局规则](rules-npu-kv-layout.md)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Route the NPU rule through the owner entrypoint

When Direct selects the model-executor owner, _routes.yaml sends it to components/model-executor/rules.md and the scoped workflow stops navigating after identifying that owner. Because this new rule is linked only from _index.md and has no trigger or link in rules.md, reviews involving this KV-layout scenario will not receive it; add it to the owner page's Direct quick map/code routing rather than relying on the index.

AGENTS.md reference: knowledge/AGENTS.md:L5-L10

Useful? React with 👍 / 👎.

Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: "Model Executor — NPU KV cache 布局规则"
created: 2026-10-12
updated: 2026-10-12
type: rule
tags: [vllm-omni, components, model-executor]
sources: ["PR #8827"]
---

# Model Executor — NPU KV cache 布局规则

## VLLM-OMNI-PR8827-CONTIG-KV — NPU K/V 布局按 page padding 判定,patch 以 __code__ 替换全局生效

- 触发:修改 `vllm_omni/platforms/npu/platform.py` 的 K/V cache 布局策略、`requires_contiguous_pa_kv_cache` patch 或 `NPUOmniPlatform.__init__` 的 patch 顺序;或 vLLM-Ascend 升级后 Ascend 上 Qwen3-TTS serving 性能回归(#8795)。
- 强制:contiguous 判定必须实现为 `spec.page_size_bytes == spec.real_page_size_bytes`:unpadded page 保持 vLLM-Ascend 0.30.0 及更早的连续分离 K/V 分配(K、V 各自 contiguous 且 storage 不别名);padded page 留在 packed 单 tensor 路径,因为 V2 的分离 K/V 分配只支持 unpadded page。
- 强制:`apply_contiguous_kv_patch()` 必须通过替换 `vllm_ascend.attention.utils.requires_contiguous_pa_kv_cache.__code__` 生效,不得重绑模块属性——这样 `attention.utils`、`worker.model_runner_v1`、`worker.v2.attn_utils` 的按引用导入才同步生效;patch 必须幂等(重复调用不叠加),并在 `NPUOmniPlatform.__init__` 的 `adapt_patch(is_global_patch=True)` 之后调用。
- 禁止:为统一布局把 padded page 也切到连续分离分配;用属性重绑或层层包装代替 `__code__` 替换,导致部分模块仍执行旧判定或重复 patch 叠加。
- 验收:`tests/platforms/npu/test_contiguous_kv_cache.py` 加入 `.buildkite/npu/test-npu-ready.yml` 的 `core_model and npu and A3 and omni` lane。v1 case(`NPUModelRunner._allocate_kv_cache_tensors`)断言 double-apply 后 `attention.utils`、`worker.model_runner_v1`、`worker.v2.attn_utils` 三处引用同一函数对象,且 K/V 各自 contiguous、storage 指针不同;v1 不做 fill 隔离。v2 case(`attn_utils._allocate_kv_cache`+`_reshape_kv_cache_v2`)断言 `original(None, None, spec)` 对 unpadded/padded 返回 True/False:unpadded 时 K/V contiguous、storage 指针不同,padded 时返回单 tensor 且 `stride(0)*element_size == page_size_bytes`;fill 隔离(K 填 1、V 填 2 后互不串扰)只在 v2 的两种参数化中断言。导入 runner 前先 import `vllm.v1.worker.gpu.spec_decode.speculator` 与 `vllm_ascend.ops`。^[PR #8827]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add the required related-page links

This new stable knowledge page contains no relative Markdown links, but doc/knowledge/SCHEMA.md lines 94–96 require every settled-layer page to link at least two related pages. The incoming _index.md link does not satisfy that page-level requirement; link the owner rules.md and relevant platform-backend or validation page from this rule.

AGENTS.md reference: AGENTS.md:L7-L10

Useful? React with 👍 / 👎.


<!-- kb:rule status=active since=v0.30.0 -->
Loading