Skip to content

knowledge(vllm-omni): 1 change(s) from intake #349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tzhouam merged 1 commit into from
Oct 11, 2026
Merged

knowledge(vllm-omni): 1 change(s) from intake #349

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions knowledge/repos/vllm-omni/components/configuration/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,4 @@ sources: ["claude-workflow-starter-private@296ea45", vllm_omni/config/]
| 冻结 topology、辅助 stage、mixed V1/MRv2 runner、deploy profile 与 `additional_config` 投影 | [topology 与部署 profile](rules-topology-profiles.md) |

- [部署 platform overlay 的最终配置合同](rules-platform-overlays.md)。
- [配置规则 — CUDA graph 显存预留与 KV 预算](rules-cudagraph-kv-reserve.md)
19 changes: 19 additions & 0 deletions knowledge/repos/vllm-omni/components/configuration/rules-cudagraph-kv-reserve.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: "配置规则 — CUDA graph 显存预留与 KV 预算"
created: 2026-10-11
updated: 2026-10-11
type: rule
tags: [vllm-omni, components, config]
sources: ["PR #7632"]
---

# 配置规则 — CUDA graph 显存预留与 KV 预算

## VLLM-OMNI-PR7632-CUDAGRAPH-KV — 启用 graph 的 stage 必须从自动 KV 预算中扣除 CUDA graph 显存

- 触发：修改 `vllm_omni/worker/base.py`、`vllm_omni/worker/memory_utils.py` 或 `vllm_omni/engine/stage_admission.py` 中的自动 KV cache sizing、CUDA graph memory estimate、`VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS` 读取，或 `gpu_memory_utilization` 预算语义的文档。

@chatgpt-codex-connector chatgpt-codex-connector Bot Oct 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Route the worker budget rule to the model-executor owner

The trigger is explicitly changes under vllm_omni/worker/, but the component map assigns workers to Model Executor and the release baseline likewise maps vllm_omni/worker/ to that owner. Keeping this rule under Configuration means a Direct review routed through components/model-executor/rules.md will not receive this CUDA-graph/KV safety rule; move or route it through the Model Executor rule entry point.

AGENTS.md reference: AGENTS.md:L12-L17

Useful? React with 👍 / 👎.

- 强制：CUDA-alike 平台且 graph mode 启用时，在既有 memory profile 之后调用继承的 graph profiler，把估计值从自动 KV 预算中扣除并 clamp 到 0；显式 `kv_cache_memory_bytes` pin 继续绕过该估计，`VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0` 跳过该 profile 并保持 graph pool 不预留。预留必须以一次性 guidance 日志说明：给出能恢复原 KV 大小的 `gpu_memory_utilization`，或当提升后的预算会超过启动时空闲显存时明确说没有任何值能恢复原大小（预算已被 `min(requested, free)` cap 时，提高 utilization 不增加预算）。新增读取的 env var 须登记 `environment_variable_inventory.py`（本例归入 INHERITED_VLLM，计数 20→21）并同步数量快照与 `docs/configuration/gpu_memory_utilization.md`。

@chatgpt-codex-connector chatgpt-codex-connector Bot Oct 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove the one-time inventory count from the active rule

The 20→21 transition is a snapshot of PR #7632 rather than an executable invariant. As soon as another inherited variable is added or removed, reviewers following this active rule will expect an obsolete count and may reject a correctly regenerated snapshot; retain the requirement to reconcile the inventory and tests, but leave the historical numeric delta in PR history.

AGENTS.md reference: AGENTS.md:L12-L17

Useful? React with 👍 / 👎.

- 禁止：让启用 graph 的 stage 把 graph pool 留给 KV cache；在预算已被 free memory cap 时仍建议提高 utilization；把 startup estimate 声称为覆盖启动后模型自捕获的 graph——`GPUARModelRunner.capture_model` 的 Talker MTP graphs 与 model `capture_auxiliary_graphs` hooks 不在估计内，使用这些 graph 的 stage 必须在 utilization 之上另留池空间；新增 env var 不登记 inventory 分类或只改数量断言（通用同步要求见 VOMNI-CFG-3a）。

@chatgpt-codex-connector chatgpt-codex-connector Bot Oct 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add the required related-page links

This new page contains no relative Markdown links, even though the repository's SCHEMA.md contract requires every sediment-layer page to link at least two related pages. The plain VOMNI-CFG-3a mention does not provide navigation; link it to the owning rule anchor and add another relevant page such as the deployment-memory-budget contract.

AGENTS.md reference: AGENTS.md:L7-L11

Useful? React with 👍 / 👎.

- 验收：tests/worker 覆盖 reserve 开关、zero clamp、capped 与 uncapped 预算、平台与 graph gate、重复 profiling、预算耗尽、显式 KV bytes、guidance 日志及继承 profiler 签名；revert clamp 或 skip 应使相应 worker-budget case 失败；`tests/config/test_environment_variables.py` 的分类数量快照与文档同步更新。 ^[PR #7632]

<!-- kb:rule status=active since=v0.30.0 -->
Loading