Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion knowledge/repos/vllm-omni/components/configuration/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,5 +49,6 @@ sources: ["claude-workflow-starter-private@296ea45", vllm_omni/config/]
| 审查或新增 process 环境变量、stage `env` 和安全诊断输出 | [environment-variable contract](environment-variable-contract.md) |
| 计算多 stage 显存预算 | [deployment memory budget](deployment-memory-budget.md) |
| 添加新模型和注册点 | [adding a model](adding-a-model.md) |
| composable strategy 的 axis、并行拓扑与 load balancing owner | [并行拓扑合同](rules-parallel-topology.md) |
| composable strategy 的 axis、并行拓扑、load balancing owner 与 resolve_omni_config 统一解析 | [并行拓扑合同](rules-parallel-topology.md) |

| 冻结 topology、辅助 stage 注入与模型专用 deploy profile | [topology 与部署 profile](rules-topology-profiles.md) |
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: "并行拓扑合同"
created: 2026-09-04
updated: 2026-09-04
updated: 2026-09-09
type: rule
tags: [vllm-omni, components, config]
sources: [vllm_omni/config/composable_parallel/, vllm_omni/config/config_factory.py, vllm_omni/config/stage_config.py, "PR #5531"]
sources: [vllm_omni/config/composable_parallel/, vllm_omni/config/config_factory.py, vllm_omni/config/stage_config.py, "PR #5531", "PR #5140"]
confidence: high
---

Expand Down Expand Up @@ -64,3 +64,10 @@ confidence: high
- 禁止:在配置解析期把省略的 DP 固定为 1 后据此生成不足的设备列表;接受与 WORLD 推导值不一致的显式 DP;允许 HSDP 与 TP、普通 DP、PP 或 EP 组合;只用 `num_gpus` 做校验却不传播到 stage 配置。
- 验收:覆盖 direct、structured 和 async stage 路径;`num_gpus=8`、TP=2、SP=2 必须得到 DP=2、WORLD=8 和 devices `0`–`7`,省略 DP 的配置在运行时解析前仍保持未指定;覆盖 HSDP mesh 不匹配及不支持组合,并确认都在进程组创建前失败。^[PR #5531]


## CONF-4f — 生产启动必须经 resolve_omni_config 统一解析

- 触发:修改 `AsyncOmniEngine`、headless stage 启动、`StageConfigFactory` 公开入口,或 registry/deploy/CLI 到最终 stage 的 source selection。
- 强制:标准与 headless 生产路径只调用 `resolve_omni_config()` 获得 `OmniConfigResolution`(structured config、runtime compatibility stages、strategy-derived LB、effective deploy path);factory 的 create/legacy 方法是 resolver 与测试的下层 primitive,engine/entrypoint 不得绕过 resolver 直接选源。通用单 stage diffusion 仅在 registry 解析失败后作为 fallback;legacy `stage_configs_path`/`stage_configs` 必须显式报错。
- 禁止:engine 侧二次合成 default diffusion stage 或重复 merge LoRA/attention/quant/cache/profiler;把 OmegaConf-compatible stages 宣传为稳定公共 ABI;静默恢复已删除的 stage-configs-path 行为。
- 验收:同一 model+deploy+override 在标准与 headless 路径得到等价 topology 与 LB policy;覆盖 registry 命中、generic diffusion fallback、以及拒绝已删除 legacy 参数。^[PR #5140]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add every new PR to its page-level sources

This rule cites PR #5140, but the page's frontmatter sources: remains unchanged; the same omission affects newly cited PRs #7078, #7195, #7041, #7071, #7191, #6876, and #7228 in the other edited pages. The knowledge schema requires page-level evidence in sources:, so prose-only footnotes lose structured provenance; add each PR to the corresponding page metadata.

AGENTS.md reference: AGENTS.md:L7-L10

Useful? React with 👍 / 👎.

9 changes: 6 additions & 3 deletions knowledge/repos/vllm-omni/components/diffusion/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,16 +37,19 @@ sources: ["PR #5543", vllm_omni/platforms/cuda/platform.py, vllm_omni/diffusion/
| CustomOp XPU platform dispatch、PyTorch-native fallback、CUDA contract 与非同义 override 边界 | [CustomOp dispatch rules](rules-custom-op-dispatch.md) |
| runtime temporary 与 loader-scoped parameter dtype、shared RMSNorm accuracy | [tensor dtype rules](rules-tensor-dtype.md) |
| Wan VAE height/width spatial reshard、empty tail、attention extent | [Wan spatial-shard rules](rules-wan-spatial-shard.md) |
| multi-DiT、dotted `_dit_modules`、loader-to-offloader handoff 与跨 Cache-DiT/compile/LoRA/offload lifecycle | [component lifecycle rules](rules-component-lifecycle.md) |
| PEFT 与 distilled LoRA、startup fusion、delta/key/alpha、Qwen/Wan transformer mapping | [LoRA rules](rules-lora.md) |
| multi-DiT、dotted `_dit_modules`、loader-to-offloader handoff 与跨 Cache-DiT/compile/LoRA/offload lifecycle、batch sampling-key 的 provided 标志 | [component lifecycle rules](rules-component-lifecycle.md) |

| PEFT 与 distilled LoRA、startup fusion、delta/key/alpha、Qwen/Wan transformer mapping、PEFT suspend/resume | [LoRA rules](rules-lora.md) |

| Blackwell FA4、显式/auto CUDA backend、per-role mask/SP pad;local FlashAttention deterministic opt-in、role-aware masked cross-attention、NPU packed mask-free/laser fallback、TRTLLM packed-padding 与 config propagation | [attention rules](rules-attention.md) |
| SP auto-padding、`mask_sp_padding`、dense/varlen 与 advanced UAA 边界 | [SP padding rules](rules-sp-padding.md) |
| video/audio mux、DLO DP wave、RPC reply ownership/result release、result queue、async pump、SHM ownership、async-output wait 与 shutdown | [output/runtime rules](rules-output-lifecycle.md) |
| distilled continuous sigma schedule、boundary/step 语义与 modality shift | [sigma schedule rules](sigma-schedules.md) |
| HunyuanImage3 Distil CFG、MeanFlow special-token / scheduler-r 合同 | [HunyuanImage3 HY3-8d](../../models/hunyuan-image3/rules.md) |
| diffusion step 与 request/continuous batching;DIFF-1af BAGEL image step wave、packed state、geometry/CFG admission 与 fallback cleanup | [step and batching](step-and-batching.md) |
| request-wave admission coalescing、stable window、deadline 与 finite config | [admission wait rules](rules-admission-wait.md) |
| paged KV/cache 预算、native/backend/platform 闭环、GQA/Ring/Ulysses layout、FlashInfer plan、能力 metadata | [paged cache 与系统运行时规则](rules-system-runtime.md) |
| paged KV/cache 预算、native/backend/platform 闭环、GQA/Ring/Ulysses layout、FlashInfer plan、能力 metadata、cache_backend None 规范化 | [paged cache 与系统运行时规则](rules-system-runtime.md) |

| Scheduler-managed diffusion KV 的请求控制面、Hunyuan layout 与未实现边界 | [paged KV control plane](paged-kv-control-plane.md) |
| Cache-DiT、TeaCache 和 prefix cache | [cache acceleration](cache-acceleration.md) |
| TP/PP/SP/CFG/VAE/HSDP 等并行策略 | [parallelism](parallelism.md) |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: "Diffusion component lifecycle 规则"
created: 2026-09-02
updated: 2026-09-05
updated: 2026-09-09
type: rule
tags: [vllm-omni, components, diffusion]
sources: ["PR #5720", "PR #5853", "PR #5882", "PR #5884", "PR #6486", "PR #6591", vllm_omni/diffusion/cache/base.py, vllm_omni/diffusion/cache/cachedit/backend.py, vllm_omni/diffusion/cache/cachedit/runtime.py, vllm_omni/diffusion/lora/manager.py, vllm_omni/diffusion/models/interface.py, vllm_omni/diffusion/offloader/module_collector.py, vllm_omni/diffusion/offloader/startup.py, vllm_omni/diffusion/registry.py, vllm_omni/diffusion/sched/interface.py, vllm_omni/diffusion/worker/diffusion_model_runner.py, tests/diffusion/cache/test_cache_backends.py, tests/diffusion/cache/test_cache_dit_request_runtime.py, tests/diffusion/models/sana_video/test_cache_offload.py, tests/diffusion/test_diffusion_model_runner.py, tests/diffusion/test_diffusion_scheduler.py, "PR #6070", "vllm_omni/diffusion/models/ltx2/ltx2_recipes.py", "PR #6072"]
sources: ["PR #5720", "PR #5853", "PR #5882", "PR #5884", "PR #6486", "PR #6591", vllm_omni/diffusion/cache/base.py, vllm_omni/diffusion/cache/cachedit/backend.py, vllm_omni/diffusion/cache/cachedit/runtime.py, vllm_omni/diffusion/lora/manager.py, vllm_omni/diffusion/models/interface.py, vllm_omni/diffusion/offloader/module_collector.py, vllm_omni/diffusion/offloader/startup.py, vllm_omni/diffusion/registry.py, vllm_omni/diffusion/sched/interface.py, vllm_omni/diffusion/worker/diffusion_model_runner.py, tests/diffusion/cache/test_cache_backends.py, tests/diffusion/cache/test_cache_dit_request_runtime.py, tests/diffusion/models/sana_video/test_cache_offload.py, tests/diffusion/test_diffusion_model_runner.py, tests/diffusion/test_diffusion_scheduler.py, "PR #6070", "vllm_omni/diffusion/models/ltx2/ltx2_recipes.py", "PR #6072", "PR #7078"]
confidence: high
---

Expand Down Expand Up @@ -119,3 +119,10 @@ residency 留在模型 owner。规则入口与其他共享机制仍见 [Diffusio
retry 与 required no-retry;断言 partial backend disable、carrier/lease close、fresh path bypasses
HWR and checkpoint mmap;另注入 unregistration failure,断言 retryable retention、backend drop 和
distributed teardown ordering。^[PR #6486] ^[PR #6591]

## DIFF-2ad — batch-first sampling gate 的 provided 标志必须进入 RequestBatchSamplingParamsKey

- 触发:修改 `RequestBatchSamplingParamsKey`、`_build_sampling_params_key`,或 pipeline 从 batch 首请求读取 `*_provided` 再决定 guidance/image-guidance 行为。
- 强制:凡 pipeline 用 “omitted vs explicit” provided 标志做 batch-local gate 的字段,都必须进入 request-mode 与 step-mode 的 sampling-params key;数值相等但 provided 不同的请求不得合批。当前合同要求 `guidance_scale_2_provided` 与数值 `guidance_scale_2` 一并参与 key。
- 禁止:只比较 auto-fill 后的数值;假设同数值即同语义;只更新 request scheduler 而漏掉 step scheduler;把 H3 等 single-request guard 当作可合批模型的例外模板。
- 验收:用真实 key builder 覆盖 omitted `guidance_scale_2` 与显式同值、数值不同,以及两套 scheduler 的 key 分离;并断言 pipeline 首请求 gate 不会跨不同 provided 语义共享 batch。^[PR #7078]
16 changes: 13 additions & 3 deletions knowledge/repos/vllm-omni/components/diffusion/rules-lora.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: "Diffusion LoRA 规则"
created: 2026-09-02
updated: 2026-09-05
updated: 2026-09-09
type: rule
tags: [vllm-omni, components, diffusion]
sources: ["PR #2783", docs/user_guide/diffusion/lora.md, vllm_omni/config/omni_config.py, vllm_omni/config/stage_config.py, vllm_omni/diffusion/data.py, vllm_omni/diffusion/lora/loader.py, vllm_omni/diffusion/lora/manager.py, vllm_omni/diffusion/lora/layers/base_linear.py, vllm_omni/diffusion/models/qwen_image/pipeline_qwen_image.py, vllm_omni/diffusion/models/wan2_2/pipeline_wan2_2.py, vllm_omni/diffusion/models/wan2_2/pipeline_wan2_2_i2v.py, vllm_omni/diffusion/utils/tf_utils.py, vllm_omni/diffusion/worker/diffusion_worker.py, vllm_omni/engine/async_omni_engine.py, vllm_omni/entrypoints/cli/serve.py, tests/diffusion/lora/test_loader.py, tests/diffusion/lora/test_lora_manager.py, tests/entrypoints/test_async_omni_diffusion_config.py, "PR #5500", "vllm_omni/diffusion/models/ltx2/ltx2_adapter_parser.py", "vllm_omni/diffusion/models/ltx2/ltx2_phase_adapter.py", "PR #6070", "PR #6476", "PR #6550", vllm_omni/diffusion/models/minimax_h3/lora.py, "PR #6268", benchmarks/kernels/benchmark_diffusion_lora_expand.py, tests/diffusion/lora/test_base_linear.py]
sources: ["PR #2783", docs/user_guide/diffusion/lora.md, vllm_omni/config/omni_config.py, vllm_omni/config/stage_config.py, vllm_omni/diffusion/data.py, vllm_omni/diffusion/lora/loader.py, vllm_omni/diffusion/lora/manager.py, vllm_omni/diffusion/lora/layers/base_linear.py, vllm_omni/diffusion/models/qwen_image/pipeline_qwen_image.py, vllm_omni/diffusion/models/wan2_2/pipeline_wan2_2.py, vllm_omni/diffusion/models/wan2_2/pipeline_wan2_2_i2v.py, vllm_omni/diffusion/utils/tf_utils.py, vllm_omni/diffusion/worker/diffusion_worker.py, vllm_omni/engine/async_omni_engine.py, vllm_omni/entrypoints/cli/serve.py, tests/diffusion/lora/test_loader.py, tests/diffusion/lora/test_lora_manager.py, tests/entrypoints/test_async_omni_diffusion_config.py, "PR #5500", "vllm_omni/diffusion/models/ltx2/ltx2_adapter_parser.py", "vllm_omni/diffusion/models/ltx2/ltx2_phase_adapter.py", "PR #6070", "PR #6476", "PR #6550", vllm_omni/diffusion/models/minimax_h3/lora.py, "PR #6268", benchmarks/kernels/benchmark_diffusion_lora_expand.py, tests/diffusion/lora/test_base_linear.py, "PR #7195"]
confidence: high
---

Expand All @@ -22,10 +22,13 @@ confidence: high
| Wan dual transformer、high/low-noise file order、partial load | `DIFF-2n` | `WanLoraLoaderMixin` → `get_transformer_from_pipeline(name)` → `transformer`/`transformer_2` |
| LoRA linear active output slices、inference expand accumulation、autograd/dtype fallback | `DIFF-2ab` | `diffusion/lora/layers/base_linear.py::DiffusionBaseLinearLayerWithLoRA.apply` |
| legacy dynamic adapter hook、packed/stacked binding、DLO sidecar residency、activation rollback | `DIFF-2x` | `DiffusionLoRAManager::{_load_adapter,_bind_adapter_weights,_activate_adapter}` → model pipeline hook |
| PEFT deactivate/suspend、slice-mask resume、同 adapter+scale 再激活不 re-upload | `DIFF-2ae` | `DiffusionLoRAManager` deactivate/activate → `DiffusionBaseLinearLayerWithLoRA::{suspend_lora,resume_lora}` |


| 审查组 | 什么时候触发 | 规则 ID |
|---|---|---|
| `checkpoint-distributed` | diffusion LoRA backend、checkpoint、融合与 transformer mapping | `DIFF-2l`, `DIFF-2m`, `DIFF-2n`, `DIFF-2x` |
| `checkpoint-distributed` | diffusion LoRA backend、checkpoint、融合与 transformer mapping | `DIFF-2l`, `DIFF-2m`, `DIFF-2n`, `DIFF-2x`, `DIFF-2ae` |


## DIFF-2l — backend 是 startup 配置,不是 request 字段

Expand Down Expand Up @@ -127,3 +130,10 @@ confidence: high
- 强制:DLO owns base-weight streaming only。DLO-enabled manager 在 wrapper replacement 时必须要求可用的 resident-buffer protocol、将 request-switchable A/B sidecars 放在 compute device,且 wrapper 因更大 rank 重新分配后必须重放该 device placement;缺 protocol fail closed。非 DLO path 不得获得此 placement side effect。
- 禁止:将已识别的专用 artifact 悄悄退回 generic loader;按名称而非真实绑定确认成功;将一个模型的 packed layout/scale 规则泛化给所有 diffusion 模型;失败后继续声明旧 adapter active;把 sidecars 加入 DLO host shards/collective,或靠每 block transfer 伪装为 DLO support。
- 验收:CPU regressions 覆盖 model hook/fallback、binding completeness、mid-loop failure 后 reset 与旧 adapter retry;DLO mock 覆盖 replacement 的 compute-device placement、缺 resident protocol 的 fail-closed 以及 rank-driven reallocation 后 A/B buffer 仍在该 device。模型 integration 另覆盖 artifact identity、target/shape rejection、packed QKV/FFN binding 和实际 request sampling contract。^[PR #6476] ^[PR #6550]

## DIFF-2ae — PEFT 去激活必须 suspend 切片掩码并保留 uploaded 权重

- 触发:修改 `DiffusionLoRAManager` 的 deactivate/activate、`DiffusionBaseLinearLayerWithLoRA` 的 active-slice mask,或混合 base/adapted 请求切换路径。
- 强制:deactivate 只 `suspend_lora()`(保存并清零 `_diffusion_lora_active_slices`),保留 stacked A/B;同一 adapter_id 与同一 rounded scale 再激活时 `resume_lora()` 复原掩码,禁止整层 `reset_lora`+rebind。`_suspended_adapter_id` 在真实 reset、rank 驱动的 `create_lora_weights`、新 bind 开始、以及 remove 被 suspend 的 adapter 时必须清空;不同 adapter 或不同 scale 仍走完整 rebind。
- 禁止:把 suspend 当成可丢弃的 upload;resume 覆盖后注册的新 layer 的 inactive mask;在失败 bind 后仍保留旧 suspended 身份。
- 验收:CPU 覆盖 suspend/resume、同 id+scale 不 re-upload、不同 scale/id 走 rebind、remove suspended 触发 reset、中途失败后 inactive,以及 add_adapter 后新 layer 不被错误 resume。^[PR #7195]
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: "Diffusion paged cache 与系统运行时规则"
created: 2026-09-03
updated: 2026-09-05
updated: 2026-09-09
type: rule
tags: [vllm-omni, components, diffusion]
sources: ["PR #5255", "PR #5344", "PR #5543", "PR #5838", "PR #6094", "PR #6102", "PR #6385", "PR #6340", "PR #6714", "PR #6814", "PR #6563", "PR #5716", "PR #6786", vllm_omni/diffusion/attention/, vllm_omni/diffusion/attention/parallel/ulysses.py, vllm_omni/diffusion/attention/parallel/ring_kernels.py, vllm_omni/diffusion/diffusion_kv/, vllm_omni/diffusion/distributed/cfg_parallel.py, vllm_omni/diffusion/distributed/parallel_state.py, vllm_omni/diffusion/worker/diffusion_model_runner.py, vllm_omni/platforms/interface.py, vllm_omni/platforms/npu/platform.py, tests/diffusion/diffusion_kv/, tests/diffusion/distributed/test_cfg_parallel.py, tests/diffusion/attention/test_piecewise_attn.py, tests/diffusion/attention/test_ulysses_uaa.py, "PR #5491", "PR #5194", "vllm_omni/diffusion/data.py", "vllm_omni/diffusion/utils/hf_utils.py"]
sources: ["PR #5255", "PR #5344", "PR #5543", "PR #5838", "PR #6094", "PR #6102", "PR #6385", "PR #6340", "PR #6714", "PR #6814", "PR #6563", "PR #5716", "PR #6786", vllm_omni/diffusion/attention/, vllm_omni/diffusion/attention/parallel/ulysses.py, vllm_omni/diffusion/attention/parallel/ring_kernels.py, vllm_omni/diffusion/diffusion_kv/, vllm_omni/diffusion/distributed/cfg_parallel.py, vllm_omni/diffusion/distributed/parallel_state.py, vllm_omni/diffusion/worker/diffusion_model_runner.py, vllm_omni/platforms/interface.py, vllm_omni/platforms/npu/platform.py, tests/diffusion/diffusion_kv/, tests/diffusion/distributed/test_cfg_parallel.py, tests/diffusion/attention/test_piecewise_attn.py, tests/diffusion/attention/test_ulysses_uaa.py, "PR #5491", "PR #5194", "vllm_omni/diffusion/data.py", "vllm_omni/diffusion/utils/hf_utils.py", "PR #7041"]
confidence: high
---

Expand Down Expand Up @@ -205,3 +205,10 @@ confidence: high
- 强制:Scheduler 只拥有 logical block allocation、generation 和 `DiffusionKVMetadata`;每个 Worker rank 消费自己的 `KVCacheConfig`,拥有物理 KV tensors、native `BlockTables` 和 request/sequence/context rows。安装 metadata 前必须校验 group/count/range/capacity,并以 staged append、apply 和失败 rollback 原子发布;重复 generation 幂等,冲突或过期快照拒绝,BlockTables 变更使已准备的 attention batch 失效,终止时先清 Worker rows 而不释放 Scheduler blocks。request-mode runner 负责从 snapshot 构造并激活 prefill/denoise row metadata;model 只提供 Q/K/V layout 与 `full_attn_spans`。
- 禁止:Worker 自行分配或释放 Scheduler-owned logical blocks;把部分 row 安装、native append/apply 异常或 stale snapshot 当作成功;从 padding 猜异构 prefix 的逻辑布局;在清理或 wake refresh 后继续复用旧的 native metadata/buffer view。
- 验收:CPU/mock 覆盖 rank-local config、请求/上下文 row mapping、重复/过期/冲突 generation、非法 block 与容量、append/apply rollback、幂等清理和 wake refresh;GPU/NPU 另验证非连续 BlockTables、32Q/8KV GQA、异构 prefix 与 piecewise attention。Hunyuan paged request execution 不证明 `denoise_step`、cross-request prefix reuse、AR KV、negative CFG、Ring 或 AllGather。^[PR #6102] ^[PR #6563]

## DIFF-4ac — 声明为 str 的 diffusion cache_backend 必须在 kwargs 边界把显式 None 规范成 none

- 触发:修改 `normalize_omni_diffusion_kwargs`、`OmniDiffusionConfig.cache_backend`,或 pipeline/runner 对 cache backend 的 admission 校验。
- 强制:字段声明为 `str` 且 canonical “无 cache” sentinel 为 `"none"` 时,key 缺失才允许走 env/`"none"` 默认;调用方传入显式 `None`(如 CLI `default=None`)必须在 kwargs 边界规范成 `"none"`,且不得回落到 `DIFFUSION_CACHE_BACKEND` 查找。下游校验可把残留 `None` 当 `"none"` 作 defense-in-depth,但不能代替边界规范化。
- 禁止:只在 key 缺失时默认、让显式 `None` 泄漏成 typed `str` 字段上的 `None`;把 `"none"` 与 env 覆盖混为一谈;把某一 pipeline 的 `None` 特判当成唯一修复点。
- 验收:`from_kwargs`/`create_default_diffusion` 对 `cache_backend=None` 断言最终为 `"none"`;至少一个严格 `("none", …)` 校验路径覆盖 `None` 与 `"none"` 均可通过,未知 backend 仍拒绝。^[PR #7041]
Loading
Loading