From f2c74e3bd0ae9747856d608d3d4981216aafa78e Mon Sep 17 00:00:00 2001 From: tzhouam Date: Tue, 13 Oct 2026 18:11:08 +0000 Subject: [PATCH] knowledge(vllm-omni): 1 change(s) from refine --- .../components/model-executor/_index.md | 1 + .../rules-orchestration-loop.md | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+) create mode 100644 knowledge/repos/vllm-omni/components/model-executor/rules-orchestration-loop.md diff --git a/knowledge/repos/vllm-omni/components/model-executor/_index.md b/knowledge/repos/vllm-omni/components/model-executor/_index.md index 674e7139..895a44d0 100644 --- a/knowledge/repos/vllm-omni/components/model-executor/_index.md +++ b/knowledge/repos/vllm-omni/components/model-executor/_index.md @@ -63,3 +63,4 @@ sources: [] - [Model Executor — NPU KV cache 布局规则](rules-npu-kv-layout.md) - [Model Executor — Dynamo/cudagraph compile guard rules](rules-compile-guard.md) - [Model Executor — INC/AutoRound 量化配置规则](rules-inc-autoround.md) +- [Model Executor — Orchestration Loop Rules](rules-orchestration-loop.md) diff --git a/knowledge/repos/vllm-omni/components/model-executor/rules-orchestration-loop.md b/knowledge/repos/vllm-omni/components/model-executor/rules-orchestration-loop.md new file mode 100644 index 00000000..269378f5 --- /dev/null +++ b/knowledge/repos/vllm-omni/components/model-executor/rules-orchestration-loop.md @@ -0,0 +1,19 @@ +--- +title: "Model Executor — Orchestration Loop Rules" +created: 2026-10-13 +updated: 2026-10-13 +type: rule +tags: [vllm-omni, components, model-executor] +sources: ["PR #8675"] +--- + +# Model Executor — Orchestration Loop Rules + +## VLLM-OMNI-PR8675-ORCH-LOOP-DEFAULT — 双 orchestration loop 必须共用同一 StagePool 读路径，成员变更唤醒 dispatcher + +- 触发：修改 `vllm_omni/engine/orchestrator.py` 的 stage-output loop（event-driven 或 legacy 1 ms poll）、`vllm_omni/engine/stage_pool.py` 的读取/成员接口（`add_client`、`remove_client`、`mark_replica_unavailable`、`evict_replica`），或 `VLLM_OMNI_EVENT_DRIVEN_ORCH` 的默认值与解析。 +- 强制：env 未设置即对所有 pipeline 启用 event-driven loop；`1`/`true`/`yes`/`on` 保持启用，其余值（含 `0`/`false`/`no`/`off` 及任何未识别值）选择 legacy poll；值做 strip 与大小写归一化，在 `Orchestrator` 构造时解析，不再有 per-pipeline 默认。两条 loop 的 stage 输出读取必须收敛到同一 `StagePool.read_llm_raw_output`（原 `_poll_stage_raw`），保证 dict pooling 的 `pooling_output_payload` 等在两条路径上一致 rehydrate。pool 成员变更必须唤醒 dispatcher 以触发 reader reconcile，使运行时新挂载的 replica 无需等 0.5 s tick 才被首次 drain；reconcile 在 dispatcher 内执行，可能排在于途 output-routing await 之后——唤醒是及时的，但不保证即时完成，0.5 s tick 保留作 backstop。serving-side final-output loop 的唤醒独立于该 flag（#8703 之后），本规则只约束 orchestrator stage-output loop。 +- 禁止：重新引入 per-pipeline 默认管道（`_event_driven_orch_default_for_pipeline` 或 `event_driven_orch_default` ctor 参数）；在任一 loop 里绕过 pool 读 helper 直接 `client.get_output_async()`；用手工挑选的 parity 清单替代把 orchestrator scenario 测试在两条 loop 上参数化；把成员变更唤醒表述为保证即时 reconcile 的同步语义；把默认切换表述为吞吐/延迟改善——其证据边界是空闲 CPU 下降，吞吐与延迟预期不变。 +- 验收：单测在 env 未设置与 `VLLM_OMNI_EVENT_DRIVEN_ORCH=0` 两种条件下运行；`tests/engine/test_orchestrator.py` 与 `test_orchestrator_error_handling.py` 通过参数化 fixture 覆盖两条 loop；新增读取或成员路径时断言 pooling payload rehydration 在两条 loop 行为一致，且成员变更后 reader 集合最终与 pool 成员一致（允许等待在途 output-routing 完成，不要求瞬时）；启动日志仍记录所选 loop 模式与 reader/poller 计数。^[PR #8675] + +