Skip to content

knowledge(vllm-omni): 1 change(s) from intake #415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tzhouam merged 1 commit into from
Oct 25, 2026
Merged

knowledge(vllm-omni): 1 change(s) from intake #415

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions knowledge/repos/vllm-omni/components/serving/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,4 @@ sources: []
- [vLLM-Omni 服务：Gradio 示例 PCM 音频转换规则](rules-gradio-pcm.md)
- [vLLM-Omni Serving：Seed-TTS 基准与准确率规则](rules-seed-tts.md)
- [vLLM-Omni Stage-CLI 测试规则](rules-stage-cli-tests.md)
- [vLLM-Omni Serving 规则 — MiniCPM-o 注意力后端对比](rules-minicpmo-attn-compare.md)
19 changes: 19 additions & 0 deletions knowledge/repos/vllm-omni/components/serving/rules-minicpmo-attn-compare.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: "vLLM-Omni Serving 规则 — MiniCPM-o 注意力后端对比"
created: 2026-10-25
updated: 2026-10-25
type: rule
tags: [vllm-omni, components, serving]
sources: ["PR #9344"]
---

# vLLM-Omni Serving 规则 — MiniCPM-o 注意力后端对比

## VLLM-OMNI-PR9344-MINICPM-ATTN-COMPARE — MiniCPM-o 注意力后端对比：出货 profile 不锁定 CUDA Thinker 后端，实验走仅改 stage 0 的 overlay

- 触发：改动 `docs/serving/realtime_duplex_api.md` 的 `#### Compare MiniCPM-o attention backends` 小节，或 MiniCPM-o 部署 profile（如 `vllm_omni/deploy/minicpmo_4_5.yaml` 及其变体）中 CUDA stage 的 `attention_backend` 取值，或 `tests/model_executor/models/minicpmo_4_5/test_pipeline.py` 对部署 profile 的结构断言；也覆盖以采样 transcript 差异主张注意力后端缺陷、或以 `response.done` 成功主张识别质量的评审证据。
Comment on lines +12 to +14

@chatgpt-codex-connector chatgpt-codex-connector Bot Oct 25, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Move the MiniCPM-o-specific rule to its model owner

This rule applies exclusively to MiniCPM-o deployment profiles, backend comparisons, and model-specific tests, but its sole index entry places it under the shared serving component. The existing owner is models/minicpm-o-4-5/, where rules-duplex.md already maintains shipping-profile attention and configuration constraints. Keeping this page under serving splits that model contract across unrelated owners and leaves it absent from the model's own knowledge entrypoints. Move the rule to that model owner and update its nearest index. Root AGENTS.md explicitly requires that “model-specific rules stay with that model owner.”

AGENTS.md reference: AGENTS.md:L12-L14

Useful? React with 👍 / 👎.

- 强制：出货 MiniCPM-o profile 保持把 CUDA Thinker 注意力后端选择交给 vLLM（文档明示该语义）。比较后端必须用只改 stage 0 的部署 overlay：`base_config` 指向 `vllm_omni/deploy/minicpmo_4_5.yaml`，在 `platforms.cuda.stages` 下为 `stage_id: 0` 设 `attention_backend: TRITON_ATTN`，经 `--deploy-config` 传入；overlay 保留 Talker 的后端选择与 profile 的 graph 设置；turn-mode profile 同样继承该基础部署，改基础 profile 会同时影响两种服务模式；启动后须逐 stage 核对启动日志中的实际后端与 graph 模式。质量对比须固定源与模型 revision、参考音频、输入音视频、帧率、节奏、窗口设置与采样参数，比对多个匹配 seed，并在全新服务器启动后重复会话，按场景与提问时间戳评估回复（较晚的回答可能描述的是模型作答期间已变化的场景）。证据语义：成功的 `response.done` 只证明协议完成，不证明识别正确或句子完整；图启用与 eager 服务须分别验证并记录实测拓扑；单个采样 transcript 的变化既不建立注意力后端缺陷，也不建立普遍质量提升；该 overlay 是对比工具，不是识别错误或 post-commit drain 超时的已确认修复。
- 禁止：在出货 profile 中固定 CUDA stage 0 的 `attention_backend`，或恢复要求 Triton 为默认的测试断言；把服务器级 `--attention-backend` 覆盖当作 Thinker-only 对比（它同时作用于 Talker，属于不同实验）；以变量不匹配的 transcript 对比或仅凭 `response.done` 成功宣称后端缺陷或质量结论；删除或改写文档小节中的 overlay 示例、对比规程与证据限定；移除 `test_pipeline.py` 中 mypy 要求的 `expected_graph_config: dict[str, object]` 类型注解。
- 验收：`docs/serving/realtime_duplex_api.md` 保留该小节（stage-0 overlay YAML 示例、`--deploy-config` 用法、Talker/graph 保留说明、与 `--attention-backend` 的区别、逐 stage 启动日志核对、固定变量与多 seed/重启规程、时间戳评估及 `response.done` 语义限定）；出货 MiniCPM-o profile 的 CUDA stage 0 未固定 `attention_backend`；`pytest tests/model_executor/models/minicpmo_4_5/test_pipeline.py` 通过且 mypy/pre-commit 通过。^[PR #9344]

<!-- kb:rule status=active since=v0.30.0 -->
Loading