Repository navigation
knowledge(vllm-omni): 1 change(s) from intake - #397
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8e5dcc9afb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| ## VLLM-OMNI-PR9284-SEEDTTS-VERBATIM — Seed-TTS 默认系统提示须令模型逐字朗读：问题/指令是待读文本，WER 门槛收紧至 0.03 并配文本级回归 | ||
|
|
||
| - 触发：改动 `vllm_omni/benchmarks/data_modules/seed_tts_dataset.py` 中的 `SEED_TTS_DEFAULT_OMNI_SYSTEM_PROMPT`，或 `tests/e2e/accuracy/qwen3_omni/test_qwen3_omni.py` 中 Seed-TTS 门槛常量（`_MAX_SEED_TTS_MEAN_WER` 等）与逐字朗读回归测试，或 `docs/cli/bench/serve.md` 的 "Seed-TTS reference speech as real audio input" 小节。 |
There was a problem hiding this comment.
For future changes limited to these paths, the owner map assigns vllm_omni/benchmarks/ to the benchmark owner (adapters/vllm_omni/release_baseline.yaml:28-30), while the WER threshold and regression are explicitly Qwen3-Omni-specific; indexing the combined rule only under serving means the relevant benchmark/model review routes will not load this guard. Move the shared dataset/CLI contract to the benchmark rules and the model-specific threshold/test contract to the Qwen-Omni rules, updating both nearest indexes.
AGENTS.md reference: AGENTS.md:L12-L17
Useful? React with 👍 / 👎.
| ## VLLM-OMNI-PR9284-SEEDTTS-VERBATIM — Seed-TTS 默认系统提示须令模型逐字朗读：问题/指令是待读文本，WER 门槛收紧至 0.03 并配文本级回归 | ||
|
|
||
| - 触发：改动 `vllm_omni/benchmarks/data_modules/seed_tts_dataset.py` 中的 `SEED_TTS_DEFAULT_OMNI_SYSTEM_PROMPT`，或 `tests/e2e/accuracy/qwen3_omni/test_qwen3_omni.py` 中 Seed-TTS 门槛常量（`_MAX_SEED_TTS_MEAN_WER` 等）与逐字朗读回归测试，或 `docs/cli/bench/serve.md` 的 "Seed-TTS reference speech as real audio input" 小节。 | ||
| - 强制：默认系统提示保留 Qwen 身份前缀，并明确要求把整条 user 消息当作待朗读文本逐字读出：问题与指令是要说的内容而非要回答或执行的任务，文本响应必须与 user 消息完全一致，不加引言、解释、回答或其他词语，不改写、翻译或续写；保留 `--seed-tts-system-prompt` 覆盖入口。依据 PR #9284：旧提示未区分"读问题"与"答问题"，nightly build #17816 中 1088 条英文 mean WER 0.224804（中位数 0）仍通过旧 0.35 门槛，`I feel sick, what should I do?` 被答成 67 秒建议音频；同 1088 条目标上旧提示 mean WER 0.267329（74 条 WER>1），逐字提示 0.028078（0 条 WER>1），据此 `_MAX_SEED_TTS_MEAN_WER` 收紧为 0.03，数据集目标与全音频 WER 计算保持不变。新增 `test_qwen3_omni_seed_tts_reads_questions_verbatim`：以默认提示直连 chat completions（temperature 0、max_tokens 128、仅 text 模态），断言 `finish_reason == "stop"` 且 `message.content == target`，参数化覆盖 nightly #17816 的两条已知失败目标。文档须说明：发送 `ref_audio`/`ref_text` 字段不等于声音克隆——克隆需要模型与端点支持这些字段，对 Qwen3-Omni chat 不生效；`--seed-tts-wer-save-items` 用于排查内容错误时在结果 JSON 保留每条目标、ASR 转写与 WER（准确率 CI 不启用逐项保存）；WER 的对比对象是数据集目标而非模型生成的回复。 |
There was a problem hiding this comment.
When this page is injected into later reviews, the nightly build number, exact sample counts and measured values, 67-second anecdote, and two historical failures become permanent rule text even though the intake contract classifies one-time counts and “why it happened then” as non-rule material (doc/knowledge/contributing/incidents.md:46). Keep the executable prompt, threshold, and regression requirements, but leave this historical evidence in the source PR so the rule remains concise and reusable.
AGENTS.md reference: AGENTS.md:L12-L17
Useful? React with 👍 / 👎.
Generated automatically by the Copilot knowledge service (changeset
vllm-omni-intake-ef0b78c1052c).repos/vllm-omni/components/serving/rules-seed-tts.mdGenerator:
zcode:GLM-5.3· judge:codex:gpt-6-sol:medium.The publisher merges it only if its local gate passes on the exact merge result.