acceptance: Qwen3.8 FP8 vs vLLM 0.32.0 — ≥25% TTFT or TPOT gain, ≤5% regression

Current owner amendment — 25% latency improvement with a 5% regression guard (2026-10-24)

The owner explicitly changed the goal in the active Codex conversation: at least one of TTFT or TPOT must be at least 25% lower than matched vLLM 0.32.0, and neither metric may be more than 5% higher. With candidate/baseline latency ratios, at least one must be ≤0.75, and both must be ≤1.05. This replaces the previous prospective 1.03 parity target; old measurements, frozen validators and verdicts remain unchanged.

The already approved protocol remains: only8192 input tokens, 2048 actual generated tokens, concurrency1, the same official client/model/tokenizer and physical H20, ten fixed inputs × three paired repetitions, original order/restart/warmup policy, and 10,000 input-cluster bootstrap draws with seed670032. For complete valid evidence, retain the original median and one-sided95% upper-bound decision structure with the new limits:

PASS: each metric's median and upper bound are ≤1.05, and at least one same metric's median and upper bound are both ≤0.75.
FAIL: either median exceeds1.05, or neither median reaches0.75.
INCONCLUSIVE: median conditions pass but confidence-bound conditions do not, or required evidence is incomplete/invalid.

This is a prospective logical gate change, not a post-result choice of inputs, metric, bootstrap method or scorer. Each metric keeps the approved marginal95% calculation; no new joint-confidence claim is introduced. A separately versioned reducer/checklist will be independently reviewed and frozen before further candidate experiments. The old1.03 formal-v2 preparation was not staged/submitted and has no GPU reservation; it is superseded for activation.

Task quality, original ten8K retrieval cases, the existing44GPU-hour phased cap, one-plane-per-step discipline, independent review and autonomous PR integration authorization remain unchanged. Final campaign acceptance and Issue closure remain with the owner. PR26 and PR27 have merged; their historical bounded evidence remains valid within its stated scope and does not establish this new objective.

Current owner amendment — 8K-only acceptance (2026-10-23)

The owner explicitly confirmed in the active Codex conversation: both retrieval and formal TTFT/TPOT acceptance now cover only 8192 input tokens. Both arms still use the same pinned official vLLM 0.32.0 streaming HTTP client, concurrency 1, the same authenticated FP8 model/tokenizer and physical H20, 10 fixed inputs, 2048 actual output tokens ignoring EOS, and three paired alternating repetitions. Required candidate/baseline TTFT and TPOT ratios remain ≤ 1.03. This prospectively supersedes the 64K/128K required lengths in earlier scope entries below; historical inputs, observations, failures and verdicts remain unchanged.

Retrieval uses the existing fixed 8K inputs and scorer. Its matched baseline is 10/10 complete with all four positions correct; candidate complete and each-position counts must not regress, so all ten 8K cases must match. No input selection or post-result scorer change is authorized. Prior vLLM 64K/128K short-output retrieval succeeded; the current native attention path has a static long-context memory limitation, and the complete 2048-output formal matrix was not validated. This scope reduction is the owner's decision, not a retroactive failure or PASS for those observations.

GSM8K/HumanEval+/MMLU quality scope and per-dataset gates are unchanged. The 44 GPU-hour phased total cap (readiness 2 / baseline 4 / search 8 / finalization 30) was already authorized; remaining cost estimates will be revised for 8K without automatically consuming the cap. Autonomous independently reviewed PR integration is authorized; final campaign acceptance/closure remains with the owner.

The owner has now approved the 8K statistical rule (decision record): ten fixed inputs × three paired repetitions; for TTFT and TPOT separately, use the median of 30 candidate/baseline ratios and 10,000 input-cluster bootstrap draws (seed 670032). Median and one-sided 95% upper bound must both be ≤1.03; median above1.03 is FAIL, while only the upper bound above1.03 is INCONCLUSIVE. A versioned 8K reducer is being independently reviewed. Complete evaluator qualification and command/runtime/validator freeze remain required; this approval does not relabel prior diagnostics or alone award campaign PASS. Earlier pending-statistics wording below is historical.

Authority, parent and lifecycle

Refs #53. The requesting owner explicitly authorized this new acceptance Issue and closure of #57 as superseded, without rewriting its body, gates, evidence or historical verdicts. Acting contributor: Codex, implementing the owner's decisions from the current requirements discussion. Final campaign acceptance and Issue closure remain with the requesting owner.

Campaign ID: qwen38-h20-fp8-single-request-vllm032.
Status: INCONCLUSIVE — requirements accepted; execution protocol and acceptance freeze incomplete.
This is a new contract, not a retrospective relaxation or PASS for #57.

Accepted scope and comparison identity

Independent nano-vllm-derived, text-only engine for the same official Qwen/Qwen3.8-27B-FP8 checkpoint as the baseline.
Baseline server: vLLM 0.32.0, default optimization configuration, with only documented settings necessary to run this workload; no baseline configuration search.
Both arms: same authenticated weights/tokenizer, same physical allocated H20-family GPU, greedy generation and no cross-request prefix cache. Record actual SKU, model limits, runtime, device conditions and every nondefault setting.
Use the same pinned vLLM 0.32.0 benchmark client against both streaming HTTP services. Record the actual benchmark timing/token-count semantics and raw per-request outputs; do not substitute an engine-specific timer for acceptance.
Baseline source: <baseline_sha>; candidate: <candidate_sha>; benchmark, model/tokenizer, evaluator, dataset and environment identities remain to be resolved and frozen. A package version alone is not a complete source identity.

Performance protocol accepted by the owner

Concurrency 1 only; multi-concurrency and production trace claims are out of scope.
Exact input lengths 8192 / 65536 / 131072 tokens, including any rendered template. These replace all earlier 4K/8K/64K/256K proposals.
Fixed synthetic token workloads: 10 distinct fixed inputs per length, identical actual input token sequences in both arms, with generation rules, special-token handling and hashes frozen before execution.
Exactly 2048 generated tokens per performance request, ignoring early EOS. Verify real generated work rather than relying on requested length or decoded-text re-tokenization alone.
Three paired repetitions, alternating engine order; startup, compilation and warmup excluded from timed requests. Freeze exact order, restart/warmup policy and measurement commands before baseline measurements.
Report TTFT, TPOT, complete request latency and output tokens/s for each length, all trials and dispersion. No cross-length averaging may hide a failing required length.
Objective (owner amendment, 2026-10-21): at each required input length, candidate/baseline TTFT and TPOT ratios must each be at most 1.03 (up to 3% slower). The owner explicitly selected this parity margin in the current execution discussion; the prior improve-only objective is superseded prospectively before any baseline/candidate comparison. Aggregation, variability handling and any guardrails remain pending baseline measurement and owner confirmation before optimization candidate search. Task-quality gates are unchanged.
Baseline readiness/feasibility work is separate from an accepted performance comparison. No performance PASS is possible while gates or evidence are missing.

Task-quality protocol accepted by the owner

Use a fixed benchmark/evaluation protocol for both engines: thinking disabled, greedy, maximum 2048 output tokens, natural EOS enabled. Preserve capped outputs, parsing failures, errors and all attempts; no post-result exclusions or prompt/scorer changes.

Task	Iteration screening	Final candidate	Prompt and score
GSM8K	Fixed 100 cases	Full official test split	0-shot, brief reasoning and final answer; frozen answer extraction and correctness
HumanEval+	Fixed 32 cases	Full evaluation set	0-shot code generation; one greedy sample, isolated execution, pass@1
MMLU	Fixed 100 cases	Fixed subject-stratified 1000-case subset	Fixed 5-shot examples, generate option letter; frozen parsing and accuracy

For each final dataset separately, candidate score must be at least the matched vLLM score minus 0.01 absolute (one percentage point). No averaging between datasets, rounding rescue or statistical-confidence claim. Compare exact correct/pass counts and denominators. Screening subsets are for early rejection/diagnosis and cannot award final quality PASS.

Freeze dataset revisions, splits, selected IDs, sampling seed, prompts/chat template, demonstration examples, scoring/extraction, sandbox and execution limits before evaluations. Maintain explicit screening/final overlap: the final sets include disclosed screening data and must not be described as wholly unseen held-out data. The rest of the final evaluation is excluded from iterative tuning. Baseline quality outputs may be reused only under matching model/runtime/config/data/scorer identities with documented validity.

Numerical diagnostics and long context

Task quality is the primary new model-quality acceptance basis.
Logit NRMSE and greedy top-1 agreement are diagnostics in this new contract; acceptance: Qwen3.8-27B-FP8 single-H20 text engine and HTTP service — phase 1 #57's original mandatory numerical gates and failed results remain unchanged and attributable to acceptance: Qwen3.8-27B-FP8 single-H20 text engine and HTTP service — phase 1 #57.
A failed old reference self-check is not repaired by this Issue and cannot be represented as a trusted numerical oracle. Identify the reference and its limitations for every new diagnostic.
Output validity, authenticated model execution, request-state integrity and cleanup remain required; diagnostic status does not excuse broken execution.
Add numerical alignment probes and fixed multi-position retrieval at 8K/64K/128K. The retrieval construction, positions, samples, score and gates must be separately specified; owner confirms retrieval gates after baseline and before search.
Ordinary GSM8K/HumanEval+/MMLU results do not establish long-context capability.

Allowed optimization changes

Only model-semantics-preserving operator, layout, graph-execution and scheduling improvements. No new quantization format, KV compression, context truncation or speculative decoding. Preserve official FP8 weights, configured computation semantics and declared state precision; no hidden dense fallback represented as native FP8 compute.
Declare one hypothesis and one SIAS plane per candidate step. Keep evaluator/protocol changes separate from model optimization and version them prospectively.

Budget, readiness and stop conditions

Owner selected bounded feasibility and runtime estimation first, followed by an itemized GPU budget for baseline, search and final acceptance for explicit owner confirmation. No total GPU-hour budget is yet authorized by this Issue.
Before any probe, record its bounded workload, allocation, timeouts, cleanup and source/input identities. Keep failures, timeouts and unsupported lengths; never silently shorten input/output or drop a length.
Reserve finalization using measured validation cost. Do not begin performance candidate search until all gates, validators and the search/finalization budget are frozen. No GPU work was performed when opening this Issue.

Required readiness includes: benchmark API compatibility (including ignore_eos and streaming usage/token accounting), exact synthetic prompt token preservation, input-plus-output context/memory feasibility, fixed no-thinking templates, and deterministic task evaluation.

Freeze checklist

Evidence and publication

Draft artifact location: evidence/qwen38-h20-fp8-single-request-vllm032/, with plan/, results/, patch/, validator/.
Initial companion files are local working-tree drafts, not yet published Git blobs; do not treat their paths as remote evidence links or as a freeze manifest. Record committed artifact links/hashes when actually available.
Keep numerical thresholds scoped to this acceptance Issue and its eventual frozen evidence; do not copy them into shared workflows or skills. Retain all rejected trials and final source bindings. Public artifacts use placeholders for private paths/endpoints and omit credentials. Raw operational data remains in controlled storage.
Verdict remains INCONCLUSIVE until all required evidence is complete. Closing #57 is administrative supersession, not evidence of successful acceptance.

Current execution delegation (2026-10-21)

The owner requested continued optimization under InferRSI contracts, authorized one four-GPU Slurm allocation (including distributing long quality evaluation across its assigned devices), and requested recording every successful and failed optimization in InferRSI knowledge. The runtime model location is supplied privately and represented in shared artifacts as <MODEL_PATH>.

Execution team: accountable parent Codex; Agent A benchmark/baseline (GPT-6.1 Sol, high; timing review at xhigh), Agent B quality/evaluation (GPT-6.1 Sol, xhigh), Agent C dedicated engine (GPT-6.1 Sol, xhigh). Four agents total, including the parent. No closure or merge authority is added. Source and environment readiness may proceed; bounded feasibility protocol must precede GPU probes, with itemized search/finalization budgets and complete freeze still required before candidate search. The 1.03 parity margin is owner-approved; aggregation/retrieval gates and total optimization budget are not yet frozen.

Readiness checkpoint (2026-10-21)

No acceptance freeze or performance claim is activated. Baseline source/client release identity: 5df63a7203cdb9328d824a27b682e77c7b9928be; official x86_64 wheel SHA-256: 5e2ecdc5b0baf72921c288486417ca82778456a448e4fe60b62ee43c0dfc8611. Current native HTTP-readiness source: 78a9c235e14d56872283c72b0f4b9dcbcb760eda (local Git objects; publication not verified). The HTTP changes preserve the model implementation and were independently reviewed with 94 synthetic CPU tests passing.

CPU-only Slurm readiness: dependency resolution 1501 completed/0 (45 s), model-byte intake 1502 completed/0 (38 s), isolated baseline install 1505 completed/0 (427 s), fixed token fixture generation 1515 completed/0. The model intake verified 74 files / 30,889,968,033 bytes, all 66 indexed weight shards corroborated. Model/tokenizer source revision is 017b9c7af6b5689d5dd426a76e0bc077eb5ca20a; report SHA-256 05b27b016c3b80c4263f34103fc775021c88d878d8944451cd57ccad8d186fcc. The fixed fixture has 10 inputs at each 32/8192/65536/131072 length, SHA-256 4f4256cc4edc9a23f55f66151738a1ecef07eb752c695e7b7fbdfa17e01d6aa9. Baseline installation checks dependencies, three benchmark files byte-for-byte against the source archive, and CLI help; GPU/model qualification is still pending.

Quality CPU preparation 1517 exited 1 during JSONL prompt re-reading after dataset/cohort preparation. Before/after source, intake, runtime and tokenizer guards remained identical. Preserve this attempt; a prospective versioned fix is being checked before retry. No GPU, generated-code execution or quality scoring occurred.

Initial GPU readiness will use one GPU for at most one hour, within the owner's permission to use up to four when useful. Both services run sequentially on the same assigned device. Baseline probes: fixed 32-input/8-output framing and cost requests, then 8192/32, 65536/8, 131072/8 cost probes. Native: 32/8 only, with long inputs explicitly not run because inspected source materializes quadratic full attention. Startup is bounded at 1200 s per arm; overall readiness at 3000 s with owned-process cleanup and input-integrity checks. Reduced-output readiness requests cannot substitute for the required 2048-token acceptance matrix.

Raw operational records stay in private task storage; this Issue deliberately omits runtime paths and device identities. The local draft evidence directory is not yet a published Git bundle. Metric aggregation/variability, retrieval gates and itemized baseline/search/finalization budgets remain pending before candidate optimization search. Verdict: INCONCLUSIVE.

轮次总账（由 InferRSI 分析维护，每轮追加一行）

规则：每个代码提交必须能指回一个 issue（这里只挂 #67，#57 已关闭授权关闭）；不能 push 的分支标"本地分支未 push"；run 记录里的 commit sha 必须与本行完全一致，对不上即视为未记录；缺项写"无"，不事后回填猜测。

轮次	变更摘要	issue	PR	commit sha	运行记录	知识条目	判定
R0 基线	eager 逐算子下发（对照起点，非提交）	#67	无	无	`qwen38-h20-fp8-single-request`（v7 卡，UUID 未记录）	`store:case.qwen38-launch-and-graph-gains-cand`	INCONCLUSIVE（点观测）
R1	融合 RMSNorm、GQA 分组 matmul、`item()` 移出步循环	#67	无	未记录	同上	同上	INCONCLUSIVE（点观测，收益保留）
R2	DeltaNet/KV/采样缓冲预分配 + 整步 CUDA Graph capture	#67	无	未记录	同上	同上 + `store:case.qwen38-graph-state-not-accumulating-cand`	INCONCLUSIVE（耗时有效，正确性签核作废）
v5	服务生命周期被 TIME_WAIT 污染	#67	无	未记录	无（作废，数值不入账）	`store:case.qwen38-v5-timewait-retracted-cand`	FAIL（作废）
v8	图桶边界未覆盖 8K，静默退回 eager	#67	无	未记录	无（作废）	`store:case.qwen38-v8-bucket-boundary-silent-eager-cand`	FAIL（绑定 bug）
v11	按桶缓存 CUDA Graph + 就地写 DeltaNet 状态缓冲	#67	待 `5c7700d` 所属仓确认 PR 流程	`5c7700d`（白名单远端 checkout，非上游仓）	待 benchmark 执行者填（Slurm 作业号、卡 UUID）	`store:case.qwen38-graph-state-not-accumulating-cand` + `store:case.qwen38-launch-and-graph-gains-cand`	PASS（tiny fixture bitwise）；全模型待实测
FP8 基线	vLLM 0.32.0 + FP8 实测（无代码变更）	#67	不适用	不适用	`fp8-baseline`（卡 UUID `667b4c91`，n=5/n=3，客户端 `vllm bench serve`）	`store:case.qwen38-fp8-baseline-measured-not-estimated-cand`	INCONCLUSIVE（点估计，未做功效核算）
客户端口径复核	`vllm bench serve` 与 `common-client.py` 的计时差异定性	#67	不适用	不适用	同上（同源数据复核，无新测量）	待并入 FP8 基线条目	不翻转 1.49×；8K 两 run 差 4.7% 属 run 差异，需在冻结 ABBA 中归并

当前缺项（不得补猜）

R0/R1/R2/v5/v8 的 commit sha 与 Slurm 作业号：历史上未捕获，需优化师/执行者确认是否可从远端 checkout 的 reflog 或作业日志恢复；恢复不了就永久写"未记录"。
5c7700d 的 PR 状态：该仓库是否走 PR 流程待优化师确认。
卡 UUID：R0–R2 使用的 v7 卡 UUID 未记录。
v11 的实测记录（回归 max_abs、TPOT ABBA、8K TTFT）：尚未产出。

目标线（据此更新）

中位线（vllm bench serve run）：8K 10.65 × 0.97 ≈ 10.33 ms。
严格线（common-client run）：8K 10.17 × 0.97 ≈ 9.86 ms；32 档 10.04 × 0.97 ≈ 9.74 ms。
两条线来自不同 run，不混用；验收时以冻结的 ABBA（common-client、3 次配对）重测值为准，accepted gate 不变（≤1.03），0.97 是实现目标。

总账修订 v2（补齐 sha / 作业号 / 卡 UUID；写入硬账判定与精度臂规则）

补齐项均逐条对照 RUN-RECORD 核验（v6/v7/v9），不是从聊天里回填：

轮次	commit sha	作业	卡 UUID	判定
R0	`78a9c235…`	1556	`cecc3184-6d37-8eb6-94b9-f422bfd76dc4`	INCONCLUSIVE（历史观测）
R1	`e0b15b661dffc194e753c51fe0828b4bcf8b7eeb`	1563	`667b4c91-5ee6-c0be-acdc-19d7903ee047`	INCONCLUSIVE（历史观测）
R2	`7080ebd375e64ef4a6e1fd76b6e11f9912715566`	1581	`667b4c91-5ee6-c0be-acdc-19d7903ee047`	INCONCLUSIVE（耗时有效、正确性签核作废）
v5	未记录	1554	未记录	FAIL（作废）
v8	未记录	1575	未记录	FAIL（桶边界绑定 bug）

R0 与 R1/R2 不是同一张物理卡（cecc3184… vs 667b4c91…），跨卡不构成配对比较。

授权事实（必须随数字一起引用）：v7 记录第 76 行写明，作业 1556/1563 两批数据依赖的 plan 原挂 #57，已被 #67 取代并关闭，因此这两批当前无有效授权。按本仓"supersession 不改写历史"的规则，它们作为历史观测保留、可引用，但不构成 #67 下的验收证据。

缺口维持：v5/v8 只有作业号，代码版本未捕获 → 永久记"未记录"，不从 reflog 猜。

硬账判定（写进性能计划，不得简化）

严线 8K 9.86 ms / 32 档 9.74 ms。GEMM 在 FP8 下 8.7 ms 时，非 GEMM 预算只剩 1.16 ms（32 档 1.04 ms），而 vLLM 现在非 GEMM 就是 1.5~2 ms 的水平；同时 GEMM 本身还须从 8.7 压到 ≤8.0 ms。

做到哪一步	总时间	对严线 9.86	结论
只砍非 GEMM 到 2 ms 出头	10.70	1.085	失败，连打平都不够
只砍非 GEMM 到 1.16 ms	9.86	1.00	刚好打平，不算达成
GEMM ≤8.0 且非 GEMM ≤1.16 ms	≤9.16	≤0.93	达成

三条推论：① 非 GEMM 单点优化最多做到打平，必须叠加 GEMM 对齐 DeepGEMM；② "总时间压到 11 ms 以内"是错误目标（11 ms 对严线是 1.12）；③ 32 档预算更紧，任何以 8K 为设计点的优化都要在 32 档复验。

精度臂规则（新增，硬约束）

candidate 当前是 BF16 臂，靶线来自 FP8。图上 2.40× / 3.22× 是跨精度比值，只作方向指针，不作达成度，每张图每个比值必须标两侧精度。

达成度的唯一定义：candidate 自己的 FP8 臂与 vLLM 0.32.0 FP8 臂，同卡、冻结 ABBA、同客户端口径、同 n 各自测出 P50 之后相比。在 candidate 的 FP8 臂产出前，不写任何"完成百分比"。

知识条目状态

五条均为 store candidate（state: candidate），未并入 package：
store:case.qwen38-launch-and-graph-gains-cand、-graph-state-not-accumulating-cand、-fp8-baseline-measured-not-estimated-cand、-v5-timewait-retracted-cand、-v8-bucket-boundary-silent-eager-cand。入仓需限定范围 PR 与 review。运行时记录不等于已认证知识。

总账 v3（目标线 9.87 / 9.74；FP8 基线 run 入账；新增环境失败行与代理口径警示）

1. 目标线统一

严线改为 8K 9.87 ms / 32 档 9.74 ms（10.17 / 10.04 × 0.97，四舍五入）。9.86 作废。中线 10.33 保留但已作废为靶线（vllm bench serve 默认无并发信号量，口径偏松）。计划侧同步：非 GEMM 预算 1.17 ms（8K）/ 1.04 ms（32 档），GEMM 仍需 ≤8.0 ms。

2. 新增/修订行

轮次	变更摘要	issue	PR	commit sha	运行记录	知识条目	判定
FP8 基线（定稿）	vLLM 0.32.0 + FP8，并发 1	#67	无（baseline 侧无代码变更）	无（baseline 侧；仓库 HEAD 当时 `5c7700d`）	job 1646 / vllm-sgp-h20-01 / 卡 `667b4c91-5ee6-c0be-acdc-19d7903ee047` / 客户端 `7f3d4d90…` / fixture `4f4256cc…`	`store:case.qwen38-fp8-baseline-measured-not-estimated-cand`	PASS（测量成立）；FAIL（"FP8 是翻盘牌"假设）
环境失败 1661	`--gres=gpu:8` 自动分配 + UUID 轮询，落到 02 节点 exit 4	#67	无	无	job 1661（未执行工作负载）	`store:case.qwen38-env-pitfalls-nodelist-rpc-nvcc-cand`	FAIL（环境，非模型结果）
v11 回归	ABBA / eager-vs-graph logits	#67	待确认	`5c7700d`	job 1663（ABBA）、1664（logits max_abs）	待出（并入 graph-state 条目）	待实测

3. 证据包核验（InferRSI 侧复算）

evidence/qwen38-h20-fp8-single-request-vllm032：plan/results/patch/validator 四段齐，INVENTORY.json 列出的 34 个文件 sha256 与字节数逐条复算一致；记录本体 rerun/RUN-RECORD-fp8-baseline.md 复算 12e7658a0eb5e93ff71cf5a8566dc1697a4865720c351daab2b645ca94984df3，与清单一致。格式与绑定合格。

4. 代理口径警示（从 1646 原始请求读出，必须随靶线一起引用）

1646 两次运行的实际输出长度是 8 token（32 档）/ 32 token（8K 档），不是 #67 要求的 2048 token：TPOT 是 7 个 / 31 个 ITL 的均值，属短请求代理。TTFT 口径不受影响；8K 档按合同生成 2048 token 时上下文从约 8.2K 涨到约 10.2K，16 层 full attention 的 KV 变长，TPOT 存在上行风险。

因此 9.87 / 9.74 记为代理口径靶线：在冻结一次 2048 输出 token 的基线之前，不得据此宣布达成或未达成；若差距只有个位数百分比，必须先排除代理偏差。

5. 达成度的前置条件（写死）

candidate 的 FP8 臂产出之前，"差多少倍"一律按方向指针表述；达成度只在 candidate FP8 臂与 vLLM FP8 臂同卡、冻结 ABBA、同客户端口径、同 n 出 P50 之后定义。若 v11 的 TPOT 仍停在 23 ms 量级，则非 GEMM 的 14 ms 未被触及，GEMM 与 DeltaNet 必须同时推进——只做一条最多打平。

总账补充：集群 commit ↔ PR 映射（patch 重放，非原 commit 推送）

仓库归属先说清：代码 PR 不在本仓（InferMatrixLab/InferRSI），而在 fork jiangkuaixue123/nano-vllm。本仓只放合同与账目。

PR feat: bootstrap CLI experiment workbench and diffusion knowledge #2：perf(qwen38): cut decode launch overhead and add CUDA Graph buckets jiangkuaixue123/nano-vllm#2 （perf/decode-launch-and-graph）
PR Acceptance: Qwen-Image-2.1-Turbo / B300×4 campaign #3：feat(qwen38): experimental w8a16 streaming GEMV for M=1 decode jiangkuaixue123/nano-vllm#3 （perf/fp8-w8a16-gemv）

凭据落在本机（gh 已授权 jiangkuaixue123），集群无凭据，因此走「集群产出 patch → 本机重放 → 推 fork → 开 PR」。原 commit 未推送到托管远端，PR 里的 sha 必然是重放后的新 sha。

集群 commit（`codex/qwen38-decode-fusion`，无托管远端）	PR（fork：jiangkuaixue123/nano-vllm）	PR commit	文件/行数	状态
`5c7700d` + `720f196`	#2 `perf/decode-launch-and-graph`	`af9e54609`	4 files, +311/−50	draft, MERGEABLE, 无 CI
`733419d`	#3 `perf/fp8-w8a16-gemv`	`4fe5c8a0c`	2 files, +49/−1	draft, MERGEABLE, 无 CI

追溯规则：

追溯以 PR 为准（PR → 本机分支 → patch → 集群 commit）；集群 sha 仅作对账，不作为合入路径。
两个 PR 都是 draft，仓库无 CI，转 ready 前必须附本机 pytest 结果。
PR feat: bootstrap CLI experiment workbench and diffusion knowledge #2 含改数值语义两处：DeltaNet conv/recurrent 原地写、attention 去 GQA 广播物化（bmm batch 布局变更）。描述必须显式声明：「改数值语义，允许 ULP 级差，8K 单步最大 2.09 abs（67 ULP），两次独立运行逐位复现，top1 与 greedy id 全同」。不要写「偶发」「≤1 ULP」。
未闭合：8K 第 6 步 67 ULP 归因（job 1686）。归因清楚前 PR feat: bootstrap CLI experiment workbench and diffusion knowledge #2 不转 ready。

勘误：本评论上一版把 PR 编号写在本仓语境下，容易被当成 InferRSI 的 PR（本仓 pulls/2 是无关的旧 PR）。以此版为准。

总账 v4：R11/R12/R13 的 PR 列复核结果（未能按映射填入）

按 Chief 的映射去改账之前，我先在 GitHub 上核了一遍指向。结论：这两条 PR 当前不存在，因此 PR 列暂不填链接。

复核依据（2026-10-22 11:55，gh 直查）：

检查项	结果
`InferMatrixLab/InferRSI` pulls/2	存在，但是已关闭的旧 PR：`feat: bootstrap CLI experiment workbench and diffusion knowledge`，作者 `david6666666`，head `feat/cli-rsi-bootstrap`
`InferMatrixLab/InferRSI` pulls/3	404 Not Found
open PR 列表	只有 #31/#51/#55，没有 `perf/decode-launch-and-graph` 或 `perf/fp8-w8a16-gemv`
fork `jiangkuaixue123/InferRSI` 分支	无 `perf/`；最近的仍是 `codex/` 与 `perf/minicpm-o45-l20x-throughput`
upstream commit `af9e54609`	422 No commit found

按本账"run 记录里的 sha 必须与本行一致、缺项写无、不许让人误以为已合入"的规则，我不能把不存在的链接写进账。R11/R12/R13 的 PR 列维持无，并加注：报称已开 PR #2/#3，2026-10-22 11:55 复核在 upstream 与 fork 均未找到，待确认真实远端后改为链接。

R11/R12/R13 其余列（按群里最新事实）

轮次	集群 commit	分支	PR	运行记录	判定
R11	`5c7700d`	`codex/qwen38-decode-fusion`（无托管远端）	无（复核未找到）	job 1663 ABBA（v11）	待 1677 归因后定
R12	`720f196`	同上	无（复核未找到）	job 1677 排队中	INCONCLUSIVE（ABBA 跨两个 revision，绑定验证失败）
R13	`733419d`	同上	无（复核未找到）	v13 W8A16 实测（92.05 → 61.25 ms）	INCONCLUSIVE（机制成立，未达 8.7 ms 预算，1.9~2.4 TB/s）

未闭合项：8K decode 第 6 步 67 ULP 未归因（job 1677 排队）；归因前 R11 不转正、任何"数值语义已解释"的结论都不成立。R13 的 BK=256/k0//128 scale 索引错误（rel_err 3e-3 → 5e-1，耗时反而降低）已回退到 BK=128，记为失败细节。

阶段出口（Chief 定的四阶段，记账用）

阶段	出口	当前状态
A	cap 抬到 11264 并重设桶 + w8a16 ≥3.1 TB/s → 第一次合同口径 FP8 对 FP8 ABBA	未开始
B	block-scaled FP8 GEMM（prefill），8K prefill GEMM ≤1.9 s	未开始
C	prefill 非 GEMM 三件（SDPA / 合并两次 solve / 去 fp32 上转），合计 ≤0.35 s	未开始
D	合同口径终局 ABBA：TTFT 与 TPOT 均 ≤0.97，P50 主指标，n=9	未开始

每个阶段过双门：ABBA 数值门 + logits 门（相对量 ≤1e-2 且 top1 相同）。合同口径前提不变：8K 输入 + 2048 输出 = 10240，当前 launcher cap 8320 直接超界，阶段 A 不做完，A–D 全部无法判定。

总账 v5：PR 归属修正（真链接已核验）、新增 TRADE-OFF 判定、靶线改合同口径临时线

1. PR 列修正（已实际打开链接核验）

上一条的"复核未找到"是我按本仓语境查的，代码 PR 在 fork jiangkuaixue123/nano-vllm（parent GeeeekExplorer/nano-vllm），不在 InferMatrixLab/InferRSI。现按真实链接填入并核验：

轮次	集群 commit	PR（仓库归属 + URL）	PR commit	文件/行数（核验值）	状态
R11/R12	`5c7700d` + `720f196`	`jiangkuaixue123/nano-vllm#2` jiangkuaixue123/nano-vllm#2	`af9e54609`	4 files, +307/−50（评论里报的是 +311/−50）	draft, MERGEABLE, 无 CI
R13	`733419d`	`jiangkuaixue123/nano-vllm#3` jiangkuaixue123/nano-vllm#3	`4fe5c8a0c`	2 files, +47/−1（评论里报的是 +49/−1）	draft, MERGEABLE, 无 CI

行数差是 PR 更新后的当前值，以 GitHub 现值为准；追溯链：PR commit → 本机分支 → patch → 集群 commit。

新增记录规则（已写入契约）：凡是指向代码的地方必须写"仓库归属 + 完整 URL"，只写编号在跨仓协作里会出事——本仓 #2 是一条 2024 年的无关旧 PR，引擎仓 #2 才是本轮的 decode PR。

2. 判定列新增 TRADE-OFF，R2 补判

判定取值扩为 PASS / FAIL / INCONCLUSIVE / TRADE-OFF。允许提交"单指标净收益"的 PR，但必须四格齐全（TPOT 32 / TPOT 8K / TTFT 32 / TTFT 8K），每格带实测值、同期基线比值、n。

情形	判定	处置
四项都不劣化且至少一项改善	净收益	合入并计入达成度
一项改善 >3%、另一项劣化 ≤3%	TRADE-OFF	合入但不计入达成度，只作技术积累
任一项劣化 >3%	不合入	除非同时给出补偿方案

3% 依据：同轮基线两遍漂移 <0.5%，n=3 点估计离散度 4.6%，低于 3% 分不清真实差异与噪声。TRADE-OFF 另需开关（默认值写清，可一键回退）与描述里单独一行写劣化项。

R2 补判：TRADE-OFF（已被 v11 修复）。 TPOT 22.39% 级收益（53.03→23.39，−55.9%）对 TTFT 32 档 0.066→0.184 s。一处如实说明：单看 R2，TTFT 32 档劣化 +179%，按 3% 规则本应"不合入"；它被记为 TRADE-OFF 的前提是 R2 与 v11 同批合入（v11 的桶缓存把 8K TTFT 收回 4.95 s；32 档 TTFT 尚未见 v11 后实测，仍按未修复处理）。若两者拆开合入，本行按单独合入重新判定。

3. 靶线改为合同口径（临时）

job 1678（FP8 基线，--max-model-len 12288，两档各 1 条 2048 输出）：8K TTFT 1710 ms、TPOT 10.59 ms；32 档 TPOT 10.22 ms。对代理口径 10.17 / 10.04 是 +4.1% / +1.8%，此前提示的 TPOT 上行风险成立。

临时靶线（合同口径）：8K 10.59×0.97 ≈ 10.27 ms、32 档 10.22×0.97 ≈ 9.92 ms。n=1 点估计，n≥3 出来再锁定。
9.87 / 9.74 降级为短请求代理线，不再用于判定。
32 档 TTFT 69.6 ms 不参与比较（该 tier 首个请求，含首次开销）；1646 的 29.2 ms 是 n=5 中位数。
非 GEMM 预算随之变为 1.57 / 1.22 ms（原代理口径 1.17 / 1.04）；GEMM 仍需 ≤8.0 ms。注意：这条放宽是口径修正的结果，不能反过来当达成手段。

4. 两个未闭合项（决定 PR 能否转正）

项	现状	结论/下一步
pytest parity 失败	`83 passed, 1 failed`；bisect：`78a9c23` pass → `e0b15b6` fail → 之后全部继承（recurrent state 对 oracle `1.5e-6` vs `1e-6` atol）	R1 已改数值语义（fused RMSNorm + 分组 matmul）。规则：修到能过，或带独立证据（fused 与非 fused 同输入互比、差值量级、top1 不变）再改阈值；禁止悄悄调 atol
8K 第 6 步 67 ULP	两次独立运行逐位相同（含 2.0938），7 步在 0.08–0.16，top1 与 id 全同	确定性差异，非竞态。归因实验 job 1686（`git archive 5c7700d` 单独跑）；另一假设：eager 用真实 KV 长度、图用固定桶 8448，tile/累加顺序不同 → 若 eager+手写 padding 也出 2.09，则验收改为 padding 一致的对照臂

PR #2 描述必须写："改数值语义、允许 ULP 级差、确定性可复现、8K 单步最大 2.09 abs（67 ULP）、top1 与 greedy id 全同"——不许写"偶发"，也不许写"≤1 ULP"。

5. 判决资格前置

候选侧 launcher --max-context-tokens 8320 < 8192 + 2048 = 10240，现在没有合同口径的判决资格。阶段 A（抬到 11264、重设桶）完成前，candidate 侧任何数字都不是合同口径。

预注册：prefill 三步出口门（在代码落地前冻结）+ 两处算术必须先定

1. 判据更正：279 不等于 280

预注册的 go 线是 280 TFLOPS，实测是 279（8192x17408x5120），差 0.36%。记录按事实写："279 实测、对照 280 线、偏差 0.36%"，不写成"279 ≥ 280"。

决策维持"接库"，理由是独立于这条线的：另外两个线性形状 277 / 275 同量级（0.36% 落在单形状测量的抖动里），且步骤 1 的端到端投影 3.36 s 已经压进 3.4 s 门。偏差入账而不是四舍五入入账。

2. 三步出口门（现在冻结，测完只填结果）

步骤	内容	出口门	投影	若投影成立
①	prefill FP8 GEMM 接上游 block-scaled 核	≤ 3.4 s	3.36 s	PASS，余量 1.2%
②	full attention 换 fused SDPA	≤ 2.9 s	2.88 s	PASS，余量 0.7%
③	linear attention 接 FlashInfer GDN/conv	≤ 1.8 s	1.95 s	FAIL，差 8.3%

③ 按当前范围结构上过不了自己的门。 两个选择必须在 ③ 开工前定，不能等测完再改口径：改成"③ 门 = 1.8 s 且允许加活"，或把 ③ 扩成四件事（去掉 393 ms 的 copy，或剩余 conv 融合）。否则 ③ 是一个注定 FAIL 的步骤。

3. "持平"这条目标，①–③ 走完也到不了

持平门 = 1710 ms × 1.03 = 1761 ms。阶梯终点是 1950 ms = 1.14×，差约 190 ms，而当前计划里没有任何一步拥有这 190 ms。两个选项：

加 ④：明确接管这 190 ms（候选来源：copy 393 ms、conv 71 ms 之外的残余、或 GEMM 从 279 往 290+ 抬）；
或把目标从"持平"改成"复刻后尽量接近"，并写清差距预期（10~15%）。

这条不写清，后面每轮都会在"为什么还没持平"上重复讨论。

4. 记录

store:case.qwen38-decide-upstream-fp8-libs-vs-own-kernels-cand 已更新：验收 279/277/275/226、M=1 3 TFLOPS（1.7 TB/s）、PASS（带回 0.36% 偏差说明）、"不覆盖 decode"入 limitations。
store:case.qwen38-prefill-ladder-gates-cand 新建：三步门表、投影与门的差、持平门算术、③ 的结构性问题。均由 KnowledgeStore.update/create 落在 candidate，未改 package 内容。

5. 记账口径

每步落地后按七项出记录：命中门 → PASS；未命中 → FAIL 并写"卡在哪一块"（GEMM 未达 / 融合未生效 / 依赖入口不通）。③ 若因门本身不可达而 FAIL，不算工程失败，但必须在同一行注明"门与投影矛盾，已于开工前记录"。

总账 v7：选中的库离 go 线差 2.5%、decode 结论翻转、warmup 剔除入约定

1. 280 TFLOPS 这条线：两个选项都没过，必须按"未达标"记

库	8192×17408×5120	另两个线性形状	M=1×17408×5120	可用性
DeepGEMM 2.8.0（vllm vendored）	279 TFLOPS	277 / 275	3 TFLOPS / 1.7 TB/s	不可用：`.so` 绑 torch 2.13，引擎 env 是 2.14.1 → `OSError`
FlashInfer 0.7.0.post1（PyPI wheel）	273 TFLOPS	272 / 269	3273 GB/s	可用；SM90 kernel 是 JIT，需 `ninja` + `nvcc`，首调用约 76 s
自写 Triton block-scaled	2.8 TFLOPS	—	—	已关闭

结论：决策维持"接库"，但账上写 PASS（接 FlashInfer，离预注册 go 线差 2.5%）。之前那条"279 达 0.36% 偏差"的记录作废——那是在 vllm env 里测的 DeepGEMM，选中的选项是 273。不许把 273 写成达标。

① 的出口门因此极窄：273 TFLOPS 对应 GEMM 1.61 s，投影合计 3.38 s 对门 3.4 s，余量 0.6%。折算不算通过，必须实测稳态。

2. decode 的权重流：结论翻转，但仍要给实测

M=1 3273 GB/s → 27 GB / 3.27 TB/s ≈ 8.3 ms，落在 8.7 ms 的 GEMM 预算内。所以"decode 权重流必须自研 w8a16"改为"接同一条 FlashInfer 的 M=1 路径"，w8a16 自研 kernel 降级为备选。

但 8.3 ms 是单形状微基准的下限，不是端到端 decode：64 层里 qkv/o_proj/gate_up/down 形状各不相同，还要加 per-token×K128 的动态激活量化开销。非 GEMM 预算因此从 1.22 / 1.57 ms 放宽到 1.62 / 1.97 ms（32 档 / 8K），这 0.4 ms 是口径变化，不许当收益。

3. 数值语义声明（PR 必写）

激活量化 = per-token × K128 动态；权重用 checkpoint 自带的 weight_scale_inv，布局 [N/128, K/128] 原样使用。这两条任一改动都会移动数值，按契约 §八必须显式声明。

4. 环境：坑扩到六条，登记表待核验

新增三条与 nodelist / RPC / nvcc 同源：/tmp 是节点本地（登录节点准备、计算节点执行 → ModuleNotFoundError，job 1686，已重提 1762）；PyPI 的 deep_gemm 是抢注假包（18 KB sdist）；vendored .so 绑定 torch 小版本。

引擎 env 新增 flashinfer-python==0.7.0.post1 + ninja，我暂不登记到 env 表：链路断着，拿不到确切路径与实际版本，按"未知标待核验"处理，恢复后由优化师给路径、我补登记。

5. 约定新增（已写入记录契约）

每 tier 第一个请求一律标 warmup 不进统计；JIT 首次编译单独记录并剔除；含首次编译的数不得作证据——145.3 s 据此降级为"含首次编译，不可作证据"，不得再作"FP8 臂慢 30 倍"的引用。
参照臂（照搬上界）不进主分支、不进 PR，且必须在 vLLM 自己的 env（torch 2.13） 里跑；在引擎 env 里跑会先撞 OSError，得到"参照臂也不行"的假结论。

状态更新 2026-10-22：decode 侧三件合入、内部工程门变更、数值诊断口径

本条只追加记录，不改本 Issue 正文与已冻结的验收目标。以下内容分两类，请勿混用。

1. 候选侧代码身份更新（candidate 从占位符变为具体 SHA）

候选实现：jiangkuaixue123/nano-vllm，main = 3f7516a
合入内容：
- PR feat: bootstrap CLI experiment workbench and diffusion knowledge #2 + PR docs: OmniRSI repo template (RSI Team chapters) #4：decode 单步 CUDA Graph（按桶复用）、KV 预分配原地写、桶容量用尽回退 eager、启动时桶预热、_IncrementalText 字节安全增量解码、契约测试 tests/test_qwen38_decode_contracts.py
- PR feat: portable RSI repo scaffold (copyable layout + four-lane CI) #5：投影融合（q/k/v、DeltaNet in_proj_*、MLP gate/up 各合成一次 matmul，只保留一份权重）
需要如实记录的一处过程错误：PR feat: bootstrap CLI experiment workbench and diffusion knowledge #2 合并时分支 head 停在旧版 af9e546，而描述写的是评审版 0ad392f；缺失部分已由 PR docs: OmniRSI repo template (RSI Team chapters) #4 补齐，补齐后 main 的树与评审版逐字节一致。这条不隐藏。

2. 与 #67 验收目标的关系（本段不是性能结论）

#67 的目标是 8K / 64K / 128K、2048 输出、候选/基线 TTFT 与 TPOT 比值各 ≤ 1.03。

当前实测（仅 8K 档、生成 32 token、非 #67 验收口径，仅作工程进度）：

项	baseline vLLM 0.32.0	candidate `3f7516a`	比值
TPOT	16.04 ms	20.38 ms	1.27
TTFT	3.06 s	4.82 s	1.57

未达 parity（≤1.03），64K 与 128K 未测。任何"加速倍数"表述都不成立。

3. 内部工程门变更（仅约束我们自己的合并动作，不改变 #67 门）

项	旧线	新线	改判理由
步内空隙	≤1.4%	中位数 ≤0.40 ms（≥100 步）	绝对量未变（0.30→0.314 ms），分母从 22.94 缩到 21.92 ms 就把比率顶到 1.44%；百分比判据会惩罚变快
TTFT 8K	单点 ≤4.80 s	两版中位数差 <2%	4.80 s 是单次测量值；作业 1723 各跑 12 次得 v7 4.7956 s vs 候选 4.8168 s，差 21.2 ms = 0.442%

这两条只用于内部合并判断，不进入 #67 的验收判定。

4. 新增启动成本（部署口径变更，必须记录）

启动时预抓 5 个 CUDA Graph 桶（256/1024/2048/4096/8448）：

启动时间 +1.45 s
常驻显存 +1821.4 MiB，其中 8448 桶 685.2 MiB、4096 桶 415.2 MiB
计量起点已冻结并对账：server start < prewarm.start < prewarm.end ≤ ready < first_probe，两轮均成立

这是用约 1.8 GiB 显存换 8K 首请求的 0.17 s capture。不接受该成本时可关 NANOVLLM_CUDA_GRAPH。

5. 数值诊断口径（按本 Issue "diagnostics" 定位执行，不作 gate）

参照实现：vLLM 0.32.0 + Qwen3.8-27B-FP8，单卡（与本 Issue 的 baseline 定义一致，不再另找 transformers 参考）。
第二层诊断：同一引擎内 CUDA Graph 路径 vs eager 路径，逐步报 max_abs、按 bf16 ULP 折算、top1 是否相同。
为什么升级：ids 一致已被证伪——v11 状态不累积时 greedy ids 全同、logits 差 0.70（67 ULP）。故 ids 一致降级为冒烟项，不再作为等价证据。
待回测：0ad392f（融合前）vs 3f7516a（融合后）。投影融合改变累加顺序，必须按上述口径复核。
阈值尚未确定，不预设、不在事后放宽。

6. 已知限制

同一进程只支持一个开启 CUDA Graph 的引擎（共享 capture pool 会互相覆盖缓冲区），代码中已改为显式 RuntimeError，不实现每引擎独立 pool。
decode 单步不做 isfinite 校验（每步 .all().item() 会插入同步并阻断 capture），prefill 仍校验。
GatedRMSNorm 的 silu 由 fp32 改为按 x.dtype 计算，属数值路径变更。
融合后 state_dict 键变为 qkv_proj / in_proj_fused / gate_up_proj，只影响导出，不影响推理。

7. 未解决项

8K TTFT 仍比 baseline 慢约 1.76 s，属 prefill 实现差异，不在 decode 范围内。
FP8 路径 8K TTFT 约 145 s，怀疑是 SM90 kernel 的 JIT 首调用编译（与本 Issue 讨论中 FlashInfer 首调用约 76 s 的现象同源），先做"同进程 cold vs warm"对照再定性。该排查归 prefill 线程，不并入 decode 改动。
8K/64K/128K 的 2048 输出长跑、task-quality 三套评测均未开始。

8. 授权

本批 decode 数据此前挂在一张已关闭的验收单下，现按本 Issue 重新 freeze：candidate 3f7516a、baseline vLLM 0.32.0 FP8、测量脚本与作业号（1712 / 1717 / 1723 / 1756 / 1757 / 1759 / 1761）均已留存。数据本身不重跑。

完整测量记录与证据见 PR 正文：jiangkuaixue123/nano-vllm#5 （融合）与 pull/4（契约与预热）。

数值判据拆成两级（改判记录，2026-10-22）

本条记录一次判据变更及其理由，供后续评审追溯。这是改判，不是 PASS。

触发

第二层（同一引擎 graph vs eager）与跨树回测暴露出：单靠端到端 ULP 阈值会把两类完全不同的东西混成一个 FAIL。

一类是语义改错：C1a 第一版融合 kernel 让归一化吃未舍入的 fp32 和，实测 8K 第 13 步 29.9 ULP、词表 78.5% 元素整体偏移、mean_abs 涨 20 倍。
另一类是纯累加顺序：C3 投影融合，算子级全部调用 ≤1.00 ULP、>1 ULP 的调用数为 0，但端到端在混沌步放大到 17.5 ULP（32 档 eager）。

任何合法 kernel 改动（哪怕只是换 cuBLAS 版本）都会产生第二类放大。用端到端 ULP 一票否决，等于禁止全部 kernel 级优化。

新判据

级别	判据	作用
一级：算子级	同输入下与被替换算子逐元素比，≤2 ULP 且能归因到累加/舍入顺序	准入
二级：端到端	不设 ULP 阈值；记录最大 ULP、越线格数、混沌步位置	记录
三级：task-quality	GSM8K / HumanEval+ / MMLU 与 vLLM 差 ≤0.01 绝对值	最终裁决

一级不过 = 不准入。一级过、二级越线 = 允许合入但必须在 PR 记录，且三级未做前不得声明等价或加速结论。三级不达标 = 回滚。

已按新判据处理的项

项	一级	二级	三级	处置
C3 投影融合（已合入 `3f7516a`）	通过（≤1.00 ULP）	记录：32/e 17.50、32/g 10.00、8K/g 9.50、8K/e 9.05；G3 绝对差 3 格越 1.0	未做	不回滚，挂 task-quality 为最终裁决
C1a add+rms 融合（PR #6）	待定	32/e 28.06、8K/e 10.50；G2/G3 均越线	未做	draft，不合

C1a 尚存的矛盾（未解决前不下结论）

算子级复核（作业 1821）报告 normed 与 F.rms_norm(x+mixed) 最大绝对差 0.0、100% 元素相同；但端到端（1825）32 档仍是 28.06 ULP。两者不可能同时成立：算子逐位一致则整树必须逐位一致。已安排 1827 逐层重测（调用前先 clone，避免上次参考值吃到被改写残差的问题）裁决。

两处此前陈述的更正

"C3 只越线 1.5 ULP" 不成立：32/g 与 8K/g 是 G3 绝对差越线（1.25 / 1.188），不是 G2。
"我们的 RMSNorm 多吃一位精度可解释第三层 normse 1.5e-2" 不成立：fused_add_rms 只存在于未合并的 C1a 树，main-3f7516a 中 grep fused_add_rms = 0。第三层那 2% 需另找原因。

第三层现状（对 vLLM 0.32.0 + Qwen3.8-27B-FP8 单卡）

拆解（只取两侧前缀相同的前 3 步）：

对照	normse
engine BF16 vs vLLM BF16（栈间基础差）	0.0051–0.0087
engine BF16 vs engine FP8（我方量化）	0.0037–0.0043
vLLM BF16 vs vLLM FP8（对方量化）	0.0149–0.0262
engine FP8 vs vLLM FP8（最终问题）	0.0148–0.0217

结论：不可互换。 依据是 2% > 1% 的既定容差；同时必须并列写明——该移动量与 vLLM 自身 BF16→FP8 的移动量同量级，没有证据指向某一侧存在 bug。ids 层面 8/8 全同、top1 全同，与分布层面的 2% 是两个口径，不得只引用其一。

机器与偏离

上述 C1a / C3 判定作业跑在 vllm-sgp-h20-05（--any-gpu 降级），ref 与 cand 同作业同卡，比较内部有效；物理卡不是 667b4c91，属已声明偏离，最终合并证据需回该卡复跑。

C1a 矛盾已解，并新增一条方法学约束：算子级门必须用真实激活

结论：此前两条陈述都成立，它们是两个层级

层级	结果
算子级（真实激活，包住每次调用）	残差写回 64 次/步全部逐位相同；归一化输出仅 0.019%–0.156% 元素不同，单次最大 1.94 ULP
端到端	32 档最坏 28.06 ULP、绝对差 3.508；峰值出现在第 24 步，而该步的算子级只有 1.80 ULP

即：不是某一步算错，是 64 层 × 25 步把每次 ≤2 ULP 的归约顺序差放大到混沌步上。 此前"算子级 0.0"与"端到端 28 ULP"看似矛盾，实为不同层级。

同树消融（唯一变量确认）

对比	最坏步	max_abs	ULP 等效
cand 真 kernel vs cand 等价 torch 回退	24	3.507812	49.5
cand 真 kernel vs ref 树	24	3.507812	49.5
cand 回退 vs ref 树	—	0.0（逐位相同）	0

ref 树干净、cand 只差一个算子、回退路径与 ref 逐位等价。

方法学约束（新增，长期有效）

算子级准入不得使用合成/随机数据。 此前一次算子级对照用温和随机数得到"最大绝对差 0.0、100% 元素相同"的假阴性——随机数不触发归约顺序差，只有真实激活的幂律极端值才会。此后算子级门必须：真实激活、包住一次完整 decode 步的全部调用（本项为 64 次/步）、报告"残差逐位相同与否 / 归一化单次最大 ULP / 差异元素比例"。

已要求判定方用自己的抓取框架独立重算，不得直接引用实现方的数字。

C1a（PR #6）当前状态

一级（算子级）：通过（最坏 1.94 ULP，残差零误差）。
二级（端到端）：记录在案（32/e 28.06、8K/e 10.50 等），按两级判据不构成否决。
三级（task-quality）：未做。
处置：等独立复核通过后合入；三级未做前不得声明等价或加速结论。

同步更正

上一轮记录的"C1a 算子级 0.0、端到端 28 ULP 二者不可同时成立"这一说法作废——成立的是"两者量的是不同层级"。

C1a 已合入（一级通过），候选身份更新为 `83b567c`

项	结果
合入	PR #6，merge commit `83b567c`（`main`）
一级（算子级，准入）	通过：残差写回 1664/1664 逐位相同；归一化单次最大 1.00 ULP（binade）／1.94 ULP（相对 eps）；差异元素 0–0.156%；可归因到 `tl.sum` 归约顺序
二级（端到端，记录）	32/e 28.06、32/g 27.88、8K/e 10.50、8K/g 14.00（ULP）；混沌步 24
三级（task-quality）	未做——最终裁决

判定由 benchmark和精度测评执行者 用自己的挂钩框架独立完成（作业 1836），未跑实现方脚本；并与实现方模板在 inputs.pt 上做逐次复算对齐（作业 1838/1839）。

ULP 口径口径差异已澄清

同一份 inputs.pt 上，两种 ULP 约定下 max_abs 与差异元素数逐次一致，只有分母不同：实现方用相对 eps（|x|·2^-8），判定方用 binade 步长（2^(floor(log2|x|)-7)），后者恒为前者的 1–2 倍。此后一律用 binade，引用相对 eps 数字须注明最多放大 2 倍。

当前未决

三级 task-quality 未做：C3（3f7516a）与 C1a（83b567c）都不得用于任何"等价"或加速结论，直到 GSM8K / HumanEval+ / MMLU 与 vLLM 差 ≤0.01。
第三层对 vLLM 0.32.0 + FP8 单卡：32 步 prompt_logprobs 参照臂已修好，待出最终"能否互换"答案。当前 3 步结论为不可互换（normse ~2%）。

第三层 32 步版（`prompt_logprobs`）：结论仍为不可互换，且尾部发散被暴露

作业 1837，参照 vLLM 0.32.0 + Qwen3.8-27B-FP8 单卡，两侧同 checkpoint。

项	结果
greedy token ids	32/32 全同
top1	全同
top-20 logprob 的 normse	1.5% – 26%（第 28、31 步跳到 0.26 / 0.13）
top-20 集合不同的步数	21 / 32
超 rtol/atol 的元素	388 个
判定	不可互换（容差 1%）

此前 3 步版（只能比同前缀的前 3 步）给的是 1.5–2.2%。扩到 32 步后，同类指标升到 1.5–26%，说明分歧在序列尾部显著放大，而不是评测方法变严。ids 层面 32/32 全同与分布层面不可互换，两个口径必须并列写，不得只引用其一。

质量评估的协议适配（已声明偏离）

run.py 要求每条响应同时具备首块 prompt_token_ids 与逐块 token_ids。我们的引擎侧可由适配层补齐；vLLM 0.32 的 OpenAI completion schema 无 return_token_ids（已核对 vllm/entrypoints/openai/protocol.py），故 baseline 必然拒绝。

决定：在 vLLM 前加一层 normalizer，请求 logprobs=1，把 vLLM 自身返回的 logprobs.token_ids 提升为 choices[].token_ids；不改 vLLM、不改冻结的 run.py。约束：

只搬运字段，normalizer 内不得出现 tokenizer encode/decode 或重新分词；
原始 SSE 另存并与 normalized 一一对应，均留哈希；
对称性对照：同一 prompt 带 / 不带 logprobs=1 各跑一次，最终文本必须完全一致，否则该方案作废；
本条作为对 baseline 请求的已声明偏离记录，normalizer 版本与哈希一并留档。

同时记录候选侧适配层：main-83b567c-api（仅改 server API、不动模型，server.py sha256 4ecb848c…），作业 1853 证明与 main-83b567c 逐位相同（0.00 ULP、top1 全同），因此质量分可归因到 83b567c 本身。

Prospective tile64/N8 CPU compile discriminator, model plane; no GPU allocation.

Freeze ff025f38162e40b48c23a2608ab42929c5fcd5ec4ad37dffb5c23dbf75117622 binds the exact reviewed source archive b79563f1bf36a9cbd9832d94b802c676ebb854139112ed0d11dd43d224bec513 (18 files), compiler manifest 8f3043d4ca7d52062f24ab06cc5071124a5dbb1e5560b1e336275810c509736c and unchanged47-file runtime payload 60e05ddc7dc9553e98df5bdaf89e3496b884f615a50e6b26ea8727c752fa1bd5.

Four functions only: original N8 controls and fixed output64/token8/four-stage/math80/grid156 candidates for N5120/K6144 and K17408. Official FP8 bytes/scales, K128 promotion/accumulation order and BF16 output are unchanged. Source, compiler, parent helper and actual inactive composition reviews found no blocker. Compiler review d47e497a07ff8b65e625b20d1734b51567760e9c15e087c7a670c545fb366c4c; composition review b9c4042e517059a801ded8e5eaeb97ff900e44a72b8f3bac8d47675c6c1bbd12. All50 authored copies and47 payloads authenticate.

One submission/no requeue/no retry: 8 CPUs,16 GiB,300 seconds,0 GPUs. Preserve exact registered toolchain/flags, one dependency pass, one shared build, three cuobjdump captures, actual before/after identities, genuine partial/error output and bounded owned cleanup. Library is retained as bytes, never loaded; no CUDA API/model/Graph execution. Literal ALL uses a fully replaced sanitized client environment. Stop without alternate flags, source, sweep or fallback on failure/unresolved evidence. Actual initial REG/static storage/spills/register-pool legality remain UNKNOWN until independent emitted-code review; math80 is only a source request, not compiled allocation or occupancy.

Search budget remains 21535 actual +0 reserved /28800 GPU-seconds,7265 remaining. This CPU-only trial reserves/charges zero GPU-seconds. Both engines stay non-speculative; formal acceptance remains INCONCLUSIVE. Actual job/terminal outcome will be recorded separately.

Parallel L0/L1 preparation corrected an initial scope assumption: retained2960/2887 data supports the prequantized GEMM consumer only, not the upstream producer. Future microbenchmarks will explicitly exclude producer cost and cannot justify service gains. No BF16 input reconstruction or extra capture was performed; later integrated HTTP evidence remains required.

CPU2984 compiler discriminator independently reviewed; no compiler/resource/code blocker for a separately frozen fixed L0/L1 step.

Job2984 COMPLETED0:0 in9 seconds,8 CPUs/16 GiB/0 GPUs, no requeue/restarts and released. Dependency pass, one shared build and three cuobjdump captures all exit0; all five owned groups clean up completely. Source, guarded718 tool/header identities and three runtime-command identities match before/after. This is 72 allocated CPU-seconds and zero GPU-seconds, counted once; search remains21535 actual+0 reserved/28800.

Exact library e4eb91a8fd55786b50c29a409f5e7bd9571093ae7c69c9c86a347460e1dcf535 (174560 bytes), never loaded. Both new output64/N8/four-stage functions have actualREG80 with256 threads; controls haveREG168 with384 threads. All four haveSTACK0/LOCAL0/staticSHARED1024 and no detected local spill accesses. Function-specific PTX/SASS confirms warpgroup-uniform dec40/inc80 candidates (inc232 controls), no CTA-private register-pool deficit, native FP8 WGMMA and BF16 x2 stores. Actual static storage and source dynamic memory remain separate; no occupancy/residency claim.

Capture archive a6b93480412d5db231659d812a4fdc330e2eb208d650ff44731ecdcd5f4c47fe, manifest 0572cbe0fd17b957198163063340c40f9ecaa3251551c1c7946c618f6677c91e: all113 payloads plus manifest authenticated, zero exclusions. Independent report 6f648269cda66c3c37bf691d3750aa96f6572cf3224ca17f408ac85cb0318587, integrity 1c6ff72273c91951e3bbbfdb2e7591b8c66c36e879b176d93b992741ace0854d, resource table 4c7cb74e92245134ad7a484cd8c5c160e2776b5922548f354f57940400824188.

The two-case prequantized-consumer L0/L1 package is independently source-reviewed (91f730f8dcb41bfc898646dc8061c147aa17079609df4222e9054b4d5ad073cc) and is being bound to these actual artifacts. No GPU reservation or run yet. Runtime numerical equality, forward progress, Graph safety and speed remain untested; formal campaign verdict INCONCLUSIVE. Both engines remain non-speculative; no source/flag retry or sweep occurred.

Prospective fixed tile64/N8 L0/L1 screen (model plane; both engines remain non-speculative).

Exact freeze 34c7af2bd0ccce0e2435f96006b419f3cdbad84497599f431e5db51414192266; executor payload 1f729a6c4cfa6db0eb3a30a4050875627c4a16f72b5b2e79d5963f9e1dc41784 (41 files). It binds actual CPU2984 library e4eb91a8fd55786b50c29a409f5e7bd9571093ae7c69c9c86a347460e1dcf535, all105 compiled-closure files and independent emitted-code/resource review 6f648269cda66c3c37bf691d3750aa96f6572cf3224ca17f408ac85cb0318587. Parent helper review 84461f5c846bfc528ea3ab55ec0839d0b98a87049bb66a0f056a5f9a6be93a34 and actual composition review b4b5b55814151b4b827990d0e108b2da694b3c98f77725fdacbe0b4a08c9028e found no actionable blocker. Only four pending metadata bindings changed; numerical/timing/input source is unchanged.

Workload: two retained prequantized consumers, GDN output N5120/K6144 and MLP down N5120/K17408; official first-layer weights/scales, retained first/last q/scales, exact native-reference checks. Controls use output128/N8/S8/math232; candidates output64/N8/S4/math80. Strict finite bit identity is required across original/changed/restored eager, Graph, native-oracle, cross-policy, ownership and trace checks. Timing retains32 alternating pairs per case (128 targets); conditioning is outside each target event. This excludes upstream producer and E2E cost and cannot award a service gain or residency claim.

Reserve480 GPU-seconds now within the existing search phase:21535 actual +480 reserved /28800,6785 unreserved. One allocation,1GPU/8CPU/16GiB/480s; no requeue, retry, variant, tolerance, rebuild or fallback. Sanitized replacement client environment with literal ALL export; preserve Slurm device visibility. Exact package will be activated only after this reservation is recorded in knowledge. Retain all partials, cleanup and terminal evidence. CPU2984 review is complete; its9s/8CPU used zeroGPU.

Scope remains8K/2048/C1, ≥25% TTFT or TPOT improvement with ≤5% regression under the approved formal statistics and unchanged quality gates. This screen is INCONCLUSIVE for that campaign; it is not a vLLM baseline refresh or formal acceptance.

Tile64/N8 L0/L1 job2986 completed and independently reviewed; reject this fixed candidate for engine integration.

Under prospective freeze 34c7af2bd0ccce0e2435f96006b419f3cdbad84497599f431e5db51414192266, the two conditioned prequantized consumers were correct but slower:

Consumer	Control median	Candidate median	Median paired ratio
GDN output N5120/K6144	13.984 µs	14.304 µs	1.0203177070 (+2.0318%)
MLP down N5120/K17408	31.200 µs	38.944 µs	1.2470225683 (+24.7023%)

All4 eager,12 Graph,12 native-oracle,6 cross-policy,2 retained-reference and4 actual trace checks completed, with128 target timings. No tolerance or fallback; changing/restored q/scales, every output slice, stable ownership and exact selected kernels verified. Actual CPU2984 library e4eb91a8fd55786b50c29a409f5e7bd9571093ae7c69c9c86a347460e1dcf535 was loaded without rebuild. Before/after guards agree and owned cleanup is complete. These are consumer-only observations, not upstream producer, E2E, occupancy or residency gains; no statistical-significance claim.

Capture archive e13590bf6fd8194527ff9011ecb43b2794d827de238e236ee1c0ec94ac8d5269 (2862618 bytes), manifest 19cb46659c69a03d4db0f45ee3d83aa3721e9d54570464c013cdd381aa9623c7:201 payloads/202 regular archive members, zero exclusions, full105-file compiled closure plus bound operands/index. Independent review f2bf6079e07463723bc7f7359c867575299aa5cde793fb7ae9b68f2759b83310, integrity 6ecbd150efd23110f932e82bdf59437882892122e4e0a38127086275fdd71771, reproduced the exact frozen audit byte-for-byte and separately recomputed all paired timings.

Terminal state COMPLETED0:0,40s,1GPU/8CPU/16GiB, no requeue/restarts; owner queue empty. Charge40 GPU-seconds once and release440 reserved: search21575 actual +0 reserved /28800;7225 remaining. CPU2984 remains zeroGPU. This negative result and aborted integration preparation are retained in knowledge. No engine commit/PR is appropriate for this regressing candidate; no E2E job will be spent on it.

Both engines remain non-speculative. Campaign25%/5% and task-quality acceptance remain INCONCLUSIVE. Next bounded work is bottleneck/headroom discrimination; generic96GB H20 or H100 peaks must not be substituted for the authenticated H20-3e SKU.

Prospective current-engine Graph diagnostic, harness plane; both engines remain non-speculative.

Freeze 3462b605bcf694612ed8249f3619c4abd0f11bd193a0edc599c94878abb962a9 binds merged source 44ca6732f466be233e1ef06ecbde1813b4de5002, tree be17d2be5ed92a71df7d2ca405ec769495c36226 (the2970 tested tree), exact2970 options,45 immutable runtime payloads and10 unchanged native artifacts. Author manifest 8062faf6d8e4d8699569f0173ede333e4d2165f6ddd988b00ec4c6f72b4623a5; payload 0cb44e7f7aabf11526e34c04c33e81b6872be0d6757e1d26c84e6ee4bfe14c8f. Independent source review 767e288295954c7025312d7f6d0fb21302583858cca2683fa58ce0f5b1169e69 and helper review 25ff4b6646ef5f1e7e3518c78c7c147550109119112bc59141604e42638a8e1f found no blocker. Parent authenticated actual composition; no engine algorithm change.

Workload: fixed n8192-s0; production full-context prewarm; ordinary8-append CUDA-event measurements followed by a separate8-append Torch trace. A separate same-history process performs7 ordinary appends, then NCU selects exactly one prepared production Graph replay at position8199; input fills/bookkeeping stay outside profiling. Full BF16 vocabulary bytes,128 live-state tensors/metadata and all predictions must match normal/trace/NCU. Each process uses unchanged eager production builds and retained build evidence.

NCU2025.3.1.0 fixed ELF and67 packaged files are bound; no completeOS/driver closure is claimed. Fixed whole-Graph/kernel replay, one launch, five hardware metrics plus two replay statistics, cache-control none/clock-control none, explicit stable pipeline boost, config off and stock deployment disabled. These are profiler observations, not production/E2E timings or per-consumer hardware attribution. Metric query success is not counter permission. Unsupported/denied collection terminates without alternate metrics, escalation or retry; normal trace and all partials remain available, with a normal-only audit unable to mark the entire diagnostic complete.

Reserve900 GPU-seconds now: 21575 actual +900 reserved /28800;6325 unreserved. OneGPU/eightCPU/96GiB/900s, no requeue;780s phase bound plus120s margin. LiteralALL exports only the sanitized replacement environment. Owned cleanup, pre/post guards, terminal accounting and complete critical result capture remain required. Activation follows this Issue and knowledge reservation; no vLLM rerun or final acceptance is included.

The rejected2986 result and source-derived headroom memo are retained. This run will distinguish current Graph gaps from dominant kernel costs and, if counters are available, measure aggregate physical traffic. Logical model bytes are not measured HBM traffic; exact H20-3e ceilings remain unknown. Formal8K/2048/C1 25%/5% and task-quality acceptance remain INCONCLUSIVE.

Current-engine diagnostic2990 failed before model execution. Terminal FAILED1:0/8s; the allocation is released. Allocation/tool checks passed, but the first NCU package path check rejected. The original log records only the combined NCU input path predicate; it does not identify which entry or whether the cause was absence, file type or canonical-path mismatch. Do not attribute a specific node/tool mismatch without new evidence.

Normal/model/NCU profiling were NOT_RUN. Applicable tool postguards agree and postflight completed; no owned model/profiler child started. Campaign/result INCONCLUSIVE. Full capture archive 3e300e517649c078ad0fc035c6066cf217c35b6edf00cf0d57b8813911682734, manifest f48c3300f7f973c339adb3d68e87202def8da98f6dd137db0bbd0df6894ae197,235 payloads/zero exclusions. Independent outcome review 1155f3deaa8187d5629995f4ecb08df7af397e15e0ef1f4bc5fcaa7960a74abf, integrity cce7c610f4b3da861a5e0c5b4edefd15b8ed5c3aa4b9ab43c6177c23a04738b3, confirms the failure and bounded evidence.

Charge8 GPU-seconds once/release892: search21583 actual +0 reserved /28800;7217 remaining. No retry of the frozen job.

Next is a separate zero-GPU metadata intake on the actual prior compute node, because system-tool paths are node-local: fixed67-file inventory, path/type/canonical-path observations and hashes only when the expected path condition holds. No CUDA, model or NCU execution. Freeze 530167b9dcde7b8a21b1f22e560e7cf0c5383c76faff9bb3fad64de80cf5f3d0; one60s allocation,2CPU/2GiB,zeroGPU,no requeue. GPU ledger unchanged. A prospective harness repair separately moves NCU checks after ordinary capture so missing tools cannot suppress the ordinary trace, and retains offending-entry diagnostics; v1 remains frozen. No new GPU reservation or activation is authorized by this metadata record.

CPU-only actual-node intake2993 completed under freeze 530167b9dcde7b8a21b1f22e560e7cf0c5383c76faff9bb3fad64de80cf5f3d0: COMPLETED0:0,1 second,2 allocated CPUs,zero GPU TRES,no requeue/restarts,released. This follow-up is separate from failed2990 and does not rewrite its anonymous original error.

New evidence resolves availability on the actual2990 node: the toolkit dispatcher is byte-identical, but all66 bound versioned2025.3.1 paths are missing (FileNotFoundError); the observed installed directory is2025.3.0. Thus the head-node2025.3.1 inventory was not portable to that compute node. No NCU executable,CUDA or model was invoked in2993; existence of2025.3.0 alone does not qualify its executable,metrics or counter permissions.

All8 captured files, exact frozen payloads, actual-node equality, terminal allocation and queue absence verified. Capture manifest 4d27fa9ef769de78eeeea54cde8e6f5ed7080344b6ee75b53c17f334d616efcb; raw intake 0d2454e36d70bfcb3c2298319a33e8650c8acc7ba64779fe356dc5bc3b5e3573. Allocated2 CPU-seconds,0 GPU cost: search remains 21583 actual +0 reserved /28800;7217 remaining.

Prospective v2 runtime repair independently reviewed with no blocker (a394f21c648cf37a55417d43bc3c6ffc6abab4ffbda2624c7ad80eb02c9881e8): ordinary capture precedes NCU preflight and failed tool checks name the offending entry. Tool binding remains PENDING and activation false. Next tool qualification will explicitly identify the version used; no silent substitution, guard weakening, new GPU reservation or retry.

Fairness is unchanged: both engines are non-speculative. N8 is the FP8 MMA tile width, not speculative decoding. Any future speculation comparison would be a separately frozen matched configuration, outside this ordinary-decoding campaign. Formal8K/2048/C1 25%/5% and quality acceptance remain INCONCLUSIVE.

Prospective zero-GPU actual-node tool qualification, after2993 established that the head2025.3.1 package is absent on the compute node. Explicitly select the existing2025.3.0 package for a new metadata-only attempt; no silent replacement of frozen2990 or activation of pending profiling-v2.

Freeze 5a8b5b697cb05299021b5ceae1a9075e7b9ef7dfe104694fc3a5fa2838a427f0, author manifest ffa03370232fa0372456c629af7d12a5e1402eb1c9220ddc3f7fa9b3aa38d436. One60-second,2CPU/2GiB,zeroGPU/no-requeue allocation on the observed2993 node. Read the same67 selected relative files using actual3.0 hashes; retain static ELF names,bundled manuals/EULA,finite directory listings and pre/post identities. Execute only the fixed versioned ELF's config-off --version and --help once each,10-second caps plus bounded owned process-group cleanup,clean replacement environment/task-owned write paths and preserved Slurm visibility. No model,target application,metric/device query,installation or counter collection.

Parent reviewed the actual source and authenticated its payloads. Initial local-only source explicitly hid device visibility; corrected before execution to preserve Slurm values, with earlier hashes retained in provenance. Any missing selected file,command error or identity drift remains a failed/incomplete intake; no automatic retry. Tool identity/help does not establish metric availability,counter permission or full OS/driver closure. Search stays21583actual+0reserved/28800;7217remaining,zeroGPU cost/reservation. Formal campaign remains INCONCLUSIVE; both engines non-speculative.

Actual-node metadata2995 completed: COMPLETED0:0/1s/2CPU,zero allocated GPU TRES,no requeue/restarts,released. Both fixed config-off --version/--help commands returned0. Actual existing version is2025.3.0.0 build36273991; all67 selected file identities match before/after. Fixed ELF is65326992 bytes, SHA 064645ab3a19d71d4ac856b9f6af3e1e234fd870907a84dcf1e81933302721de.

Capture manifest 5dc9686af2c09fe9cb0f33e9d85804a8621d41d46911ebb73bc78faa4772e9c7 authenticates20 retained files; inventory bb1a4061b0f245943f195f39754b0a4646bdd99edc383d8bf99df281144fe7d2; receipt aa13a9dbddb888cc82e48e3c970f9af40832e6a098c36d85df0528272c32cf4c. Parent verified frozen inputs, all captured bytes, source-node equality, allocation and queue absence. Independent intake inspection confirms the same facts. Only static ELF dependency names are retained, not a completeOS/driver closure.

Actual help/manuals support the intended existing whole-Graph/kernel-replay, one-launch, start/stop, cache-none/clock-none/stable-boost and profiling/stats query controls. This is CLI metadata qualification only: device metric availability,counter permission,injection and actual profiling remain untested. Prospective profiling-v2 will explicitly bind this3.0 version and its node, preserve ordinary capture before NCU, and receive a new exact review/freeze before GPU activation. No workload,numerical or metric policy change is introduced.

Allocated2 CPU-seconds,zeroGPUcost. Search remains21583actual+0reserved/28800,7217remaining. The earlier local-only visibility source was corrected before the successful run; frozen2990 failure is unchanged. Both engines remain non-speculative; formal performance/quality remains INCONCLUSIVE.

Prospective current-engine Graph diagnostic v2 is frozen after actual-node tool qualification; harness plane, both engines non-speculative.

Freeze 9b2f691153ee8a5cb223bfa80e92d89682e1904a7b59ffc1029a559e47ea34d7 binds merged44ca6732f466be233e1ef06ecbde1813b4de5002/treebe17d2be5ed92a71df7d2ca405ec769495c36226, unchanged2970 options/source/numerics,48 immutable runtime payloads and10 unchanged native artifacts. Author manifest e50b4e4460bf325407b1a302a518f9f9a930bb7b92159fcfb6c572b8745255ac; payload f40aa83cdbece9510afb8d5eb35ba7e9d2306ef9cbfe18bd986b22b1289c5a02. Independent runtime review ea1ef96827e24c6eca40b07ce4cbce7154e00092274d8d676e0979bd1f6ddd98 and helper review 10f74eb7029a9d4c58c5fdf752172d6cf51a879522c19f630da9844259fc7a74 found no blocker. Helper manifest64d3e1e8e24a585773410541788cb37adfd54b5f54cdbf36f2f2e4d89e3e0ca6; parent authenticated actual composition.

This is a new bounded diagnostic, preserving failed2990. Ordinary8-append event timing and separate8-append Torch trace now complete before NCU preflight; file errors retain their offending identity. Explicitly bind actual2995 Nsight Compute2025.3.0.0/build36273991/ELF064645ab, exact67-file inventory and observed node. Node constraint is required by the local tool installation. No silent fallback or substituted metrics. Same fixed n8192-s0,10240 prewarm,one preparedGraph atposition8199 for whole-Graph/kernel-replay counters. Full BF16 vocabulary/state/history equality, unchanged normal production builds and postguards remain required. Unsupported metrics/denied counters terminate without retry; separately complete ordinary evidence remains usable under the normal-only audit, with overall INCONCLUSIVE.

Reserve900 GPU-seconds now: 21583 actual +900 reserved /28800;6317 unreserved. OneGPU/8CPU/96GiB/900seconds,no-requeue;780-second phase bound+120margin. No vLLM rerun or formal acceptance is included. Counter timing/traffic are profiler observations, not production or per-operator gains. Exact workload/device metric support,injection and permission remain to be tested. Both25%/5% performance and quality acceptance remain INCONCLUSIVE.

Diagnostic2998 ended FAILED1:0/102s before normal or NCU workloads. The exact dependency log identifies a packaging omission: unchanged qk_runtime_guard.py reads primary-source/provenance.json, but that file was absent from the frozen diagnostic package. Installed Triton drift was not established; the expected-source file could not be read. The earlier source review missed this read dependency. Declared manifest verification alone did not establish complete file-read closure.

All266 captured payloads authenticate: archive 554a5f7948540d1869468dc7f53a76a7688472fb8068eb0638625b7f18b4f625 (3775552B), manifest cbb473df107fa9feced07278767925ba0923224ef9e06dfaea6750b7e1748780,zero exclusions. Independent review 50d3db4ad59f46ffb020fb96e116876895afb77bc13ccf145295e9e3f3592d7b, integrity 7715a0f05843655b864c6bdde5cc4f3b1d6c0484dd9074bb76e98ceac32f4d8b. Model/tool pre/post snapshots agree and applicable finalizers completed. The dependency-before group only partly ran; dependency-after did not run. No owned model/profiler child or library build started. Normal/state/counter/performance outcomes are NOT_RUN, overall INCONCLUSIVE.

Allocation released; charge102GPU-seconds once/release798 reserved: search21685 actual +0 reserved /28800;7115 remaining. No retry of2998. The original freeze/evidence remain unchanged.

Separate minimal v3 preparation restores the exact317-byte provenance file from the authenticated2970 donor without changing its expected compiler hash or any model/workload/metric policy. Before another GPU allocation, run the full dependency metadata group with zero GPUs and inspect adjacent literal read dependencies. This repairs packaging, not numerical/performance behavior. Both engines remain non-speculative; final25%/5% and quality acceptance stay open.

Prospective zero-GPU dependency preflight before any next GPU attempt. Freeze f5a9f6289dca2f7cb2a55e43eec00993c242a6abe35dc6101a6458ca5fb3036b binds v3 author 0efd177a669130320132e41397d5b412f920e86a70a942cbe5034c5f92bcab8b and immutable49 payload 03a5983091da8c6095e6e047cdfd5b7678d537fc9b851269d2063760c877ffb9, including the exact restored317-byte compiler provenance. All Python/shell/runtime behavior is unchanged fromv2; only this missing input and budget metadata were corrected. Source review5741ce94 confirms the bounded repair, not runtime correctness.

Stage exactly51 package members (49payload+payloadinventory+authormanifest) and10 unchanged native artifacts in a fresh task. On the known tool node, run the unchanged full six-check dependencies-before group (20s), compiler-file guard(10s), and NCU file-only snapshot(10s), with5s termination grace each. One60s/2CPU/2GiB/zeroGPU/no-requeue allocation. Clean replacement environment matches the GPU harness; preserve Slurm visibility. No model initialization, native load, NCU executable, metrics or fabricated probe. Historical accepted-FA3 evidence remains historical. Inputs checked before/after; partial/timeout results stay incomplete.

CPU author manifest fc8de77b48bdd8f20d7a963ed49c2d7cc9fe54fbf7fadea2cb987c28ac68d555; independent source review 97f8ec13329e8072988077a40e73e391956d91bb6350a69a65fb7abc0ca9f872 found no blocker. Parent inspected the actual sources and bounded staging/submission. This is a separate metadata attempt, not retry2998. GPU ledger remains21685actual+0reserved/28800,7115remaining. ActualCPU success is mandatory before prospectiveGPU qualification; no GPU reservation yet.

CPU dependency preflight3005 completed successfully under freezef5a9f628: COMPLETED0:0/5s/2CPU,zeroGPU TRES,no requeue/restarts,released. All three phases returned0: the entire unchanged six-check dependencies-before group,compiler-file guard,and NCU file snapshot. All eight required receipts exist,all51 staged package/10native files remain unchanged,and the restored compiler provenance matches the actual installed compiler.py digest. This closes the specific omitted-file prerequisite without a GPU retry.

Capture manifest f49881317e81e52b0210c1df44a84b1bf61026c10fb074dfaeaa8e5ed9f93052 authenticates84 files. Completion ddbf9e85bd3191653c26e6ba8fb33c0eebb07ab4338f23465c8e1ac05f30254c, terminal 9dc47eea213fbaeb13f6590cadf3e0c02b514cfcef05e2e6d80890301b906c65. Parent verified actual artifact and terminal evidence; independent outcome review ced5339b34d75f7ccfad93e0dd70d3962f41fcc647d1e0fad42617d62a1c7a65, integrity ee0b1ef301e1df78fb2aadbee4e3205b9e73164ab9a58365061dfae9e8faab3a agrees. All67 NCU packaged-file identities are available on the observed node; no NCU executable,model,native load or performance measurement was performed. Historical acceptedFA3 evidence is not a fresh library-load result.

Allocated10CPU-seconds,zeroGPUcost. Search remains 21685actual+0reserved/28800;7115remaining. Prospective v3 helper now binds this real CPU success before qualification; no GPU reservation/activation yet. Source/runtime/numerical/metric policy stays unchanged except the restored authenticated provenance input. Formal performance/quality remains INCONCLUSIVE; both engines non-speculative.

Prospective diagnostic v3 freeze after successful real CPU3005 preflight: a5f9a807ede4f8f6a286d6e7fa079543920ccf937009c79629cbfbd725f72449.

Author0efd177a669130320132e41397d5b412f920e86a70a942cbe5034c5f92bcab8b/payload03a5983091da8c6095e6e047cdfd5b7678d537fc9b851269d2063760c877ffb9 include49 immutable files and the exact restored compiler provenance. Runtime Python/shell,merged44ca6732/treebe17d2be,2970 options,numerics,workload,metrics and native10 artifacts are unchanged. Runtime review5741ce94866978434ee9560a517b7a57af2060eb824132618a11ae7c0bb48d88 and final helper review8a2a8f883336ee376c5d1faf32df6101c89bcb32297c1c6614e6f30a8dc62483 found no blocker. Parent authenticated actual composition. Helper73a0f829afa2c2dafa8b367cd454449f3f97944b24cf492d6c545e00e16e5048 requires exact captured CPU3005 success before prepare/activation; its binding is in this freeze.

Reserve900 GPU-seconds now: 21685 actual +900 reserved /28800;6215 unreserved. OneGPU/8CPU/96GiB/900s,no requeue,one attempt on the tool-qualified node. Ordinary8-append event/trace collection precedes the optional fixed whole-Graph NCU collection, with unchanged complete state/logit/history checks,normal production builds,postguards and owned cleanup. Missing/unsupported counters retain ordinary evidence without retry. No vLLM rerun or formal acceptance in this diagnostic.

Prior2990/2998 failures remain frozen and settled. This v3 admission is based on the authentic missing-input restoration plus observed CPU dependency success, not a weakened guard. Both engines remain non-speculative; formal25%/5% performance and quality acceptance remain INCONCLUSIVE.

evidence: retain approved 8K search and FP8 results (#97)

acceptance: Qwen3.8 FP8 vs vLLM 0.32.0 — ≥25% TTFT or TPOT gain, ≤5% regression #67

Description

Current owner amendment — 25% latency improvement with a 5% regression guard (2026-10-24)

Current owner amendment — 8K-only acceptance (2026-10-23)

Authority, parent and lifecycle

Accepted scope and comparison identity

Performance protocol accepted by the owner

Task-quality protocol accepted by the owner

Numerical diagnostics and long context

Allowed optimization changes

Budget, readiness and stop conditions

Freeze checklist

Evidence and publication

Current execution delegation (2026-10-21)

Readiness checkpoint (2026-10-21)

Activity

jiangkuaixue123 commented on Oct 22, 2026

轮次总账（由 InferRSI 分析维护，每轮追加一行）

当前缺项（不得补猜）

目标线（据此更新）

jiangkuaixue123 commented on Oct 22, 2026

总账修订 v2（补齐 sha / 作业号 / 卡 UUID；写入硬账判定与精度臂规则）

硬账判定（写进性能计划，不得简化）

精度臂规则（新增，硬约束）

知识条目状态

jiangkuaixue123 commented on Oct 22, 2026

总账 v3（目标线 9.87 / 9.74；FP8 基线 run 入账；新增环境失败行与代理口径警示）

1. 目标线统一

2. 新增/修订行

3. 证据包核验（InferRSI 侧复算）

4. 代理口径警示（从 1646 原始请求读出，必须随靶线一起引用）

5. 达成度的前置条件（写死）

jiangkuaixue123 commented on Oct 22, 2026

总账补充：集群 commit ↔ PR 映射（patch 重放，非原 commit 推送）

jiangkuaixue123 commented on Oct 22, 2026

总账 v4：R11/R12/R13 的 PR 列复核结果（未能按映射填入）

R11/R12/R13 其余列（按群里最新事实）

阶段出口（Chief 定的四阶段，记账用）

jiangkuaixue123 commented on Oct 22, 2026

总账 v5：PR 归属修正（真链接已核验）、新增 TRADE-OFF 判定、靶线改合同口径临时线

1. PR 列修正（已实际打开链接核验）

2. 判定列新增 TRADE-OFF，R2 补判

3. 靶线改为合同口径（临时）

4. 两个未闭合项（决定 PR 能否转正）

5. 判决资格前置

jiangkuaixue123 commented on Oct 22, 2026

预注册：prefill 三步出口门（在代码落地前冻结）+ 两处算术必须先定

1. 判据更正：279 不等于 280

2. 三步出口门（现在冻结，测完只填结果）

3. "持平"这条目标，①–③ 走完也到不了

4. 记录

5. 记账口径

jiangkuaixue123 commented on Oct 22, 2026

总账 v7：选中的库离 go 线差 2.5%、decode 结论翻转、warmup 剔除入约定

1. 280 TFLOPS 这条线：两个选项都没过，必须按"未达标"记

2. decode 的权重流：结论翻转，但仍要给实测

3. 数值语义声明（PR 必写）

4. 环境：坑扩到六条，登记表待核验

5. 约定新增（已写入记录契约）

jiangkuaixue123 commented on Oct 22, 2026

状态更新 2026-10-22：decode 侧三件合入、内部工程门变更、数值诊断口径

1. 候选侧代码身份更新（candidate 从占位符变为具体 SHA）

2. 与 #67 验收目标的关系（本段不是性能结论）

3. 内部工程门变更（仅约束我们自己的合并动作，不改变 #67 门）

4. 新增启动成本（部署口径变更，必须记录）

5. 数值诊断口径（按本 Issue "diagnostics" 定位执行，不作 gate）

6. 已知限制

7. 未解决项

8. 授权

jiangkuaixue123 commented on Oct 22, 2026

数值判据拆成两级（改判记录，2026-10-22）

触发

新判据

已按新判据处理的项

C1a 尚存的矛盾（未解决前不下结论）

两处此前陈述的更正

第三层现状（对 vLLM 0.32.0 + Qwen3.8-27B-FP8 单卡）

机器与偏离

jiangkuaixue123 commented on Oct 22, 2026

C1a 矛盾已解，并新增一条方法学约束：算子级门必须用真实激活

C1a 已合入（一级通过），候选身份更新为 `83b567c`

第三层 32 步版（`prompt_logprobs`）：结论仍为不可互换，且尾部发散被暴露