docs: record Open-Jev PR-by-PR optimization A/B results

.agents/skills/write-optimization-blog/SKILL.md

@@ -0,0 +1,130 @@
+---
+name: write-optimization-blog
+description: Write or update Markdown engineering blog posts with PR-by-PR inference optimization progress, isolated A/B records, implementation explanations, and sourced reproducible comparison figures.
+---
+
+# Write an optimization blog
+
+Turn implementation and experiment records into a readable engineering post.
+Explain the bottleneck, the change that addresses it, and the measured result.
+Keep measured behavior distinguishable from implemented support and future work.
+
+## Choose the destination and evidence
+
+Inspect the target repository's blog/docs conventions and contribution rules.
+Use its existing Markdown, metadata, image paths and navigation. When the user
+names another blog as a reference, study its presentation without silently
+changing the publication destination. Read
+[the vLLM examples](references/vllm-blog.md) when using that style or contributing
+to `vllm-project/vllm-project.github.io`.
+
+Record the inspected repository SHA, the post's evidence cutoff and the actual
+measurement revisions. Current main can contain subsequent changes that were
+never timed. Prefer immutable links to the measured code and raw records; link
+current recipes separately for setup. Disclose missing raw artifacts rather
+than implying every historical result can be reproduced from current main.
+
+Build a compact evidence ledger before drafting. For each result, retain:
+
+- Experiment identity/date and baseline/candidate revisions or artifact hashes.
+- Hardware, model/checkpoint, dtype, workload/token lengths, concurrency, cache
+  and graph settings, and the timed boundary.
+- Excluded preparation, readiness, first inference and warmups; measured run
+  counts and per-run values; output-fidelity checks and acceptance criteria.
+- The source of the values and whether they are raw measurements, a documented
+  summary, an inference, or a correctness-only check.
+
+Use existing authorized records. A writing request does not authorize a new
+benchmark campaign. If raw data is unavailable, a published summary can support
+a labeled historical figure; preserve its source and displayed precision.
+
+## Write the engineering story
+
+For a concise post, lead with headline numbers and a comparison figure. Let
+figures and compact tables carry the results; give each change a short mechanism
+explanation. Avoid repeating the same results in a figure, table and paragraph.
+Put full protocols, source hashes, per-run tables and rejected-variant detail in
+linked evidence records. Keep workload/timer labels, regressions, missed gates
+and material numerical limits visible in the post. Follow the user's requested
+depth; keep the full evidence template in the linked record.
+
+Give enough model/request-path context to understand what is accelerated. Connect
+each observed cost to the implementation and its measured effect. Include rejected
+approaches when they explain a precision constraint, cache policy or unresolved
+limit. For compact presentation examples, see
+[OpenJev-Fast alongside the vLLM references](references/vllm-blog.md#compact-reference-openjev-fast).
+
+Separate complete backend comparisons, isolated optimization A/B runs, kernel
+microbenchmarks and correctness checks. Do not add independently measured gains,
+draw a cumulative speedup ladder from separate campaigns, or treat a faster
+kernel as a measured end-to-end improvement. Keep different hardware/workloads
+in distinct comparisons. Mark unmeasured combinations explicitly.
+
+Preserve numerical limits: matching threshold decisions does not establish
+identical probabilities; a small subset's score is not general accuracy; two
+run means are variability observations rather than confidence intervals.
+Explain a missed acceptance criterion alongside a favorable timing.
+
+Use implementation and PR links near the relevant mechanism. Adapt the section
+structure to the material; a performance-first post, a debugging narrative and
+a stage-by-stage walkthrough need different emphasis. Verify quickstart commands
+against current recipes and separate implemented features from the target design.
+
+## Maintain a PR-by-PR progress record
+
+When adding or revising a progress update, use
+[the per-improvement A/B record](references/pr-update.md) for the detailed evidence;
+summarize it in the article with figures and links.
+
+Include an experiment map linking each relevant PR to its implementation changes
+and A/B evidence. Split independently tested improvements within one PR into
+separate updates; a PR containing normalization, activation and graph changes
+needs more than one aggregate before/after row.
+
+For each update's evidence record, keep the date, bottleneck, implementation
+mechanism, isolated variable, measured baseline/candidate revisions or source
+hashes, fixed controls, timer, run budget and exclusions. Show each measured run for both arms, relevant
+kernel and serving results separately, numerical/parity checks, acceptance gates
+and their outcome. Link raw samples, the frozen protocol and reproduction method.
+Preserve regressions, missed gates and rejected variants alongside improvements.
+
+Label regression checks as such; small nominal differences do not establish a
+speedup. When an improvement lacks an isolated A/B, mark that gap and define the
+missing comparison instead of assigning it part of a bundled backend gain.
+Obtaining new measurements still requires the user's experiment scope and budget.
+
+Append dated updates as PRs land without replacing earlier measurements with
+current-head claims. Retain distinct hardware, workloads and timer boundaries.
+Snapshot compact measured samples and source hashes when the underlying report is
+local or a PR description can change; state what remains outside the public record.
+
+## Make figures from the evidence
+
+Use standard plotting tools for quantitative figures. Save the input values and
+their provenance beside a CPU-only regeneration script. Derive aggregates from
+raw records when available; otherwise label values taken from a summary/table.
+Keep exact inputs in the data file and round only for display.
+
+Use one coherent experiment per chart/panel. Label units, request counts,
+hardware and warm/cold state. Bar axes start at zero; label any zoom or log axis.
+Show individual run values when available and explain what markers represent.
+Use readable colors, alt text and captions; export SVG for sharp embedding and
+PNG when a raster preview or sharing format is useful. Inspect the rendered
+figures and the Markdown/site, including labels, clipping and source links.
+
+Mechanism diagrams should identify real components and data movement. Do not
+invent GPU timeline spans, occupancy, stalls or bandwidth diagnoses from wall
+times. Give profiler limitations where they affect the interpretation.
+
+## Check and hand off
+
+Check every headline against the ledger and recompute displayed ratios. Ensure
+links/images resolve, scripts run without accelerator use, metadata matches the
+target, and the site's normal documentation build passes where available.
+Keep plot dependencies separate from inference dependencies. Run checks relevant
+to the change instead of unrelated model/GPU tests.
+
+Deliver the skill/post paths, figures, validation results and evidence gaps.
+Commit, open a PR, publish or contact others only within the user's authorization
+and repository workflow; this skill grants no additional permission. Preserve
+contributor-owned review checklists when preparing an agent-assisted PR.

.agents/skills/write-optimization-blog/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Write Optimization Blog"
+  short_description: "Write PR-by-PR optimization posts with A/B evidence"
+  default_prompt: "Use $write-optimization-blog to write a concise Markdown post led by measured results and figures, with short mechanisms per PR improvement and linked A/B records containing frozen controls, repetitions, parity and raw samples."

.agents/skills/write-optimization-blog/references/pr-update.md

@@ -0,0 +1,57 @@
+# A per-improvement progress update
+
+Use this record for the evidence behind an optimization update. Omit
+inapplicable metrics. One PR with independently tested changes
+gets separate records; a bundled backend comparison remains a bundled result.
+
+A concise post shows the headline, the A/B figure or compact table, a short
+mechanism and the result/limit.
+Link the full record for controls, repetitions, hashes and rejected variants;
+avoid reproducing every field in the main narrative.
+
+## Date, PR and implementation change
+
+Link the PR and measured implementation. Explain where the previous path spent
+time, the operation or data movement that changed, and the hypothesis. Describe
+the concrete before/after execution steps, rounding points and fallbacks that
+matter to the result. Keep profiler observations distinct from inferred causes.
+
+## Frozen A/B pair and protocol
+
+Record baseline/candidate commits, patches or artifact hashes. Identify the
+isolated variable and necessary consequences. State the fixed exact device,
+model/checkpoint, dtype, workload/token lengths, concurrency, affinity, graph and
+cache settings. Give the timer's start/end boundaries and whether profiling was
+active. Link the frozen protocol, preparation/reproduction commands and raw runs.
+
+Record feasibility, first inference/capture, warmups and measured repetitions
+separately, including arm order, success criteria and the declared stop condition.
+Use the recorded protocol's run count; an ordinary two-run comparison shows both
+runs per arm. Preserve failures and stop at the agreed budget.
+
+## Results and acceptance
+
+Use a table that exposes repetitions, rather than only an aggregate speedup:
+
+| Metric and unit | Baseline measured runs | Candidate measured runs | Aggregate change / variability | Declared gate / result |
+| --- | --- | --- | --- | --- |
+| Relevant kernel/call metric | Each run's value | Each run's value | Change and observed spread | Pass, miss or unverified |
+| Warm HTTP mean / p95 / throughput | Separate row for each relevant metric | Each run's value | Change and observed spread | Evaluate the original gate |
+
+Keep standalone kernel timings, trace-family totals and complete HTTP latency in
+separate rows or tables. Explain whether the gate applies to each run or to an
+aggregate. Report numerical drift/tolerances, decision flips, failures and tested
+sample counts alongside speed. Matching decisions alone is not probability parity.
+Identify the acceptance result even when a kernel improves but HTTP misses its
+gate; small refactor differences can support a regression check without a speedup.
+
+## Evidence, limits and next comparison
+
+Link raw samples, source hashes, measured code and figure inputs/regeneration.
+State which full responses, traces or protocols remain outside the public record.
+Figures should show the relevant repetitions and label their timing boundary.
+
+Finish with what this pair establishes and the next unresolved comparison. Mark
+missing isolated A/B evidence as unverified and name the variable to isolate.
+Do not carry an older measurement forward as current-head performance, add gains
+from independent campaigns, or fabricate a stage timeline from aggregate times.

.agents/skills/write-optimization-blog/references/vllm-blog.md

@@ -0,0 +1,55 @@
+# vLLM blog references
+
+Reviewed on 2026-10-05 at blog repository revision
+`f9792a2e4f2b65c5ef3291396f941a1d831cc6b1`. These are presentation examples,
+not benchmark evidence for another project. Refresh target conventions when
+preparing a later contribution.
+
+## Different useful structures
+
+- [Qwen3-Omni optimization](https://github.com/vllm-project/vllm-project.github.io/blob/f9792a2e4f2b65c5ef3291396f941a1d831cc6b1/_posts/2026-07-01-qwen3-omni-optimization.md):
+  introduce the execution path before individual optimizations; tie each change
+  to a concrete bottleneck and implementation; place configuration sweeps and
+  mechanism illustrations near the discussion. Its incremental sweep is valid
+  because successive configurations were measured with earlier changes enabled.
+  Do not imitate that cumulative table when your experiments are independent.
+- [Kimi K3 performance](https://github.com/vllm-project/vllm-project.github.io/blob/f9792a2e4f2b65c5ef3291396f941a1d831cc6b1/_posts/2026-09-13-kimi-k3-performance-optimization.md):
+  start with serving results, then select representative scheduling, state/cache,
+  data-movement and kernel changes. A before/after illustration can explain the
+  removed operation more clearly than a large diff. Link implementation PRs and
+  the wider tracking issue without turning the article into a PR inventory.
+- [Kimi K3 DSpark](https://github.com/vllm-project/vllm-project.github.io/blob/f9792a2e4f2b65c5ef3291396f941a1d831cc6b1/_posts/2026-09-15-kimi-k3-dspark.md):
+  relate the headline to a defined workload and show performance alongside the
+  mechanism, hardware setup and practical workflow. Keep throughput, per-user
+  responsiveness and latency distinct rather than substituting one for another.
+- [Kimi K2 tool-calling accuracy](https://github.com/vllm-project/vllm-project.github.io/blob/f9792a2e4f2b65c5ef3291396f941a1d831cc6b1/_posts/2025-10-28-Kimi-K2-Accuracy.md):
+  a debugging narrative can organize the post around concrete compatibility
+  failures and fixes. Preserve the validation context when explaining why a
+  result changed; performance work also benefits from recording rejected changes
+  that violate the output contract.
+
+## When the destination is the vLLM blog repository
+
+Read its current [README](https://github.com/vllm-project/vllm-project.github.io/blob/f9792a2e4f2b65c5ef3291396f941a1d831cc6b1/README.md)
+and a nearby post before adding files. The reviewed conventions are:
+
+- Markdown under `_posts/YYYY-MM-DD-slug.md`.
+- YAML frontmatter with `layout: post`, title, author, tags, and a specific
+  `summary` of at most 240 characters. Add image/social-image fields only for
+  actual included assets; use verified authorship and the intended publication date.
+- Figures under `assets/figures/<post-slug>/`, with captions and useful alt text.
+- Build with the repository's Jekyll/Bundler setup. The main-branch workflow
+  deploys the blog, so drafting files or a PR does not itself mean publishing.
+
+For a different repository, adopt its renderer and relative-link conventions.
+Do not add Jekyll or migrate a documentation site merely to mimic these examples.
+
+## Compact reference: OpenJev-Fast
+
+Reviewed [OpenJev-Fast](https://yiqilyu.me/open-jev-fast/) on 2026-10-05 for
+presentation: headline metrics, an optimization chart, short mechanism blocks,
+a benchmark figure/table, and brief correctness/setup notes. Detailed experiments
+and failed attempts are linked in a separate report. Use that separation when
+the user wants figures and numbers to carry the story. Its B300 forward/head
+chart and HTTP benchmark use distinct timers; its values are not evidence for
+System1-Omni's H200 experiments.

README.md

@@ -161,6 +161,9 @@ See the [GPU serving benchmark](benchmarks/README.md) for request replay,
 output-fidelity checks, and the CUDA comparison protocol. The
 [Open-Jev H200 results](recipe/open_jev/validation.md) cover 74 single-candidate
 requests and a matched comparison with raw HF Transformers and OpenJev-Fast.
+The [Open-Jev optimization notes](docs/blog/2026-10-05-open-jev-optimization.md)
+record PR-by-PR Rust/CUDA changes and isolated A/B measurements, with figures,
+numerical checks and links to the separate experiments.
 
 ## Roadmap