[Levius-Fubuki]

Required human contributor answers

The contributor explicitly reviewed and confirmed these answers and PR necessity in the implementation task on 2026-10-25.

1. What outcome are you trying to achieve?

   Reduce repeated language computation when Cua-S1 answers several independent questions about one screenshot. Run the common multimodal prefix once without changing the prompt or question semantics. First scope is request-local, opt-in, serial branching; no cross-request cache or batching.

2. What is the highest-leverage variable for that outcome?

   Recomputed language-prefix tokens and their execution time. Measure complete-request p50/p95 by1/2/4/8 questions, GPU memory and the single-question overhead; account for capture/restore and fixed-GEMM costs.

3. How will this work move that variable rather than create busywork?

   The screenshot precedes question text in the current prompt. Reuse its attention K/V, GDN recurrent state and convolution history, then execute independent suffixes. First compare against complete forwards using the same fixed GEMMs; separately measure ordinary-default drift and performance, and preserve independent execution where sharing is not useful.

4. Who is the customer or user, and what do they actually need?

   Agents making multiple decisions about one screenshot, such as choosing an action and checking state/completion. They need faster multi-question responses with stable question order, independent answers and unchanged probability/accounting semantics. Maintainers need explicit reuse boundaries, isolation and failure-recovery evidence.

After answering the four questions: is a PR needed at all to achieve this outcome?

Yes. The contributor confirmed this scope and requested submission after validation. It connects existing backend continuation to Cua-S1 with model-level tests, lifetime checks and recipes.

For a bug fix: link actual observed failure evidence.

Core scope is a performance capability. During development, independent review identified malformed public prepared-input gaps; logs/prefix-malformed-red.log and logs/prefix-image-index-red.log in the qualification package reproduce them before the final guards. They are fixed and covered by the final green/recovery test, not claimed as pre-existing main-worker defects.

Purpose

Reuse the aligned screenshot token/3D-position prefix once per request, then execute independent question suffixes. The new opt-in CUA_S1_MM_PREFIX=matched uses the ordinary full-prompt GEMM algorithms, avoiding the old fixed-M64 mixed-latency penalty on the measured workload while preserving ordinary outputs.

For each projection, admission checks the complete algorithm descriptor of every full question and support/workspace for every prefix/suffix window. A mismatch or unsupported window falls back to ordinary independent full forwards. Single questions/no aligned prefix also use the ordinary path. Allocation/setup/execution errors remain errors. Descriptor equality is admission, not a universal cuBLAS numerical guarantee.

off remains the default; fixed/shared retain their original M64 arithmetic. All active modes require host features and language Graph off; vision Graph is independent. The matched handle cache has at most16×32MiB workspaces, drained before eviction/destruction; scope selection is restored after errors and panics. Snapshot allocation is retained, but every request captures fresh contents. No cross-request screenshot-result cache, batching, automatic policy, scheduler change or new precision setting.

The backend adds optional ABI9 symbols; existing modes do not resolve them. Rebuild the CUDA library to use matched mode. CPU letter head, request/answer order, prompt construction and usage accounting are unchanged.

Test Plan

Current PR base: 28f4957a6087d23baec6341ac689dda87119c799. Original performance base: 05075d0f36e399e3a001ffedea400c666990cda9. Previous PR head/control: d4908d95c3bb9e1e96e4ad64cff916c1dcfda7c8. Current head: 9e353b08741d942904cd49f175691e62d4a67ccd.

• Ran cargo fmt --all --check; cargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warnings; cargo test --workspace --locked --features omni-laya/serve; cargo build --workspace --release --locked --features omni-laya/serve; strict MkDocs; CUDA contract checker and backend/checker tests.
• On reserved RTX4090/SM8.9, driver595.71.05,450W, CUDA13.0/cuBLASLt13.1.1.3: freshly built baseline and candidate CUDA libraries/workers with pinned BF16 merged language, original BF16 vision and FP32 vision adapter; source/binary/model hashes recorded.
• Ran all19 ignored root CUDA kernel tests, including320 matched projection-window comparisons; ran all3 prefix model regressions with vision Graph0 and1. Checked actual admitted sharing, changed screenshots/geometries,1/2/4/8 questions, fixed/shared/default preservation, malformed inputs and suffix-error recovery. Host tests cover later-question/window/projection mismatches, unsupported windows, bounded workspace retirement, error/panic recovery and no synchronization omissions; four temporary mutants were caught. The final scope test also passed on Linux.
• Expanded qualification: five profiles each run80 frozen real/stress layouts twice plus32 single-question extractions (960 calls), with public prepared-input recovery probes. Require exact ordinary-profile logit bits, inputs and responses; M64 is a separate numerical control.
• Formal performance: concurrency1, direct native-worker HTTP POST through full JSON/schema validation. Two visits per configuration in balanced O/S/C/C/S/O order per suite, four traversals per visit. Per worker, exclude one all-shape growth traversal, one stabilization traversal and5warmups; retain all raw/excluded samples. One feasibility comparison per profile is separate. Synthetic8cases/64 measured requests per profile; real12screenshots/96 per profile,480 total formal timings. Vision-cache8entries/1024MiB for synthetic,16entries/2048MiB for real; these are controls, not default recommendations.

GPU numerical/performance qualification and its source-identity mapping are pinned to the optimization revision 02d75b3b41423dc8d61b23e2fd508f6095a2e388; its 133 production files and five relevant test files matched the server sources. The current head adds the startup-symbol guard below. No fresh GPU numerical or latency run of this guard is claimed; the server remains shut down.

Startup capability fix

926855967 fixes a readiness gap: a selected matched worker with an older ABI9 library could pass its single-question startup warmup without resolving the matched symbols, report ready, then fail its first shareable multiquestion request. Both symbols now resolve before the non-sharing early return, so warmup rejects an incomplete library before health binding. No matched context/workspace is allocated for this check. Kernel math, algorithm selection, prefix-state handling and scoring are unchanged.

The existing host-only loading regression independently omits cs1_gemm_create_matched and cs1_gemm_compatible in subprocesses, checks the rebuild/symbol error, no extra GEMM allocation, ordinary/fixed dispatch compatibility and cleanup. It failed on the previous implementation (matched warmup accepted an incomplete CUDA library: None) and passes after the guard. Full required Rust/Clippy/release/docs/contract checks were repeated: 365 passed,0 failed,74 default ignored. This fix changes three existing files; no dependency or new source/test file was added.

Startup fix red/green evidence, SHA256 44d0aa8b4a362d54ea3d1cd929e5aab555307ec650092072a4a97b846c9dc938, includes the exact patch, commands, logs and revision pins. The following GPU and latency results describe the earlier 02d75b3b4 performance source, not a remeasurement of this startup fix.

The feature branch was subsequently synchronized with main 28f4957a6 (upstream PR #186); 9e353b087 retains both the matched kernel regression and all upstream test-module registrations, resolving the append-only test conflict without rewriting history. Relative to current main the PR still changes16files. Source review confirms the new OpenJev options remain disabled in Cua and the continuation still uses its selected fixed/matched GEMM handle. After synchronization, required Rust checks passed again: 375 passed,0 failed,86 default ignored, plus strict docs, CUDA contract and26backend Python tests. No merged-head GPU remeasure is claimed. Integration validation logs and exact-head pins, SHA256 6f256ca0f3e2ce08e51a019747ca90ce04768f53997e5d0d3b685ee75614f111.

Test Result

Local final suite: 375 passed,0 failed,86 default ignored, including doctests; strict Clippy, build, fmt, docs and CUDA text checks passed. Local ignored GPU tests are not counted as execution. GPU regressions at 02d75b3b4: 25 passed,0 failed.

All960 qualification calls succeeded. Candidate matched/off raw FP32 logit bits, complete responses, token IDs and all three position axes equal the new ordinary baseline on every full/repeated/single case. Candidate shared preserves the original M64 results. Single-question extracts equal their multi-question answers. Recovery probes pass. All formal HTTP response bytes match within the ordinary or M64 numerical profile.

Workload / mode	timed requests	mean ms	p50 ms	p95 ms	sampled max MiB
synthetic / ordinary off	64	133.30	119.31	277.13	10138
synthetic / previous shared M64	64	134.73	139.35	215.54	10288
synthetic / matched	64	107.17	103.24	173.69	10448
real / ordinary off	96	256.36	265.33	271.98	10210
real / previous shared M64	96	205.53	209.87	215.85	10318
real / matched	96	152.83	152.94	160.08	10606

On the frozen real suite, mean latency falls 25.6% versus previous shared M64 and 40.4% versus ordinary off; p95 falls25.8%/41.1%. Paired-workload-round bootstrap95% mean-reduction intervals are25.1–26.1% and39.9–40.7%, descriptive of this repeated timing rather than task-population generalization. Synthetic mean reductions are20.5%/19.6%; three small-case means regress0.4%/4.8%/0.1% versus off. Per-case results and all exclusions are preserved. Real-suite sampled maximum grows288MiB over M64 and396MiB over off; memory is ~100ms process sampling, not an exact allocator peak. No formal timed Graph capture/failure or foreign GPU process occurred; matched admission/fallback counts were not instrumented.

Quality and support limits: the adapted closed-candidate task over48 pinned ScreenSpot-v2 screenshots/192 correlated choice permutations scores119 for off/matched and120 for M64. Matched preserves ordinary outputs, not the different M64 profile; no accuracy improvement or official ScreenSpot/interactive task success is claimed. The earlier off-versus-FP32-reference high-margin winner gate failures remain unresolved and were not relabelled passing. Other GPUs, libraries, concurrency, long-term working sets and cold-start/throughput benefits need separate qualification. Keep all modes opt-in/default off.

Observed CI on current head 9e353b087: Rust, CUDA contract, benchmark checks, CUDA compile and documentation build all succeeded; deployment was skipped. CI run, documentation build.

Demo / evidence

2026-10-26 matched qualification, SHA256 4ca69525da3447ebbecc0ccef78c8291fc887e4645b384018247279240044f04. Includes exact initial source archives, accepted-source patch/mapping, binaries/libraries and model pins, raw/excluded/feasibility HTTP samples, full logits/input/response outputs, error-recovery and red/green/mutation logs, per-case/memory results, offline audit and reproduction/corpus acquisition scripts. Original real screenshot pixels, model weights and credentials are omitted. CUDA dependency hashes accompany the release. Video: N/A for this execution change.

Earlier qualification preserves initial malformed-input/allocation red/green evidence and the failed reference-quality gates; it is historical evidence, not a fresh run of this head.

Self-review

Full diff reviewed against the base: 1039 authored-code changed lines,1231 total. Model-specific planning/head logic stays under Cua-S1; reusable CUDA operations and shared Qwen execution retain their ownership. No remaining actionable agent review finding; tests were strengthened for later-shape admission and synchronization ordering, then relevant checks repeated. This is contributor evidence, not a replacement for the human's full-diff attestation or maintainer review. The checklist below remains for the contributor's own attestation.

• I have reviewed the full diff and addressed the issues I found.
• I have checked that the change follows the project's architecture and stays focused on the stated purpose.
• I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
• I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.