[hsliuustc0106]

Required human contributor answers

Answer every question before implementation and PR submission, including drafts.
Reuse explicit, still-applicable human answers with a traceable source reference;
ask the human for anything missing and wait. An agent must not invent answers.
Blank fields, placeholders, or an agent-written rationale do not satisfy this gate.
Share only information and source links appropriate for this PR's audience.

1. What outcome are you trying to achieve?

   Answer: Cut StartLux Instant long-sequence GDN bandwidth without changing BF16 values, by packing QKV / z / b / a out of the fat gdn_in row before cs1_gdn_conv, cs1_gdn_gates, and cs1_gated_rms_norm. Source: InferRSI tip3 authorization in the system1 room (2026-10-26), the task that opened this draft. Slack search from this run did not return a permalink; the authorization is that room post as relayed in the task.

2. What is the highest-leverage variable for that outcome?

   Answer: The post-in_proj leading dimension. Instant is 24 GDN layers + 8 full attention, Hk=16, Hv=32, Dk=Dv=128. In-kernel GVA is already inside cs1_gdn_prefill. After gemm_group, QKV still sits in the fat gdn_in row (width 12352) while the QKV channels are 8192, so 4160 elements (33.67%) of each row are past the channels conv indexes. Gates and gated RMSNorm use that same fat stride for b/a and z. Source: same InferRSI tip3 authorization, plus the layout in Widths::of and cs1_gdn_conv / cs1_gdn_gates / cs1_gated_rms_norm.

3. How will this work move that variable rather than create busywork?

   Answer: After Instant in_proj (and the LoRA apply, which is a no-op on this path), copy each sequence's QKV, z, and contiguous b|a strip into scratch planes with the existing cs1_copy2d / cuda::copy2d, then call conv / gates / norm with those buffers and tight leading dimensions (8192, 64, and 4096). cs1_gdn_prefill still reads contiguous lq / lk / lv. No new CUDA symbol, no kernel math change, no CUDA graph, no fused GDN prep, no tip2 scratch pre-size, no tip1 attention V pack. Source: same authorization.

4. Who is the customer or user, and what do they actually need?

   Answer: StartLux Instant InferRSI. They need a draft Instant-4B-only tip that can be smoked and fast-screened on hk2, then full Public231 C1 only if the screen is not an early DROP. They do not need a merge or a ready-for-review flip. Source: same authorization ("Keep PR draft. Instant 4B only.").

After answering the four questions: is a PR needed at all to achieve this outcome?

Human decision and rationale (or explicit source reference): Yes. InferRSI authorized opening this draft PR for tip3 (system1 room, 2026-10-26). The task says to keep it draft. The hk2 screen then early-DROPped the tip. The draft stays open as the record of that DROP. It is not a merge candidate.

For a bug fix: link actual observed failure evidence.
Include the observed result of a minimal reproduction, failing regression test,
or relevant trace. Label untested diagnoses as unverified. For a non-bugfix,
state why this is not applicable.

Evidence: Not a bug fix. This is a latency layout tip. The hk2 fast-screen below is a measured regression of this tip SHA, not a functional failure. GPU output parity was not reported with that screen.

Purpose

Tip3 only: pack Instant 4B GDN in_proj planes before conv, gates, and gated RMSNorm. DROP. Stay draft. Do not merge. Do not mark ready. Do not close this PR from the screen. Do not stack another commit on this tip. Do not adopt the screen p50/p95 as the next baseline.

Git base: 9466e0edf24e16669c2136fecc0d3dcdad20811e (origin/main). Tip SHA: 15cd1c19d96f304947dadc9832a84189e6fa6ac7.

Verdict: regressed. fast_screen_decision: drop. Early-DROP yes. Full Public231 C1 was not run.

Instant-only entry: Model::forward_prompt_bidirectional (the StartLux Instant worker). The pack is selected only when the text config is Instant 4B (hidden 2560, intermediate 9216, 32 layers with full attention on i % 4 == 3, attention 16/4/256, GDN Hk=16 / Hv=32 / Dk=Dv=128). Causal run, run_window / prefix conv tails, CUDA-graph capture, and every other backbone keep the fat gdn_in stride. Qwen3.5-9B has the same GDN row width 12352 and the same attention-head counts; it does not take this pack.

What the kernels actually index (verified in this tree, not a new measurement):

• cs1_gdn_conv loads qkv[s * ld + c] for c in [0, channels) and four taps s = t-3+j. It does not index the tail past the QKV channels. Today ld = gdn_in = 12352. After the pack, ld = channels = 8192.
• cs1_gdn_gates loads b[t * ld + h] and a[t * ld + h]. Today ld = 12352 with b and a at columns 12288 and 12320. After the pack they are one b|a strip, ld = 64.
• cs1_gated_rms_norm loads z at token stride ldz. Today ldz = 12352 and z starts at column 8192 (width 4096). After the pack, ldz = 4096.
• cs1_gdn_prefill is unchanged and still reads contiguous lq / lk / lv.

The 4160/12352 = 33.67% figure is the unused tail of the fat row. It was the bandwidth hypothesis. The hk2 screen rejected it. The copy was intended to keep the same BF16 values. Output parity on this tip was not reported.

RSI phase: launch/post. The owner name for this tip is #166-class long-seq layout/contiguity. It is not a bare kernel swap. gdn_prefill.cu and attention are untouched. The gate's phase list is bring-up/scheduling, parallelism/comms, kernel, and launch/post; this tip is launch/post because the change is which buffers and leading dimensions the existing kernels are launched with, plus existing cs1_copy2d launches.

Primary change: per-sequence cs1_copy2d of QKV, z, and the b|a strip into scratch, then tight leading dimensions on conv, gates, and gated RMSNorm. Instant 4B prompt-bidirectional prefill only.

Alternatives considered, not in this tip:

• Tip1 Pack Instant V into contiguous scratch before bidirectional attention #178 attention V pack (closed). Different tensor: attention V, ldv = Hk*Dh = 1024, source column 9216, QKV stride attn_in = 10240, into cs1_attention_gated_bidirectional. That column sits inside GDN z on Instant; it is not a GDN plane boundary. That path is its own DROP. This tip does not pack attention V and does not edit attention.cu.
• Tip2 Pre-size StartLux Instant prompt scratch to the serve limit #180 scratch pre-size to the serve limit. Not included. Scratch still grows to the next multiple of 1024 tokens.
• Instant CUDA graph / load_with_graph(..., true). Out of scope. The Instant worker still loads with graph off, and forward_prompt_bidirectional still refuses graphs.
• Fusing the GDN prep launches into one kernel. Out of scope.
• A new CUDA symbol, or any math change in gdn_prefill.cu. Out of scope.
• Changing the in_proj weight layout so the GEMM writes tight planes directly. Larger ABI change; not this tip.

Hypothesis and measured screen

Hypothesis, not a result: versus the owner-rounded Instant screen 19.113 / 73.461 (n=58), Δp50 ≈ −0.2…−0.6 ms and Δp95 ≈ −2…−6 ms. The measured screen went the other way.

Screen primary the runner compared against (do not type these into --baseline-p50-ms / --baseline-p95-ms):

screen_primary_sha: 51f80ad681c8f7b1be9b22b5e700bef34e33534c
screen_primary_p50_ms: 19.112976267933846
screen_primary_p95_ms: 73.46072793006897
n: 58
bucket_counts: 0-127:10, 128-255:10, 256-511:8, 512-1023:8, 1024-2047:6, 2048+:16
58/58 success, evidence_complete true, decision screen_primary, quota_shortfall {}
device: L20X
compute capability: sm_89
clock: post_to_body_json_and_schema_validation

Source of that primary: .agents/skills/public231-c1/SKILL.md and .agents/skills/optimize-remeasure-gate/SKILL.md. The owner rounds it to 19.113 / 73.461 (n=58).

hk2 fast-screen of this tip, reported 2026-10-26. This agent did not rerun it and did not read the raw responses.jsonl. The pasted summary did not restate a fresh nvidia-smi string or a dropped_at timestamp. The comparison is the same hk2 screen primary, whose recorded identity is device L20X and compute capability sm_89. No different identity was reported.

label: fast-screen
runtime_revision: 15cd1c19d96f304947dadc9832a84189e6fa6ac7
smoke: PASS, evidence_complete
baseline_screen: Instant@51f80ad681c8f7b1be9b22b5e700bef34e33534c
baseline screen p50/p95: 19.113 / 73.461 ms (n=58)
tip p50: 20.029 ms
tip p95: 77.020 ms
Δp50: +0.916 ms (+4.79%)
Δp95: +3.559 ms (+4.84%)
fast_screen_decision: drop
early_DROP: yes
full_C1: NOT RUN
2048+: tip 54.96 vs baseline 50.24 (worse)
clock: post_to_body_json_and_schema_validation
device: L20X
compute capability: sm_89
artifacts: ~/experiments/2026-10-26-pr187-tip3-screen/ on hk2

Both percentiles are worse by more than +2% and more than +0.5 ms, which is the early-DROP rule. The 2048+ bucket was worse as well, so this is not a long-only gain with a flat pooled p50. Full C1 7.338/51.816 was not used and was not run. These screen numbers are not a new baseline and not a KEEP.

Owner profile cited when the tip was opened, separate from this screen: hk2, main 9466e0edf24e16669c2136fecc0d3dcdad20811e, length-bucket p50 5.5 / 5.9 / 8.4 / 13.7 / 24.0 / 49.8 ms at 2048+.

Full Public231 C1 primary, not used for this screen, and not run for this tip:

runtime_revision: 51f80ad681c8f7b1be9b22b5e700bef34e33534c
pooled p50: 7.338 ms
pooled p95: 51.816 ms
clock: post_to_body_json_and_schema_validation

The gate text does not restate a serving pass rate or a separate device line on that 7.338/51.816 sentence. This PR does not invent either.

Protocol

1. Smoke. Passed on tip 15cd1c19, evidence_complete. A failed smoke would not have been a DROP.
2. One fast-screen against the Instant@51f80ad n=58 --baseline-screen (owner-rounded 19.113/73.461). Seed 231, concurrency 1, warmup 5, quotas 10/10/8/8/6/16. Clock post_to_body_json_and_schema_validation. device L20X, compute capability sm_89.
3. Early DROP because both screen p50 and screen p95 are worse (Δp50 +0.916 ms / +4.79%, Δp95 +3.559 ms / +4.84%).
4. Full C1 was not run. Do not run it on this tip. Do not force KEEP.
5. dropped_at was not in the summary pasted for this update. The hk2 artifact directory above is the place the runner writes fast_screen_drop.json. This PR does not invent that timestamp.

A negative screen delta is not a gain and not a KEEP. This decision is a regression, not INCONCLUSIVE: smoke passed, the screen decision is drop, and both percentiles cleared the worse-than threshold.

Negative lesson

This DROP: packing Instant GDN in_proj from the fat gdn_in row (12352) into contiguous QKV / z / b|a planes regresses the screen dual percentiles (20.029 / 77.020 versus 19.113 / 73.461). The contiguous layout was slower on the pooled screen, and 2048+ was worse (54.96 versus 50.24). Do not retry this pack as the next baseline.

Distinct from tip1 #178 attention V-pack DROP. Tip1 copied attention V (ldv = Hk*Dh = 1024, column 9216, stride attn_in = 10240) before cs1_attention_gated_bidirectional. This tip copied GDN QKV (8192), z (column 8192, width 4096), and b|a (columns 12288 and 12320) before cs1_gdn_conv, cs1_gdn_gates, and cs1_gated_rms_norm. Different tensor, different leading dimension, different kernel. Both are DROPs. One does not explain the other. Tests that lock the layout distinction:

• instant_4b_gdn_in_proj_pack_widths_are_not_tip1_attention_v_pack
• non_instant_and_tip1_attention_shapes_do_not_take_the_gdn_in_proj_pack_path
• packed_gdn_planes_match_strided_qkv_z_and_gate_columns

Qwen3.5-9B shares the GDN row width and the tip1 attention V geometry and returns None from this pack. The 27B fixture stays on the fat stride.

Test Plan

System1-Omni Version / Commit: tip 15cd1c19d96f304947dadc9832a84189e6fa6ac7 on cursor/instant-gdn-in-proj-pack-a459, git base 9466e0edf24e16669c2136fecc0d3dcdad20811e. Screen primary and full-C1 primary remain 51f80ad681c8f7b1be9b22b5e700bef34e33534c. The full-C1 pair was not the screen baseline and was not run.

Offline checks that were run when the draft opened (rustc 1.99.1):

• cargo test -p omni-qwen3-5-native --locked
• cargo test -p omni-startlux-native --test contract --locked
• cargo clippy -p omni-qwen3-5-native --all-targets --locked -- -D warnings

hk2, as reported with this update: smoke, then one fast-screen. Full C1 stays unrun because the decision is drop.

Test Result

Offline on tip 15cd1c19d96f304947dadc9832a84189e6fa6ac7, rustc 1.99.1, from the draft open:

• cargo test -p omni-qwen3-5-native --locked — lib 25 passed / 14 ignored (GPU); config, device_features, gdn_in_proj_pack (3), graph_loading, and image_decode passed; graph, kernels, prefix, and readout ignored for lack of a GPU checkpoint. Doc-test 1 passed.
• cargo test -p omni-startlux-native --test contract --locked — 8 passed.
• cargo clippy -p omni-qwen3-5-native --all-targets --locked -- -D warnings — clean.

hk2 screen, reported in the follow-up that asked for this body update. Not recomputed from raw samples in this run:

• Smoke PASS for tip 15cd1c19, evidence_complete.
• Fast-screen versus Instant@51f80ad baseline-screen 19.113/73.461, n=58.
• Tip p50 20.029 ms, p95 77.020 ms.
• Δp50 +0.916 ms (+4.79%), Δp95 +3.559 ms (+4.84%).
• fast_screen_decision: drop. Early-DROP yes.
• 2048+ worse: tip 54.96 versus baseline 50.24.
• Full C1 not run.
• Artifacts: ~/experiments/2026-10-26-pr187-tip3-screen/ on hk2.
• device L20X, compute capability sm_89, from the screen primary this comparison used. This paste did not include a new nvidia-smi line or dropped_at.

Still unverified: GPU output parity, bench.py compare, and any full-C1 figure for this SHA. Workspace clippy and cargo test --workspace were not run. No code change accompanies this result.

Demo / evidence

N/A for a video. This is a data-layout change with no UI.

The hk2 fast-screen is an actual reported run of tip 15cd1c19d96f304947dadc9832a84189e6fa6ac7 against the 51f80ad screen primary. It is a DROP, not a speedup. Local artifacts stay on hk2 at ~/experiments/2026-10-26-pr187-tip3-screen/. Full C1 p50/p95, pooled 2048+ p95, and output parity for this SHA were not measured. The 7.338/51.816 pair remains the full-C1 primary of 51f80ad and is not a result of this tip.

Controls on the reported screen: warmup 5, concurrency 1, clock post_to_body_json_and_schema_validation, n=58 quotas, device L20X, compute capability sm_89. A different device string or capability would be a separate group. None was reported.

Self-review

Before marking this PR ready for review or requesting maintainer review, complete
the self-review checklist.
Keep the PR in draft while this work is incomplete.
For agent assistance, use the optional precheck-pr skill.

The code diff is unchanged: base 9466e0edf24e16669c2136fecc0d3dcdad20811e, head 15cd1c19d96f304947dadc9832a84189e6fa6ac7. This update is the screen record only. The tip is a DROP. Do not mark the PR ready.

• I have reviewed the full diff and addressed the issues I found.
• I have checked that the change follows the project's architecture and stays focused on the stated purpose.
• I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
• I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

  

[hsliuustc0106]

DROP (InferRSI system1 room, 2026-10-26 Asia/Shanghai).

• Tip 15cd1c19: Instant GDN in_proj pack via cs1_copy2d (fat gdn_in → contiguous).
• Smoke PASS; fast-screen early-DROP vs Instant@51f80ad screen primary 19.113/73.461 (n=58): tip 20.029/77.020 → Δp50 +0.916 ms (+4.8%), Δp95 +3.559 ms (+4.8%).
• dropped_at=fast-screen. Full C1 not run. 2048+ also worse (54.96 vs 50.24).
• Negative sample: GDN in_proj pack copy tax > stride savings on Instant screen dual gate; distinct from tip1 V-pack DROP. tip4 deferred pending attribution.