Skip to content

ptx-shapes: re-pin the sm_121a rows the HC up-mix split and the MXFP8 TMA tile moved - #1026

Merged
juliankang4 merged 3 commits into
midagedev:mainfrom
juliankang4:dgx-spark-q38-device-target-and-ptx
Oct 30, 2026
Merged

juliankang4 merged 3 commits into
midagedev:mainfrom
juliankang4:dgx-spark-q38-device-target-and-ptx

Conversation

@juliankang4

Copy link
Copy Markdown
Collaborator

RTX 5090처럼 모든 카드가 cc 12.0인 머신이 sm_120a로 빌드하도록 tools/arch.sh를 고치고, 그 target의 PTX 표를 추가하고, GB10의 낡은 sm_121a 표 4행을 다시 핀했습니다.

What and why

On a machine whose cards are all cc 12.0 (an RTX 5090), tools/arch.sh picked sm_86, so the PTX was JIT-compiled and the native block-scaled FP4 path stayed off. just narrow and ptx-compare refused anything but sm_86, and there was no spill table for sm_120a.

  • tools/arch.sh maps each card's capability to a target: 12.0 to sm_120a, 12.1 to sm_121a, anything else to sm_86. The target is used only when every card agrees; mixed cards build sm_86, the one target all of them load.
  • tools/narrow-scan.sh and tools/ptx-compare.sh build at the selected arch instead of the fixed sm_86. The arch is part of their cache key, so an sm_86 entry is never reused for sm_120a.
  • tools/ref/ptx-shapes/sm_120a-cc12.0.tsv is new: 524 rows from an RTX 5090 scan, with the 29 non-zero rows under one PIN(2026-10-31) block.
  • tools/ref/ptx-shapes/sm_121a-cc12.1.tsv re-pins four rows the GB10 spill gate reported as out of date on main (the measured values are under What I ran). Three hc_gated_up_mma_mix_4* rows drop because the HC up-mix's passes split the warp's columns. mxfp8_w8a8_gemm gains JIT local bytes because the tile's stages arrive by TMA on stage barriers; ptxas still reports no spill for it. Each re-pin carries its PIN line and reason.
  • docs/BUILD.md, docs/HARDWARE.md, docs/contrib/* and the comments in the scan tools name the selected arch instead of a fixed sm_86. tools/check-recipes.sh gains four selection cases (12.0, 12.0/12.0, 12.0/8.6, 8.6/12.0).

Release notes: Changed: a machine whose cards are all cc 12.0 (RTX 5090) builds for sm_120a by default, and just narrow and ptx-compare build at the selected arch

Code shape

  • No second copy. The target choice lives only in tools/arch.sh; narrow-scan.sh and ptx-compare.sh call bloomery_arch instead of keeping their own sm_86 constant.
  • Model code holds only architecture facts. No model file changes.
  • Dead code is gone. The fixed ARCH=sm_86 and its refusal in narrow-scan.sh and ptx-compare.sh are removed.
  • No silent failure. A nvidia-smi failure and an invalid BLOOMERY_ARCH are still named refusals.
  • Comments state what is true now. The updated comments describe the current selection rule.
  • No relaxed gate. The re-pins move the table to the values the gate measured, each with a PIN(2026-10-31) line and reason; no band or lint moves.
  • cargo fmt and cargo clippy are clean on what I touched; the lint count did not go up.

Duplication found or removed (path:line): the fixed sm_86 arch in tools/narrow-scan.sh and tools/ptx-compare.sh, now bloomery_arch from tools/arch.sh.

What I ran

Base 989d0106b, head under test 6a8ea5f85 (the table re-pin commit changes only the sm_121a table).

RTX 5090, before the table existed. The build picked sm_120a and the gate refused:

ARCH-SELECTED sm_120a
ptx-spill-check: no ptx-shapes table for sm_120a; the tables pin sm_86 on cc 8.6, sm_121 on cc 12.1, sm_121a on cc 12.1, sm_86 on cc 12.1
GATE-PTX-SPILL-BEFORE-TABLE rc=64

RTX 5090, after. Built with cargo oxide build --arch sm_120a, then the spill gate, the gate_gemm cases and the user binary:

ptx-spill bin=generate_ds41 table=tools/ref/ptx-shapes/sm_120a-cc12.0.tsv arch=sm_120a jit-cc=12.0
ptx-spill bin=gate_qwen3vl_tower entries=26 pinned=26 nonzero=none violations=0 PASS
GATE-PTX-SPILL rc=0
GATE-GEMM-CASE dense_mma rc=0
GATE-GEMM-CASE mxfp8_w8a8 rc=0
GATE-GEMM-CASE nvfp4_native rc=0
GATE-GEMM-CASE nvfp4_swiglu rc=0
GATE-GEMM-CASE nvfp4_gate_up rc=0
GATE-GEMM-CASE zgate rc=0
RECOMMEND build {"cuda_target": "sm_120a", "fp4": true} class blackwell cards [('NVIDIA GeForce RTX 5090', '12.0')]
SERVE-READY after 3s
bloomery-serve: request 1 /completion: prompt 5 ids (0 reused) in 0.07 s, 73.9 ids/s; generated 24 ids in 0.42 s, 57.2 ids/s; stop length; ctx 28/262144

The serve run used bloomery-serve --model qwen3 (Qwen3-30B-A3B-Instruct-2507 Q4_K_M). The generate_ds41 spill gate on the 5090 covers ptx-spill bin=generate_ds41 entries=498 pinned=498.

GB10, sm_121a. The build still selects sm_121a (ARCH-SELECTED sm_121a). The spill gate on main's table reported these rows, which the re-pin sets to the measured values:

generate_ds41 hc_gated_up_mma_mix_4 spill=56 pinned=252 BELOW
generate_ds41 hc_gated_up_mma_mix_4 jit_local=24 pinned=64 BELOW
generate_ds41 hc_gated_up_mma_mix_4_bf16 spill=56 pinned=292 BELOW
generate_ds41 hc_gated_up_mma_mix_4_bf16 jit_local=24 pinned=56 BELOW
generate_ds41 hc_gated_up_mma_mix_4_mxfp8 spill=56 pinned=276 BELOW
generate_ds41 hc_gated_up_mma_mix_4_mxfp8 jit_local=24 pinned=56 BELOW
generate_ds41 mxfp8_w8a8_gemm jit_local=8 pinned=0 ABOVE
ptx-spill bin=gate_qwen3vl_tower entries=26 pinned=26 nonzero=none violations=0 PASS

Host checks (aarch64, same head):

bash tools/check-recipes.sh --block archowner      check-recipes: arch owner ok (200 recipe lines build with --arch {{arch}}, 15 ownership fixtures, 28 selection cases)
bash tools/narrow-scan.sh --self-test              narrow-scan: self-test ok (141 cases, 0 failed)
bash tools/ptx-compare.sh --self-test              ptx-compare: self-test ok (37 cases, 0 failed)
bash tools/release/build.sh --self-test            release: self-test ok (22 cases)
python3 tools/check-ptx-names.py check             check-ptx-names: ok (2 binaries, 524 rows = 524 declared entries; ...)
cargo test -q -p bloomery-levers                   test result: ok

Hardware (if the change is for a machine)

NVIDIA GeForce RTX 5090, 12.0, 595.99.02
NVIDIA GB10, 12.1, 595.84

The RTX 5090 machine builds sm_120a and the DGX Spark builds sm_121a, both with ptxas-version=13.3.73 (CUDA 13.3).

AI help

AI 코딩 에이전트가 코드를 작성한 후 설명을 썼고, 테스트를 마쳤습니다. diff 리뷰와 PR 초안 리뷰는 제가 직접 했습니다.

gate-ptx-spill had no table for sm_120a, so on an RTX 50 series card a
build for the card's native target was refused before any scan (rc 64,
"no ptx-shapes table for sm_120a"). The new table holds every entry of
generate_ds41 and the tower's vis_ entries at sm_120a on compute capability
12.0, the native FP4 entries of the bloomery_nvfp4_mma modules included,
from ptx-scan on an RTX 5090 (ptxas 13.3.73, driver 595.99.02).
…w and ptx-compare build at the selected arch

tools/arch.sh picked sm_121a only when every card was cc 12.1 and sm_86 for
everything else, so an RTX 50 series machine built sm_86 and its kernels ran
without the native FP4 and FP8 products build.rs turns on for sm_120a. The
pick now maps each card's capability (12.0 to sm_120a, 12.1 to sm_121a, any
other to sm_86) and keeps it when every card agrees; cards of different
capabilities still share sm_86.

just narrow and just ptx-compare built both sides at a fixed sm_86 and refused
any other BLOOMERY_ARCH, so neither ran on a GB10 or an RTX 50 series machine
through the justfile. Both now take tools/arch.sh's arch, export it to their
box calls, and key their base caches on it (ptx-compare's cache directory did
not carry the arch).

docs/BUILD.md's commands take the target from tools/arch.sh instead of
spelling sm_86, and the docs, the BLOOMERY_ARCH lever row and the tool
headers that described the old pick follow.
… TMA tile moved

gate-ptx-spill on a GB10 (sm_121a, driver 595.84) read four rows of
sm_121a-cc12.1.tsv away from their pins at main 989d010:

  hc_gated_up_mma_mix_4        spill 252 -> 56, jit_local 64 -> 24 (BELOW)
  hc_gated_up_mma_mix_4_bf16   spill 292 -> 56, jit_local 56 -> 24 (BELOW)
  hc_gated_up_mma_mix_4_mxfp8  spill 276 -> 56, jit_local 56 -> 24 (BELOW)
  mxfp8_w8a8_gemm              jit_local 0 -> 8 (ABOVE)

The three HC rows fell with 92997d0, which split the up-mix's passes
across the warp's columns; no later commit touches those kernels' device
code. mxfp8_w8a8_gemm's last device-code change is e36687d, which moved
the W8A8 tile's stages to TMA on stage barriers with two grid-constant
tensor maps; the later commits to nvfp4mma.rs and f32tile.rs change the
NVFP4 walk and host launch helpers only. At this head ptxas spills nothing
for it and the cc 12.1 JIT keeps 8 bytes a thread, the same as the cc 12.0
JIT on an RTX 5090 (sm_120a-cc12.0.tsv). The kernel is the dense MXFP8
prompt product of the native build (site.rs, E4M3 planes on a dense MMA
card). Neither commit updated the native table, so the gate has read these
rows red on a GB10 since they landed.
@github-actions github-actions Bot added dgx-spark The DGX Spark (aarch64, GB10) port documentation Improvements or additions to documentation host Host crates: qdot, threads, engram, engram-lab, runtime, sampler, levers, decision, vision, refset tools Tools, recipes, CI and the toolchain: tools/, justfile, .github/, Cargo labels Oct 30, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CLEAN: No blocking defect found in target selection, scan execution or cache isolation.

Checked

  • Duplicates: 0 new function, type or constant definitions. Both scan tools reuse the existing architecture owner in tools/arch.sh.
  • Complexity: the changes affect build tooling, documentation and resource tables, with no inference hot-path edits.
  • Silent failure: 2 fallback sites followed into build arguments and exported architecture values. Mixed or unsupported capabilities select the documented sm_86; query failures and malformed overrides return a named error with exit 64.
  • Tests: the PR reports passing architecture-owner checks (28 selection cases), narrow-scan (141), ptx-compare (37), release (22) and PTX-name checks. Static mutation review: changing the 12.0 mapping at tools/arch.sh:28 makes selection assertions fail; accepting malformed targets at line 37 makes both CLI refusal assertions fail. Hard-coding sm_86 at tools/narrow-scan.sh:178 or tools/ptx-compare.sh:61 makes the new target assertions fail. Dropping the arch from tools/ptx-compare.sh:103 makes the cache assertion fail.
  • Gates: the new table has 524 unique entries and 29 nonzero rows. All 4 changed GB10 rows carry dated PIN reasons and reported measurements showing why the previous pins fail.
  • Escape catalog: no changed code matches C1–C4.
  • Adversarial checks: the selector keeps sm_86 after capabilities disagree. Malformed overrides return 64 before builds, and both scan caches include the selected arch.

Notes

GPU evidence is incomplete: the PR supplies card and driver lines, commit hashes and run output, but omits the exact gate commands. The validator did not rerun host tests or GPU gates.

🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)

@juliankang4

Copy link
Copy Markdown
Collaborator Author

The review is done and the contribution check passed, so I'm merging this.

@juliankang4
juliankang4 merged commit 9fa4e00 into midagedev:main Oct 30, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dgx-spark The DGX Spark (aarch64, GB10) port documentation Improvements or additions to documentation host Host crates: qdot, threads, engram, engram-lab, runtime, sampler, levers, decision, vision, refset tools Tools, recipes, CI and the toolchain: tools/, justfile, .github/, Cargo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant