BLAS beta==0 write-only semantics (fixes 0*NaN=NaN poisoning from cold scratch)#19
Merged
Conversation
…output read (0*NaN=NaN fix)
…s); update mirror+docs
… solver-choice guidance
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Every beta-taking op used to blend with a raw
alpha*res + beta*dst, reading the destination even whenbeta == 0— so cold/uninitialized scratch holding NaN poisoned the result (0*NaN = NaN). This violates BLAS/cuBLAS convention (beta==0 ⇒ output is write-only) and was documented as a caller-side footgun (old debugging-guide §1c). It is the root cause of the GRiD/Parallel-DDP "heavy plant give-up" NaN: GRiD's generated RNEA writess_vafv/a bodies viagemv_segmented(..., beta=0)over cold smem, and diverged line-search rollouts leave NaN on the SM.All blends now route through
beta_blend(acc, beta, dst)(src/base/barrier.cuh):dstis read only when beta != 0, and the hot arm is an explicit__fmaf_rn/__fma_rn— a plain expression left the FMA-contraction choice to ptxas per kernel and broke warp-vs-block bit-identity (caught bytest_syrk_warp; the explicit intrinsic restored it, 72/72).Covered: gemv / gemv_strided / gemv_segmented / gemv_reduced, gemm (untiled + tile4 + tiled) / gemm_strided / gemm_reduced, syrk / syrk_reduced, symm, congruence, axpby — block,
warp::, andcgrps::surfaces (shared impls). Docstrings + debugging-guide §1c updated to the new guarantee.Bit-compat: identical results at beta ∈ {0, 1} (every known consumer's usage — GRiD/PDDP gates verified bit-identical downstream); last-ULP may shift at fractional beta, where the old contraction was unspecified anyway.
Tests
beta=0into NaN-poisoned destinations must come back clean —test_l2.py::test_gemv{,_strided,_segmented}_beta0_poisoned_y_no_read,test_l3.py::test_gemm_rt_betaform_beta0_no_read(incl. a tile4-path shape).Handoff TODO (before merge to main)
pytest test/)./test/run_gpu_proof.sh) bundled per the main-branch gateDownstream validation already done (Parallel-DDP)
Vendored this diff + regenerated iiwa14/iiwa14_heavy: qNaN-poisoning the whole vaf arena is now a no-op (was 6272 nonfinite → solve death), all BP inputs finite over full solves, iiwa gates bit-identical (51170.97265625 iLQR / 55011.00390625 DDP), and the heavy-swing repro goes from rho give-up at iter 57 (cost 4267.9) to a converged swing (cost 1725.1, pos_err 0.38 rad).
🤖 Generated with Claude Code