[perf+docs]: p-value path ~70-78x faster + optimization backlog (optimization-backlog)#38
Open
haoyu-zc wants to merge 5 commits into
Open
[perf+docs]: p-value path ~70-78x faster + optimization backlog (optimization-backlog)#38haoyu-zc wants to merge 5 commits into
haoyu-zc wants to merge 5 commits into
Conversation
Collaborator
Author
|
Codex review flagged that the |
haoyu-zc
force-pushed
the
refacotring-07-16-2026
branch
from
July 17, 2026 02:30
810970c to
aad009f
Compare
haoyu-zc
force-pushed
the
pr-f-optimization-backlog
branch
from
July 17, 2026 02:31
4758d80 to
1cd90da
Compare
added 2 commits
July 17, 2026 00:53
…mization-backlog) OpenSpec change: optimization-backlog. Docs-only exploration (zero production-code changes) to guide future performance work. - New docs/source/development/optimization_backlog.rst: the ranked K/H/A opportunity table (kernel / host / algorithmic) with current file:line refs, impact/effort/risk, measurement status, and tranche assignments. Cross-references what fix-cuda-correctness already landed (memory-bounded p-value batching + permutation sub-batching) and the remaining headroom. - Captured baseline via the ccc-gpu-bench CLI on the RTX 4090 (commit 4209d17): GPU coefficient-only scaling (0.67s@1k .. 28.1s@10k features), GPU-vs-CPU speedups (121x@500f, 210x@1000f), the CPU category split (ARI 83.1%, Coordination 12.9%, Partitioning 1.2%), and the p-value blow-up that motivates the top item: 200 features go from 0.11s (coef-only) to >300s with 100 permutations, dominated by the per-comparison launch+sync loop (H2). - Documents the nsys/ncu profiling procedure; kernel-counter capture was NOT possible here (nsys absent; ncu blocked by RmProfilingAdminOnly=1), so the profile-dependent items stay flagged "needs profiling" with exact commands. Tranche 1 (schedule now, inspection-justified): H1/H2/H3/A3. Tranche 2 (kernel restructures, gate on golden-parity tests): K1+K2, K6.
Addresses codex review of the optimization-backlog PR: the tasks.md checkboxes over-claimed completion of the profiling baseline. - 2.1/2.2/2.3 changed [x] -> [~] (partial): wall-clock baselines are captured, but the nsys timeline + ncu kernel counters and the evidence-based re-ranking of the counter-dependent items are DEFERRED (not capturable in this env). - Documented that the categorical-heavy / fixed-n_parts=9 workload for A2 cannot be produced by the current `ccc-gpu-bench coef` (no --n-parts / categorical generator), so A2 needs a small bench enhancement plus the counters before it can be ranked. No fabricated kernel numbers.
haoyu-zc
force-pushed
the
pr-f-optimization-backlog
branch
from
July 17, 2026 06:53
1cd90da to
ad5d261
Compare
added 3 commits
July 17, 2026 01:07
The p-value path launched computePermutationCCC once per feature comparison (one thread per permutation, ~n_perms threads -> GPU ~99% idle) with a full device sync after every launch, so wall time scaled as O(n_feature_comp): 200 features took >294 s. New computePermutationCCCBatched maps one thread to each (comparison, permutation) pair, so a single launch covers n_comps * n_perms threads and all comparisons run concurrently. Each thread derives its feature pair from the condensed comparison index and picks the permuted feature from a precomputed per-feature valid-partition count (matching the CPU rule). The host now steps over comparison sub-batches sized to a scratch budget, collapsing the launch count from n_feature_comp to a handful. Removed the now-dead per-comparison kernel. Semantics unchanged (categorical->0, singleton->NaN, clamp>=0); GPU still matches the CPU reference. Measured ~31-34x speedup, and inputs that previously timed out now complete: 30 features: 7.64 s -> 0.225 s 50 features: 18.96 s -> 0.613 s 80 features: 46.82 s -> 1.528 s 200 features: >294 s -> 9.175 s Parity: 30/30 p-value + cuda-correctness tests pass; full GPU suite 79 passed; compute-sanitizer memcheck clean.
…(H2b) The batched permutation kernel kept each thread's k*k contingency in global scratch at scratch + gid*scratch_stride, so consecutive warp lanes were scratch_stride ints apart -> every access spanned 32 cache lines (uncoalesced), and the buffer cost up to ~788 MB. For the common small-k case (k <= 16, k*k+2k <= PERM_LOCAL_CAP=288) build the contingency in a per-thread LOCAL array instead. CUDA interleaves local memory across a warp, so the accesses become coalesced and L1-cached; the global scratch buffer collapses to one element and the per-launch comparison bound is removed. Large k (>16) still uses the global-scratch fallback (its parity test passes). Same math; GPU still matches the CPU reference. ~2.3-2.4x on top of H2: 30 features: 0.225 s -> 0.099 s 50 features: 0.613 s -> 0.269 s 80 features: 1.528 s -> 0.640 s 200 features: 9.175 s -> 3.760 s Combined with H2 the p-value path is ~70-78x faster than baseline. Parity 30/30; full GPU suite 79 passed; memcheck clean.
haoyu-zc
changed the title
Collaborator
Author
|
Codex review of the optimization commits ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements OpenSpec change optimization-backlog (proposal 8/8) and the first pass of the optimizations it ranks. Targets
refacotring-07-16-2026.1. The ranked backlog + captured baseline (docs)
docs/source/development/optimization_backlog.rst— the ranked K/H/A opportunities with file:line refs, impact/effort/risk, tranches, and a capturedccc-gpu-benchbaseline on the RTX 4090. (Kernel-counter profiling deferred —nsysabsent,ncublocked byRmProfilingAdminOnly=1.)2. Implemented optimizations — p-value path ~70–78× faster
Two GPU-side rounds against the marquee bottleneck (the permutation p-value path), each parity-verified against the CPU reference:
Verification (every round): full GPU parity suite 79 passed; p-value + correctness 30 passed; k>16 fallback passed;
compute-sanitizer memcheck0 errors. Same coefficients and p-values as before, bit-for-bit.Deferred (with reasons): H1/H3 (coefficient path is compute-bound — per-batch overhead measurably <1%), A3 (robustness, not speed), and K1/K2/K3 (ARI-kernel restructures — need
ncuprofiling this environment can't provide). Each stays scoped in the backlog.3. Report
Full write-up in
docs/optimization_results.md, and a shareable rendered report:https://claude.ai/code/artifact/22b07f45-61ea-4ea0-b0b2-8b1113e7d526