Skip to content

Add compositional Muon CE-track leaders#11

Open
gabrielnan wants to merge 2 commits into
devfrom
add-comp-muon-leader
Open

Add compositional Muon CE-track leaders#11
gabrielnan wants to merge 2 commits into
devfrom
add-comp-muon-leader

Conversation

@gabrielnan

@gabrielnan gabrielnan commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds two CE-track compositional Muon submissions:

  • nanogpt_comp_muon_mp1: OV-only isotropic Compositional Muon, 2150 steps.
  • nanogpt_comp_muon_mp4: OV isotropic Compositional Muon plus row-uniform Muon on the tall MLP expansion matrix, 1600 steps.

nanogpt_comp_muon_mp4 is now the lower-energy CE-track leader under the benchmark objective: pass native CE and accuracy gates, then rank by training energy.

Results

Submission Total energy Training energy Val acc Native CE Status GPU
nanogpt_comp_muon_mp4 42,059 J 34,235 J 0.7286 1.3182 pass A100-SXM4-80GB
nanogpt_comp_muon_mp1 58,224 J 48,309 J 0.7344 1.2856 pass A100-SXM4-80GB

Changes

  • Adds nanogpt_comp_muon_mp1 submission, README, result.json, nvml.json, and run.log.
  • Adds nanogpt_comp_muon_mp4 submission, README, result.json, nvml.json, and run.log.
  • Updates the CE-track leaderboard wording to rank passing rows by training energy.
  • Adds both rows to the CE-track leaderboard, with nanogpt_comp_muon_mp4 first by energy.

Energy Variance Check

We repeated the nanogpt_comp_muon_mp4 1600-step config with SEED=42 to sanity-check energy variance. Two full SXM4 repeats passed with total energies 48,013 J and 43,710 J; population CV was ~4.7% for total energy and ~5.1% for GPU energy. The original 42,059 J run is low but within observed host/power variance, and the improvement over the previous 58,224 J leader is much larger than the repeat variance.

Verification

  • python3 -m py_compile submissions/nanogpt_comp_muon_mp1/submission.py
  • python3 -m py_compile submissions/nanogpt_comp_muon_mp4/submission.py
  • submit.precheck_submission(...) returns @gabrielnan for both submissions
  • Modal benchmark runs completed successfully and wrote artifacts.

@gabrielnan gabrielnan changed the title Add compositional Muon CE-track leader Add compositional Muon CE-track leaders Jun 16, 2026
@gabrielnan gabrielnan requested a review from ab-10 June 17, 2026 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant