DeepSpeed parity hacks#502

Closed

bigximik wants to merge 25 commits into

mainfrom

Commits on Apr 27, 2026

Commits on Apr 28, 2026

gspo: add sequence-level IS-ratio clipping loss
bigximik
committed
schedule: add rollouts_per_step to auto-set depth_first_micro_batches
bigximik
committed
grpo: dynamic docs_per_step accumulation and normalize_by_documents
bigximik
committed

Commits on Apr 29, 2026

grpo: temperature scaling for IS ratio parity with actor sampling
bigximik
committed

Commits on May 4, 2026

head: fp32_lm_head flag to match vLLM bf16_last_layer_fp32 precision
bigximik
committed
head: fix fp32_lm_head gradient flow via detach + manual weight grad accumulation
bigximik
committed

Commits on May 5, 2026

Commits on May 6, 2026

Commits on May 7, 2026

Commits on May 15, 2026