DeepSpeed - Q3 Roadmap

[PKUWZP]

This is the roadmap for DeepSpeed Q3 2026. Feedback is welcome — please leave comments on this issue or join the #2026q3-roadmap channel on the DeepSpeed Slack.

Some of the items are the extension of the Q2 roadmap: #7861

New feature and enhancement

Unified Expert-Parallelism(EP) support (Q3)

• AutoEP Support: AutoEP enables Expert Parallelism (EP) for major Mixture-of-Experts (MoE) models out of the box, eliminating the need for users to write model-specific parallelization code. By automatically distributing expert layers across devices, AutoEP allows users to scale MoE training with minimal configuration changes.
• Combining AutoEP with AutoTP, and benchmarking TP in Attention across more models with Multi-head Attention.
• Unified EP Kernels for improving large-scale training performance of MoE architectures
• Extend AutoTP capabilities by integrating Liger Kernel (sharding LM heads and adopting Online Softmax)

New Accelerators Support (Q3)

• DeepSpeed TPU accelerators support
• Performance Optimization on Emerging AI Accelerators (e.g. Biren AI Accelerators)

On-Policy Distillation Trainer Support (Q3)

• Systems Design, prototyping and benchmarking (reference: https://arxiv.org/abs/2604.14084)

DeepCompile efficiency and robustness improvements

• Formal pass contracts and validation of optimization passes: Add lightweight optimization pass contracts for automatic compatibility validation and ordering.
• Composable AutoTP / AutoEP / SP optimization passes: Implement AutoTP and AutoEP at a compiler level and integrate them with AutoSP.
• Enhance optimization passes: Add some optimization passes including activation offloading.

Pipeline parallelism with Ray

• Ray-backed pipeline stage placement and execution: Prototype pipeline stage placement and execution using Ray actor groups for a representative transformer workload, enabling heterogeneous resource allocation across pipeline stages. (Will be a generalized implementation of this).
• Create some examples: Provide example implementations of Ray-backed pipeline parallelism for representative transformer workloads, demonstrating the benefits of heterogeneous resource allocation.

Tuning guide with benchmarking results

• Recommended configurations for representative models: Provide practical configuration guidance for representative dense and MoE models across several common GPU setups.
• Benchmark-backed guidance: Run targeted benchmarks to support the recommended configurations and document the observed throughput, memory usage, and limitations.

Stability (Q3)

• Performance regression test
• Enable nightly full test
  • CUDA
  • AMD
  • Intel XPU
  • Intel Gaudi
  • NPU