Efficient LLM inference on Slurm clusters.

LLM 推理服务性能测试

Layered prefill changes the scheduling axis from tokens to layers and removes redundant MoE weight reloads while keeping decode stall free. The result is lower TTFT, lower end-to-end latency, and lower energy per token without hurting TBT stability.

eLLM can infer LLM on CPUs faster than on GPUs