Discrete-event simulation of SLO-aware autoscaling for multi-instance LLM serving — comparing reactive, predictive, and conservative policies across workload patterns and startup delays.
python simulation latency capacity-planning inference slo autoscaling llm-serving serving-infrastructure startup-delay mlsystems workload-patterns
-
Updated
Jul 19, 2026 - Python