---
title: "SplitScaling: Adaptive Scaling for Disaggregated LLM Serving Against Traffic Bursts via DRL"
authors:
  - "Wei Xiao"
  - "Xuefeng Huang"
  - "Weijia Shi"
  - "Baokang Zhao"
doi: "10.24963/ijcai.2026/568"
url: "https://ijcai.org/proceedings/2026/568"
pdf: "https://ijcai.org/proceedings/2026/0568.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "5101-5109"
track: "Main Track"
keywords:
  - "Planning and Scheduling: Learning in planning and scheduling"
  - "Machine Learning: Reinforcement learning"
---

# SplitScaling: Adaptive Scaling for Disaggregated LLM Serving Against Traffic Bursts via DRL

Wei Xiao, Xuefeng Huang, Weijia Shi, Baokang Zhao

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 5101-5109. DOI: [10.24963/ijcai.2026/568](https://doi.org/10.24963/ijcai.2026/568)

## Abstract

The disaggregated Prefill-Decode (PD) architecture has emerged as a prominent paradigm for efficient Large Language Model inference serving. However, resource management remains a critical challenge, particularly under the dual burstiness of real-world scenarios—characterized by volatile fluctuations in both request arrival rates and Prompt-to-Response ratios. Existing rule-based heuristics often fail to accurately identify system bottlenecks, leading to severe resource misallocation and Service Level Objective (SLO) violations. To address this, we propose a Deep Reinforcement Learning-based auto-scaling framework tailored for the PD architecture. By modeling the resource allocation problem as a Markov Decision Process, our framework enables the agent to capture non-linear load dynamics, thereby achieving decoupled and precise scaling for prefill and decode pools. Furthermore, to mitigate Head-of-Line blocking caused by scaling latency, we design an immediate rescheduling mechanism that migrates queued tasks to newly ready nodes in real-time. Experimental results driven by Azure bursty load traces demonstrate that our framework significantly reduces computational costs by 25.2% and 28.3% compared to the Static configuration and HeteroScale, respectively, while strictly adhering to SLOs.