🖥️ AI InfrastructureID:
prefill-decode-splitPrefill/Decode Disaggregation
Prefill/Decode 分离
🎯Core Definition
Prefill/Decode disaggregation (PD split) = running the two phases of a request on different instance types: Prefill processes the whole prompt in parallel — matrix-matrix (GEMM) work that is compute-bound with high GPU utilization; Decode generates one token at a time — a matrix-vector (GEMV) step per token that is bandwidth-bound, reading the full weight set to emit a single token, with low utilization. After the split, Prefill instances turn prompts into KV caches and ship them over high-speed networks (RDMA/InfiniBand) to Decode instances, which focus purely on low-latency batched generation. Scheduling: the two pools scale independently; Decode instances are load-balanced by occupied KV size / sequence length (not connection count), with token-level scheduling and pipelined KV transfer.
💡Use Cases
high-concurrency online serving (chat/agent workloads mixing long prompts and long outputs) that must keep both TTFT and ITL low; vLLM/SGLang have native PD support and top vendors like DeepSeek run this architecture in production; interview favorites: why Prefill and Decode have different bottlenecks, and what PD split solves.
⚡Key Problems Solved
when colocated, Prefill bursts (large batches of big prompts) saturate compute and wreck the TTFT/ITL of in-flight Decode requests — the two loads interfere and their scheduling goals conflict. Splitting them: ① each pool is provisioned and scaled to its own bottleneck (compute vs bandwidth), raising GPU utilization; ② TTFT is decided by Prefill instances, ITL by Decode instances, so latency targets are independently tunable; ③ bursts are absorbed by the Prefill pool while Decode stays stable — at the cost of KV transfer over the network and more complex scheduling.
🎯5 High-Frequency Exam Points
1
Why is Prefill compute-bound while Decode is bandwidth-bound (GEMM vs GEMV — reading all weights per step to emit one token)? Explain with a Roofline model.
2
PD split architecture: how does the KV cache travel from Prefill to Decode instances (network / async pipelining); what new bottleneck does the split introduce?
3
Why is Decode-instance load balancing based on occupied KV size / sequence length instead of connection or request counts? How do you avoid imbalance with mixed long and short requests?
4
Which phase decides TTFT vs ITL (TPOT)? How does PD split let you optimize them independently, and why do they fight each other when colocated?
5
Relation between PD split, continuous batching and chunked prefill: why is chunked prefill a "soft disaggregation" alternative, and what are the trade-offs of each?
📖 In-depth Guide:📄 high-concurrency-ai-system →
Updated 2026-08-12🎯
🚀 Start Card Practice ➔Test Your Knowledge: Practice Questions for "Prefill/Decode Disaggregation"
Single choice pitfall questions with instant feedback and mistake tracking.