Capacity Estimation & Latency Budgeting

Capacity Estimation & Latency Budgeting is the first-principles quantitative derivation of throughput, storage footprint, network bandwidth, hardware compute costs, and latency SLAs during System Design interviews and engineering capacity planning; it formalizes 5 core derivation equations: 1) Throughput: $\text{Average QPS} = \frac{\text{DAU} \times \text{Requests/User}}{86,400\text{s}}$, with $\text{Peak QPS} = (2 \sim 5) \times \text{Average QPS}$; 2) Storage: $\text{Daily Storage} = \text{DAU} \times \text{Events/User} \times \text{Bytes/Event}$, scaled across 5-year retention with 3x replication factor; 3) Network Bandwidth: $\text{Peak QPS} \times \text{Payload Bytes} \times 8\text{ bits}$; 4) GPU Compute Fleet: $\text{GPUs} = \lceil \frac{\text{Peak QPS} \times \text{Latency/Request}}{\text{BatchConcurrency/GPU}} \rceil$; 5) Latency SLA decomposition allocating strict millisecond slices across Network RTT, Feature Store IO, Neural Forward Passes, and Serialization.

Quantitative phase of ML system design interviews, cloud infrastructure cost budgeting, and microservice SLA contract definitions.

Qualitative designs without numbers collapse in production (e.g. proposing a single PostgreSQL node for a 10M QPS streaming event ingest); rigorous back-of-the-envelope calculations mathematically justify component selection (Redis vs S3).

Test Your Knowledge: Practice Questions for "Capacity Estimation & Latency Budgeting"

Single choice pitfall questions with instant feedback and mistake tracking.