CUPED Variance Reduction Derivation

Controlled-experiment Using Pre-Experiment Data (CUPED, Deng et al. 2013, Microsoft) establishes optimal variance reduction for online experimentation, slashing required sample sizes and experiment durations by up to 50% without altering sample traffic; Mathematical formulation: let $Y$ denote the experimental metric, and $X$ a pre-experiment covariate strictly independent of treatment (e.g. pre-experiment 7-day metric where $\mathbb{E}[X_T] = \mathbb{E}[X_C]$); constructing the unbiased adjusted estimator $\tilde{Y} = Y - \theta (X - \mathbb{E}[X])$, expanding variance yields $\text{Var}(\tilde{Y}) = \text{Var}(Y) + \theta^2 \text{Var}(X) - 2\theta \text{Cov}(Y, X)$; setting first derivative $\frac{\mathrm{d}\text{Var}}{\mathrm{d}\theta} = 0$ yields the optimal regression projection parameter: $\theta^* = \frac{\text{Cov}(Y, X)}{\text{Var}(X)}$; substituting $\theta^*$ yields the minimized variance: $\text{Var}(\tilde{Y}^*) = \text{Var}(Y) \cdot (1 - \rho_{XY}^2)$; when correlation $\rho = 0.7$, variance plummets by $51\%$, doubling experimental sensitivity.

Enterprise A/B experimentation platforms (Microsoft, Meta, ByteDance) and detecting sub-percent metric lifts.

Mature products struggle to measure 0.2% lifts due to high background user variance; CUPED strips out predictable user baseline variance, halving sample requirements at zero cost.

Test Your Knowledge: Practice Questions for "CUPED Variance Reduction Derivation"

Single choice pitfall questions with instant feedback and mistake tracking.