Sample Size Derivation & MDE Budget

The A/B Test Minimum Sample Size Formula & MDE Derivation establishes the exact sample requirements per variant under significance $\alpha$ and power $1 - \beta$; for continuous mean metrics, equating critical $Z$-score thresholds under $H_0$ and $H_1$ yields the closed-form sample size formula: $n = \frac{(z_{\alpha/2} + z_\beta)^2 \cdot 2\sigma^2}{\delta^2}$ (where $\delta = |\mu_T - \mu_C|$ is target effect size, $\sigma^2$ metric variance); for binomial ratio metrics (CTR/CVR with baseline probability $p$), substituting $\sigma^2 = \bar{p}(1-\bar{p})$ derives: $n = \frac{(z_{\alpha/2} + z_\beta)^2 \cdot 2 \bar{p}(1-\bar{p})}{\delta^2}$; inverted under fixed traffic constraints, it computes the Minimum Detectable Effect: $\text{MDE} = (z_{\alpha/2} + z_\beta) \sqrt{\frac{2\sigma^2}{n}}$.

Pre-experiment traffic budgeting, duration scheduling, and statistical sensitivity audits.

Underpowered tests miss real wins due to noisy variance, while overpowered tests waste valuable user traffic and delay product deployment cycles.

Test Your Knowledge: Practice Questions for "Sample Size Derivation & MDE Budget"

Single choice pitfall questions with instant feedback and mistake tracking.