M1-083M1: Mathematics & Statistics Fundamentals实验设计 (A/B)Medium
⚡Flashcard Drill➔Mastery:

Explain Stratified Randomization in Experimental Design and When It Is Strictly Required

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M1-084 →
📐 Mathematical Definition
stratified:randomize within each stratum s\text{stratified}: \text{randomize within each stratum } s
⚡ One-Sentence Core Answer
Stratified randomization divides the population into homogeneous strata based on key covariates (e.g. Operating system, spend tier) and randomizes within each stratum, guaranteeing perfect covariate balance and reducing treatment effect variance.

📌 Key Takeaways

  • •
    Core Mechanism: Partitions population into KK mutually exclusive strata S1,…,SKS_1, \dots, S_K; within each stratum, exactly 50%50\% of units are allocated to Treatment and 50%50\% to Control.
  • •
    Guaranteed Balance: Eliminates the possibility of accidental pre-experiment covariate imbalance (e.g. 60% iOS in treatment vs 40% in control).
  • •
    Variance Reduction: Removes between-strata variance from the standard error of treatment effect: Var(τ^strat)≤Var(τ^simple)\text{Var}(\hat{\tau}_{\text{strat}}) \le \text{Var}(\hat{\tau}_{\text{simple}}).

📐 Mathematical Derivations

By the Law of Total Variance, population metric variance decomposes into: σ2=Var(Y)=∑k=1Kpkσk2+∑k=1Kpk(μk−μ)2=σwithin2+σbetween2\sigma^2 = \text{Var}(Y) = \sum_{k=1}^K p_k \sigma_k^2 + \sum_{k=1}^K p_k (\mu_k - \mu)^2 = \sigma_{\text{within}}^2 + \sigma_{\text{between}}^2, where pkp_k is stratum weight and σk2\sigma_k^2 is within-stratum variance. Under simple random assignment, variance of estimated treatment effect is Var(τ^simple)=2σ2N=2(σwithin2+σbetween2)N\text{Var}(\hat{\tau}_{\text{simple}}) = \frac{2\sigma^2}{N} = \frac{2(\sigma_{\text{within}}^2 + \sigma_{\text{between}}^2)}{N}. Under stratified randomization, because each stratum is balanced exactly, the between-strata component σbetween2\sigma_{\text{between}}^2 is completely purged from estimation noise: Var(τ^strat)=∑k=1Kpk2(2σk2Nk)=2σwithin2N\text{Var}(\hat{\tau}_{\text{strat}}) = \sum_{k=1}^K p_k^2 \left(\frac{2\sigma_k^2}{N_k}\right) = \frac{2\sigma_{\text{within}}^2}{N}.

🏭 Production Trade-offs

When stratified randomization is mandatory: (1) Small sample experiments (N<5000N < 5000, such as enterprise B2B A/B tests or hospital trials) where simple random sampling frequently produces bad imbalances. (2) Presence of heavy whales: In gaming or ad monetization, stratified bucketing based on historical spend tiers prevents top-spending 'whales' from clustering by chance into one bucket. In large-scale consumer apps (N>107N > 10^7), simple randomization achieves balance by the Law of Large Numbers, making post-experiment CUPED adjustment mathematically equivalent and operationally simpler.
⚠️ Common Interview Pitfalls
  • ✕
    Stratifying on post-treatment variables (must strictly stratify on pre-treatment baseline covariates).
  • ✕
    Over-stratifying across too many crossed variables, leading to empty or single-unit strata.
🎯 Interviewer Follow-up Inquiries
  • ?
    Why is post-stratification (or CUPED ANCOVA) asymptotically equivalent to pre-experiment stratified randomization in large samples?
  • ?
    How does re-randomization (Morgan & Rubin, 2012) reject unbalanced assignments before deploying treatments?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM1-082: Explain Block Bootstrap and Its Vital Role in Preserving Dependent Time Series Autocorrelation📋 Back to BankNext →M1-084: Explain Switchback Experiments and Their Applications in Marketplace and Two-Sided Delivery Platforms