M1-049M1: Mathematics & Statistics FundamentalsConfidence Intervals & BootstrapMedium
⚡Flashcard Drill➔Mastery:
Contrast Bootstrap vs. Permutation Tests and Detail Their Respective Valid Assumptions and Use Cases
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Bootstrap resamples with replacement to estimate estimation uncertainty (confidence intervals); Permutation tests shuffle group labels without replacement to evaluate the sharp null hypothesis of identical distributions.
📌 Key Takeaways
- •Bootstrap: Resamples with replacement ( from ) under general population model; primary goal is Confidence Intervals and standard errors.
- •Permutation test: Shuffles labels without replacement under exchangeability assumption; primary goal is Exact P-values for hypothesis testing.
- •Exchangeability: Permutation strictly requires that under , observations across treatment and control are identically distributed and swappable.
📐 Mathematical Derivations
Permutation test mechanics: Testing against with observed statistic . Under , group labels ('Treatment' vs 'Control') are arbitrary. Pool all observations together. For : randomly shuffle the combined labels, reassign points to pseudo-group and points to pseudo-group , and compute . The exact two-sided p-value is . The added '1' in numerator and denominator ensures valid, non-zero exact frequentist type I error control.
🏭 Production Trade-offs
Practical selection guidelines: (1) Use Permutation Tests when sample size is small, distribution is completely unknown/skewed, and you require an exact, distribution-free p-value testing whether treatment has zero effect. (2) Use Bootstrap when you need to construct confidence intervals around treatment effect , quantify estimation uncertainty, or compute ratios/quantiles.
⚠️ Common Interview Pitfalls
- ✕Using permutation tests when group variances differ under the null hypothesis (Behrens-Fisher problem invalidates exchangeability, causing inflated false positives).
- ✕Using sampling with replacement in a permutation test (which turns it into a bootstrap test).
🎯 Interviewer Follow-up Inquiries
- ?Why does the permutation test provide an exact finite-sample test while bootstrap provides only asymptotic guarantees?
- ?How can permutation tests be applied to test independence between two continuous variables?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚