📈 Data ScientistID:
ds-p-hacking-peeking-msprtP-hacking, Peeking & mSPRT Sequential
P-hacking 偷窥与序贯检验 mSPRT
🎯Core Definition
The Continuous Monitoring / Peeking Problem & Mixture Sequential Probability Ratio Test (mSPRT) establishes rigorous martingales-based defenses against false-positive rate inflation during continuous experimental monitoring; Peeking Pathology: repeatedly computing -values daily and terminating early upon reaching inflates the nominal false positive rate to $>30-40\%$ within 30 days under the null hypothesis $H_0$ due to random walk fluctuations crossing critical boundaries; Statistical Remedies: 1) Pre-registered Fixed-Horizon Testing (enforcing full-duration runs); 2) Group Sequential Boundaries (O'Brien-Fleming / Pocock alpha-spending functions); 3) Continuous Sequential Testing via mSPRT (constructing martingale-based Always-Valid -values and time-uniform Confidence Sequences) enabling safe continuous monitoring and anytime early stopping.
💡Use Cases
Real-time experiment monitoring in enterprise A/B platforms (Netflix, Uber, ByteDance) and rapid negative-impact kill switches.
⚡Key Problems Solved
Eliminates rampant -hacking caused by stakeholders prematurely halting experiments on transient random fluctuations, ensuring mathematically provable Type-I error control.
🎯5 High-Frequency Exam Points
1
Derive why Brownian Motion first hitting times cause cumulative Type-I error rates to asymptotically approach 100% under continuous unadjusted peeking?
2
Derive the mathematical proof of mSPRT's error bounds using non-negative martingales and Doob's Maximal Inequality?
3
Explain how Lan-DeMets alpha-spending functions allocate Type-I error budgets conservatively across interim looks?
4
Why do Always-Valid Confidence Sequences guarantee coverage probability uniformly across all stopping times?
5
When to deploy mSPRT for rapid disaster mitigation vs standard fixed-horizon testing for retention metrics with long carryover?
🔗Foundational Prerequisite Cards (Click to Review)
📖 In-depth Guide:📄 ds-statistics-and-experiment-design →
Updated 2026-08-14🎯
🚀 Start Card Practice ➔Test Your Knowledge: Practice Questions for "P-hacking, Peeking & mSPRT Sequential"
Single choice pitfall questions with instant feedback and mistake tracking.