M2-076M2: Classical Machine LearningMissing Values & Data LeakageEasy
⚡Flashcard Drill➔Mastery:
Three Missing Data Mechanisms (MCAR, MAR, MNAR) and Treatment Strategies
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Rubin's taxonomy defines MCAR (completely random), MAR (conditional on observed data), and MNAR (dependent on unobserved missing values); treatments range from simple imputation to pattern modeling.
📌 Key Takeaways
- •MCAR: missingness independent of both observed and unobserved data; deletion is unbiased but loses power
- •MAR: missingness depends on observed features; multiple imputation (MICE) and tree surrogate splits are valid
- •MNAR: missingness depends on the missing value itself; requires explicit missing indicators or Heckman selection models
📐 Mathematical Derivations
Let and be the binary missingness indicator ( if missing).
① MCAR (Missing Completely at Random): . Deletion retains unbiased parameter estimates. ② MAR (Missing at Random): . Missingness is explained entirely by observable covariates (e.g., young people are less likely to report income, but conditional on age, missingness is random). Standard imputation methods (KNN, MICE, EM) are unbiased. ③ MNAR (Missing Not at Random): depends on directly (e.g., individuals with extremely high income systematically refuse to report income). Standard imputation is biased; requires joint modeling (Heckman selection model or pattern-mixture models).
① MCAR (Missing Completely at Random): . Deletion retains unbiased parameter estimates. ② MAR (Missing at Random): . Missingness is explained entirely by observable covariates (e.g., young people are less likely to report income, but conditional on age, missingness is random). Standard imputation methods (KNN, MICE, EM) are unbiased. ③ MNAR (Missing Not at Random): depends on directly (e.g., individuals with extremely high income systematically refuse to report income). Standard imputation is biased; requires joint modeling (Heckman selection model or pattern-mixture models).
🏭 Production Trade-offs
Practical engineering strategies: In tabular ML, always append a Missing Indicator Column () alongside continuous imputation. This preserves MNAR informative absence. Tree algorithms like LightGBM and XGBoost natively assign missing values to optimal split directions during training.
⚠️ Common Interview Pitfalls
- ✕Using listwise deletion on MAR/MNAR data, introducing severe selection bias
- ✕Imputing missing values with mean or median without adding a missing indicator, destroying MNAR signal
🎯 Interviewer Follow-up Inquiries
- ?How does XGBoost determine which branch a missing value should traverse during inference?
- ?What statistical test can be used to distinguish MCAR from MAR (e.g., Little's MCAR test)?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚