M2-078M2: Classical Machine LearningMissing Values & Data LeakageMedium
⚡Flashcard Drill➔Mastery:
When Should You Add a 'Missing Indicator Variable'?
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Add a binary missing indicator whenever missingness conveys predictive intent (MNAR/informative missingness) or when pairing with mean/median imputation.
📌 Key Takeaways
- •Informative missingness: absence of data often correlates directly with the target (e.g., omitted credit history)
- •Variance preservation: allows linear models to decouple the imputed default value from true observed zeros/means
- •Redundancy check: native tree algorithms (LightGBM/XGBoost) learn missing pathways automatically
📐 Mathematical Derivations
Mathematical formulation in linear models: Suppose true data generating process is . If we impute with mean without an indicator, the model must fit , forcing the expectation of missing cases to lie exactly on the linear line at . By adding indicator , the model becomes: . For observed samples (), ; for missing samples (), , allowing the intercept to shift flexibly to capture distinct missingness behavior.
🏭 Production Trade-offs
Model-specific guidance: ① Linear Models and Neural Networks: Mandatory when using constant/mean imputation, as missingness signals are otherwise lost. ② Tree-based Models: LightGBM and XGBoost natively treat NaN as a separate category, testing left vs right splits for NaNs automatically. Adding explicit indicators is usually redundant for GBDTs unless missingness cross-interacts across multiple columns simultaneously.
⚠️ Common Interview Pitfalls
- ✕Adding missing indicators for features that are strictly MCAR with missingness, causing sparse column bloat
- ✕Forgetting to impute the continuous feature after creating the indicator, causing downstream matrix operations to fail
🎯 Interviewer Follow-up Inquiries
- ?How do modern GBDTs handle NaNs natively without requiring explicit missing indicators?
- ?Why is a missing indicator column especially critical for logistic regression in credit risk scoring?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚