M2-080M2: Classical Machine LearningMissing Values & Data LeakageHard
⚡Flashcard Drill➔Mastery:

Why Global Normalization Before Splitting Causes Data Leakage and How to Fix It

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M2-081 →
📐 Mathematical Definition
μtrain,σtrain only\mu_{train},\sigma_{train}\ \text{only}
⚡ One-Sentence Core Answer
Global standardization uses test set mean and variance to scale training data, artificially shrinking generalization error; fix by fitting scalers strictly on training folds.

📌 Key Takeaways

  • •
    Mechanism: computing μglobal\mu_{\text{global}} and σglobal\sigma_{\text{global}} incorporates test distribution information into training inputs
  • •
    Consequence: optimistic evaluation bias, masking true out-of-distribution generalization failure
  • •
    Fix: encapsulate transformations in Pipelines, fitting only on training data and transforming test data

📐 Mathematical Derivations

Mathematical Formulation: Let dataset D=Dtrain∪DtestD = D_{\text{train}} \cup D_{\text{test}}. Global standardization computes: μ=1Ntr+Nte(∑i∈Dtrxi+∑j∈Dtexj)\mu = \frac{1}{N_{\text{tr}} + N_{\text{te}}} \left( \sum_{i \in D_{\text{tr}}} x_i + \sum_{j \in D_{\text{te}}} x_j \right), σ2=1N∑k∈D(xk−μ)2\sigma^2 = \frac{1}{N} \sum_{k \in D} (x_k - \mu)^2.
When training samples are standardized via zi=xi−μσz_i = \frac{x_i - \mu}{\sigma}, their values depend directly on xj∈Dtestx_j \in D_{\text{test}}. This violates the fundamental statistical premise of supervised learning: f^=A(Dtrain)\hat{f} = \mathcal{A}(D_{\text{train}}) must be an independent mapping of DtrainD_{\text{train}} alone. If DtestD_{\text{test}} has a shifted mean μte≠μtr\mu_{\text{te}} \ne \mu_{\text{tr}}, global normalization partially aligns training features with the test domain, artificially inflating cross-validation performance.

🏭 Production Trade-offs

Correct Implementation Pattern:
```python
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

# Pipeline guarantees scaler.fit() is executed strictly on training folds
pipe = Pipeline([
('scaler', StandardScaler()),
('model', LogisticRegression())
])
pipe.fit(X_train, y_train)
`
During inference, use scaler.transform(X_test) using the frozen parameters (μtr,σtr)(\mu_{\text{tr}}, \sigma_{\text{tr}}).
⚠️ Common Interview Pitfalls
  • ✕
    Calling scaler.fit_transform(X) on the complete dataset before calling train_test_split
  • ✕
    Calling scaler.fit_transform(X_test) on the test set instead of scaler.transform(X_test)
🎯 Interviewer Follow-up Inquiries
  • ?
    Why is the performance inflation from global normalization worse when sample sizes are small?
  • ?
    What other preprocessing steps are prone to this exact same leakage failure mode?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM2-079: How to Systematically Audit Data Leakage? An Actionable Checklist📋 Back to BankNext →M2-081: Defining Model Calibration: Why High Accuracy Does Not Imply Good Calibration