M2-075M2: Classical Machine LearningClass Imbalance LearningHard
⚡Flashcard Drill➔Mastery:
Cost-Sensitive Learning: Implementation and Relationship with Re-sampling
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Incorporate class weights into the loss function; it is asymptotically equivalent to re-sampling but avoids data duplication and stabilizes empirical training variance.
📌 Key Takeaways
- •Loss weighting: with inverse class frequencies
- •Equivalence with re-sampling: weighting an instance by contributes the same expected gradient as sampling it times
- •Key divergence: re-sampling alters the empirical data distribution, affecting calibration and batch norm statistics
📐 Mathematical Derivations
Mathematical Connection: Expected risk under importance sampling: . Oversampling the minority class by factor scales its empirical frequency from to , which corresponds to multiplying its loss term by .
Divergence Points: ① Variance: Oversampling draws duplicate samples with replacement, introducing discrete covariance across mini-batch gradients; class weighting applies deterministic scaling without inflating batch sample count. ② Calibration: Re-sampling shifts the model's internal prior towards 50/50, requiring post-hoc log-odds correction: . Class weighting maintains raw feature density.
Divergence Points: ① Variance: Oversampling draws duplicate samples with replacement, introducing discrete covariance across mini-batch gradients; class weighting applies deterministic scaling without inflating batch sample count. ② Calibration: Re-sampling shifts the model's internal prior towards 50/50, requiring post-hoc log-odds correction: . Class weighting maintains raw feature density.
🏭 Production Trade-offs
Implementation preference: In modern production pipelines, Class Weights or Focal Loss are preferred over oversampling due to zero memory overhead, reproducible deterministic batches, and absence of synthetic sample artifacts. Reserve undersampling for situations where the majority class is overwhelmingly large ( rows) and compute budget is the primary bottleneck.
⚠️ Common Interview Pitfalls
- ✕Treating class-weighted model outputs as raw calibrated probabilities without performing Bayesian prior correction
- ✕Believing oversampling provides new information, whereas it simply repeats existing samples and increases overfitting risk
🎯 Interviewer Follow-up Inquiries
- ?How do you mathematically correct the output probabilities of a model trained on an undersampled dataset?
- ?Under what conditions does Focal Loss provide superior gradients compared to simple inverse class weighting?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚