M2-071M2: Classical Machine LearningClass Imbalance LearningEasy
⚡Flashcard Drill➔Mastery:

Major Approaches for Handling Class Imbalance

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M2-072 →
📐 Mathematical Definition
class weight: wc=N2Nc\text{class weight}:\ w_c=\frac{N}{2N_c}
⚡ One-Sentence Core Answer
Address imbalance at the data level (re-sampling), algorithmic level (cost-sensitive loss, Focal Loss), and post-processing level (threshold tuning).

📌 Key Takeaways

  • •
    Data-level: random undersampling, oversampling, SMOTE, ADASYN
  • •
    Algorithm-level: class-weighted loss, Focal Loss, balance cascade ensembling
  • •
    Post-processing: probability calibration followed by cost-optimal threshold shifting

📐 Mathematical Derivations

Taxonomy of Class Imbalance Solutions: ① Re-sampling: Adjust the empirical class distribution P(y)P(y) to P′(y)P'(y). Undersampling drops majority samples (discards data); Oversampling replicates minority samples (risks overfitting); Synthetic generation (SMOTE) interpolates between nearest minority neighbors. ② Cost-Sensitive Learning: Modify the loss objective L=−∑iwyiℓ(yi,y^i)\mathcal{L} = - \sum_i w_{y_i} \ell(y_i, \hat{y}_i) with inverse-frequency weights wc=NK⋅Ncw_c = \frac{N}{K \cdot N_c}. ③ Focal Loss: LFL=−αt(1−pt)γlog⁡pt\mathcal{L}_{\text{FL}} = - \alpha_t (1 - p_t)^\gamma \log p_t, dynamically downweighting easy negative examples. ④ Threshold Moving: Adjust decision boundary τ\tau based on Bayes risk or business trade-offs.

🏭 Production Trade-offs

Engineering decision matrix: In massive datasets (N>108N > 10^8), Random Undersampling combined with ensembling (e.g., EasyEnsemble) is computationally optimal. In medium datasets, Class Weights or Focal Loss are preferred because they preserve original data distribution and maintain consistent sample sizes. For business KPIs, always couple model training with rigorous post-calibration threshold optimization.
⚠️ Common Interview Pitfalls
  • ✕
    Evaluating models on imbalanced data using accuracy instead of PR-AUC, F1, or cost-weighted matrices
  • ✕
    Applying SMOTE to the entire dataset before train/test splitting, leaking synthetic test points into training
🎯 Interviewer Follow-up Inquiries
  • ?
    When is random undersampling strictly superior to SMOTE in large-scale production systems?
  • ?
    Why does modifying class weights during training distort predicted probabilities, and how do you calibrate them?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM2-070: How to Evaluate Feature Selection Effectiveness? Key Considerations and Validation📋 Back to BankNext →M2-072: SMOTE Algorithm: Core Mechanism, Theoretical Risks, and Limitations