M2-109M2: Classical Machine Learning梯度提升 (GBDT/XGBoost)Medium
⚡Flashcard Drill➔Mastery:

Methods for Handling Class Imbalance in GBDT and Their Trade-offs

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M2-110 →
📐 Mathematical Definition
scale_pos_weight=#neg#pos\text{scale\_pos\_weight}=\frac{\#\text{neg}}{\#\text{pos}}
⚡ One-Sentence Core Answer
GBDT handles imbalance via positive class loss reweighting (scale_pos_weight), bagging undersampling, or custom focal loss; each affects calibration and split gains differently.

📌 Key Takeaways

  • •
    scale_pos_weight: multiplies minority positive class gradients and Hessians by factor w=Nneg/Nposw = N_{\text{neg}} / N_{\text{pos}}
  • •
    Gradient impact: positive samples contribute w⋅giw \cdot g_i and w⋅hiw \cdot h_i, shifting split gain toward positive recall
  • •
    Calibration distortion: scale_pos_weight distorts output probability calibration, requiring post-hoc recalibration

📐 Mathematical Derivations

Mechanisms in GBDT:
① `scale_pos_weight` ($w$): Modifies the binary cross-entropy loss: L=−∑i∈poswlog⁡pi−∑j∈neglog⁡(1−pj)\mathcal{L} = - \sum_{i \in \text{pos}} w \log p_i - \sum_{j \in \text{neg}} \log(1 - p_j). The first and second derivatives for positive samples scale directly: gi=w(pi−1)g_i = w(p_i - 1), hi=wpi(1−pi)h_i = w p_i(1 - p_i). This amplifies positive instance contributions in the split gain formula without changing physical dataset size.
② Focal Loss: Downweights easy negative examples via (1−pt)γ(1 - p_t)^\gamma, preventing massive easy negatives from swamping split gradients.
③ Sub-sampling (Bagging): Downsamples majority negatives per boosting tree iteration, accelerating training while maintaining ensemble diversity.

🏭 Production Trade-offs

Engineering decision: Use scale_pos_weight when tuning for maximum PR-AUC or Recall. However, if predictions must serve as calibrated probabilities in downstream economic calculations (e.g., expected ad revenue), avoid scale_pos_weight and use threshold tuning or isotonic recalibration.
⚠️ Common Interview Pitfalls
  • ✕
    Using scale_pos_weight and directly interpreting model output as true positive probability without sigmoid logit correction
  • ✕
    Applying SMOTE to GBDT pipelines, which introduces noisy synthetic samples and slows training compared to native scale_pos_weight
🎯 Interviewer Follow-up Inquiries
  • ?
    How do you mathematically recover calibrated true probabilities from a GBDT trained with scale_pos_weight = ww?
  • ?
    Why is focal loss technically challenging to implement in standard GBDT frameworks (dealing with non-positive Hessians)?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM2-108: Monotonicity Constraints in GBDT and Their Industrial Business Value📋 Back to BankNext →M2-110: Early Stopping in GBDT and Determining the Optimal Number of Iterations