M2-073M2: Classical Machine LearningClass Imbalance LearningMedium
⚡Flashcard Drill➔Mastery:
Why Prioritize PR-AUC Over ROC-AUC in Highly Imbalanced Regimes?
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
ROC-AUC uses False Positive Rate, which is diluted by a massive negative class; PR-AUC uses Precision, which directly penalizes false positives against the rare positive class.
📌 Key Takeaways
- •FPR dilution: remains artificially small when TN is huge
- •Precision sensitivity: drops drastically even with modest false alarms
- •Baseline difference: ROC-AUC baseline is always 0.5; PR-AUC baseline equals the positive class prevalence
📐 Mathematical Derivations
Mathematical proof of dilution: Let positive count and negative count (prevalence ). Suppose a model produces and .
ROC Metric: ; . The ROC curve reflects an apparently exceptional point , yielding an ROC-AUC well above 0.95.
PR Metric: , but . In practice, over 92% of flagged alerts are false alarms, which the PR curve exposes transparently.
ROC Metric: ; . The ROC curve reflects an apparently exceptional point , yielding an ROC-AUC well above 0.95.
PR Metric: , but . In practice, over 92% of flagged alerts are false alarms, which the PR curve exposes transparently.
🏭 Production Trade-offs
Diagnostic rules: Use ROC-AUC when the performance on both classes matters equally or when evaluating class-invariant ranking capability. Use PR-AUC when the positive class is rare and false positives carry high operational costs (fraud detection, rare cancer diagnosis, search ad click prediction).
⚠️ Common Interview Pitfalls
- ✕Reporting an ROC-AUC of 0.98 on 1:1000 imbalanced data as evidence of high accuracy, masking a precision of under 5%
- ✕Comparing PR-AUC to 0.5 as a baseline, forgetting that a random classifier's PR-AUC equals the minority class prior
🎯 Interviewer Follow-up Inquiries
- ?Can a model have an ROC-AUC of 0.99 and a PR-AUC of 0.05 simultaneously? Explain with numbers.
- ?Why is the baseline of PR-AUC dependent on the class distribution while ROC-AUC is strictly 0.5?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚