M2-073M2: Classical Machine LearningClass Imbalance LearningMedium
⚡Flashcard Drill➔Mastery:

Why Prioritize PR-AUC Over ROC-AUC in Highly Imbalanced Regimes?

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M2-074 →
📐 Mathematical Definition
PR-AUC 更敏感于 #P\text{PR-AUC}\ \text{更敏感于}\ \#P
⚡ One-Sentence Core Answer
ROC-AUC uses False Positive Rate, which is diluted by a massive negative class; PR-AUC uses Precision, which directly penalizes false positives against the rare positive class.

📌 Key Takeaways

  • •
    FPR dilution: FPR=FP/(FP+TN)≈FP/TN\text{FPR} = \text{FP} / (\text{FP} + \text{TN}) \approx \text{FP} / \text{TN} remains artificially small when TN is huge
  • •
    Precision sensitivity: Precision=TP/(TP+FP)\text{Precision} = \text{TP} / (\text{TP} + \text{FP}) drops drastically even with modest false alarms
  • •
    Baseline difference: ROC-AUC baseline is always 0.5; PR-AUC baseline equals the positive class prevalence P/(P+N)P / (P + N)

📐 Mathematical Derivations

Mathematical proof of dilution: Let positive count P=1,000P = 1,000 and negative count N=1,000,000N = 1,000,000 (prevalence π=0.001\pi = 0.001). Suppose a model produces FP=10,000\text{FP} = 10,000 and TP=800\text{TP} = 800.
ROC Metric: TPR=800/1000=0.80\text{TPR} = 800 / 1000 = 0.80; FPR=10,000/1,000,000=0.01\text{FPR} = 10,000 / 1,000,000 = 0.01. The ROC curve reflects an apparently exceptional point (FPR=0.01,TPR=0.80)(\text{FPR}=0.01, \text{TPR}=0.80), yielding an ROC-AUC well above 0.95.
PR Metric: Recall=0.80\text{Recall} = 0.80, but Precision=800/(800+10,000)=800/10,800≈0.074\text{Precision} = 800 / (800 + 10,000) = 800 / 10,800 \approx 0.074. In practice, over 92% of flagged alerts are false alarms, which the PR curve exposes transparently.

🏭 Production Trade-offs

Diagnostic rules: Use ROC-AUC when the performance on both classes matters equally or when evaluating class-invariant ranking capability. Use PR-AUC when the positive class is rare and false positives carry high operational costs (fraud detection, rare cancer diagnosis, search ad click prediction).
⚠️ Common Interview Pitfalls
  • ✕
    Reporting an ROC-AUC of 0.98 on 1:1000 imbalanced data as evidence of high accuracy, masking a precision of under 5%
  • ✕
    Comparing PR-AUC to 0.5 as a baseline, forgetting that a random classifier's PR-AUC equals the minority class prior π\pi
🎯 Interviewer Follow-up Inquiries
  • ?
    Can a model have an ROC-AUC of 0.99 and a PR-AUC of 0.05 simultaneously? Explain with numbers.
  • ?
    Why is the baseline of PR-AUC dependent on the class distribution while ROC-AUC is strictly 0.5?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM2-072: SMOTE Algorithm: Core Mechanism, Theoretical Risks, and Limitations📋 Back to BankNext →M2-074: Classification Threshold Selection Strategies Across Diverse Business Objectives