M7-003Medium排序与推荐 · Ranking & recsys🔥 大厂高频出现 9 次
AUC 有什么局限?什么场景下不该用 AUC?
What are the limitations of AUC, and when should you not use it?
💡 考场高分一句话答案:
AUC 衡量整体排序,与 top-k 表现无关;类别极不平衡或关心校准(如出价)时不该用。
AUC measures global ranking and is unrelated to top-k performance; it is also inflated under extreme class imbalance and says nothing about calibration.
📌 核心考察要点 (Key Points)
- 与业务脱节:业务只关心前 3 条,AUC 却对全量排序敏感
- 不平衡下的虚高:负样本极多时,AUC 容易看起来很好
- 不反映校准:AUC 只看排序,不看概率是否可信
- 替代:precision@k、NDCG@k、校准曲线(ECE / reliability diagram)
- Disconnected from the business: the product only shows the top three, but AUC is sensitive to the whole ordering
- Imbalance: with overwhelmingly many negatives AUC looks good easily
- No calibration: ranking quality is not probability quality
- Alternatives: precision@k, NDCG@k, calibration curves and ECE
⚖️ 业务权衡与取舍 (Trade-offs & Calibration)
广告出价场景里 AUC 尤其危险:出价需要的是校准后的概率(pCTR × value),而不是排序。一个 AUC 很高但系统性高估 2 倍的模型会导致超支。因此这类场景必须看校准误差与分桶后的实际 CTR。
⚠️ 常见高频面试陷阱与避坑指南
- 用 AUC 比较两个模型的业务价值
- 忽略校准,直接用模型输出做乘法(pCTR × bid)
🔍 大厂面试官连环追问 (Follow-ups)
- 如何画校准曲线?
- 为什么 pCTR 必须校准才能用于出价?
🎯 关联大厂考题实战训练 (6 题)
嫌字多看太累?进入苏格拉底式对话推演实验室
零打字负担,以大厂考官真实困境切入,通过点选直觉选项暴露思维盲区,手把手推导 Why。
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.