M7-011Hard评估体系 · Evaluation stack

如何评估一个排序模型的“公平性”?

How would you evaluate the fairness of a ranking model?
💡 考场高分一句话答案:

分群评估质量差异(NDCG/曝光/误差),检查是否存在系统性低估,并区分“统计差异”与“不合理差异”。

Measure quality differences across groups (NDCG, exposure, error rates), then separate statistical differences explained by relevance from unjustified ones.

📌 核心考察要点 (Key Points)

  • 度量:分群 NDCG、曝光占比 vs 人口占比、假阳性/假阴性率的群间差异
  • 判据:差异是否由相关性差异解释;无法解释的部分才是问题
  • 缓解:重采样、约束优化(在质量约束下最大化公平)、后处理阈值调整
  • 权衡:公平约束通常会降低整体指标,需要明确可接受的损失
  • Metrics: per-group NDCG, exposure share versus population share, group differences in false positive and false negative rates
  • Criterion: is the gap explained by genuine relevance differences? Only the unexplained part is a problem
  • Mitigation: resampling, constrained optimisation, or post-hoc threshold adjustment
  • Trade-off: fairness constraints usually cost overall metric performance, so the acceptable loss must be stated

⚖️ 业务权衡与取舍 (Trade-offs & Calibration)

公平性评估的难点是缺少 ground truth:如果标注本身有偏,分群指标也会继承偏差。因此需要先审计标注流程(标注者构成、标注指南),再谈模型公平性。这一点在面试里能体现你对数据链条的理解。

⚠️ 常见高频面试陷阱与避坑指南

  • 只看整体指标,不做分群
  • 把任何统计差异都判定为不公平(忽略真实相关性差异)

🔍 大厂面试官连环追问 (Follow-ups)

  • 如何处理标注偏差?
  • 公平约束会带来多大指标损失?

嫌字多看太累?进入苏格拉底式对话推演实验室

零打字负担,以大厂考官真实困境切入,通过点选直觉选项暴露思维盲区,手把手推导 Why。

💬 开启对话式破冰 ➔
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔