M7-011Hard评估体系 · Evaluation stack
如何评估一个排序模型的“公平性”?
How would you evaluate the fairness of a ranking model?
💡 考场高分一句话答案:
分群评估质量差异(NDCG/曝光/误差),检查是否存在系统性低估,并区分“统计差异”与“不合理差异”。
Measure quality differences across groups (NDCG, exposure, error rates), then separate statistical differences explained by relevance from unjustified ones.
📌 核心考察要点 (Key Points)
- 度量:分群 NDCG、曝光占比 vs 人口占比、假阳性/假阴性率的群间差异
- 判据:差异是否由相关性差异解释;无法解释的部分才是问题
- 缓解:重采样、约束优化(在质量约束下最大化公平)、后处理阈值调整
- 权衡:公平约束通常会降低整体指标,需要明确可接受的损失
- Metrics: per-group NDCG, exposure share versus population share, group differences in false positive and false negative rates
- Criterion: is the gap explained by genuine relevance differences? Only the unexplained part is a problem
- Mitigation: resampling, constrained optimisation, or post-hoc threshold adjustment
- Trade-off: fairness constraints usually cost overall metric performance, so the acceptable loss must be stated
⚖️ 业务权衡与取舍 (Trade-offs & Calibration)
公平性评估的难点是缺少 ground truth:如果标注本身有偏,分群指标也会继承偏差。因此需要先审计标注流程(标注者构成、标注指南),再谈模型公平性。这一点在面试里能体现你对数据链条的理解。
⚠️ 常见高频面试陷阱与避坑指南
- 只看整体指标,不做分群
- 把任何统计差异都判定为不公平(忽略真实相关性差异)
🔍 大厂面试官连环追问 (Follow-ups)
- 如何处理标注偏差?
- 公平约束会带来多大指标损失?
嫌字多看太累?进入苏格拉底式对话推演实验室
零打字负担,以大厂考官真实困境切入,通过点选直觉选项暴露思维盲区,手把手推导 Why。
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.