🏗️ System DesignID: recsys-offline-metrics

RecSys Metrics: GAUC & NDCG

Q: Derive mathematically why Global AUC suffers cross-user bias and demonstrate cases where Global AUC rises while GAUC drops?

【RecSys Metrics: GAUC & NDCG 核心考点 1】Derive mathematically why Global AUC suffers cross-user bias and demonstrate cases where Global AUC rises while GAUC drops?。技术解析：Definition: RecSys Offline Evaluation Metrics quantify ranking quality, personalization accuracy, and relevance fidelity before pushing model candidates online; the two flagship metrics are: 1) GAUC (Group Area Under ROC Curve, weighted by user impression volume w_u): \text{GAUC}... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/recsys-offline-metrics

Q: Derive NDCG@K, DCG@K, and IDCG@K formulas, explaining the psychological grounding of logarithmic position discount $\log_2(i+1)$?

【RecSys Metrics: GAUC & NDCG 核心考点 2】Derive NDCG@K, DCG@K, and IDCG@K formulas, explaining the psychological grounding of logarithmic position discount $\log_2(i+1)$?。技术解析：Definition: RecSys Offline Evaluation Metrics quantify ranking quality, personalization accuracy, and relevance fidelity before pushing model candidates online; the two flagship metrics are: 1) GAUC (Group Area Under ROC Curve, weighted by user impression volume w_u): \text{GAUC}... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/recsys-offline-metrics

Q: How should users with only positive (all-click) or only negative (zero-click) labels be filtered/handled when aggregating GAUC?

【RecSys Metrics: GAUC & NDCG 核心考点 3】How should users with only positive (all-click) or only negative (zero-click) labels be filtered/handled when aggregating GAUC?。技术解析：Definition: RecSys Offline Evaluation Metrics quantify ranking quality, personalization accuracy, and relevance fidelity before pushing model candidates online; the two flagship metrics are: 1) GAUC (Group Area Under ROC Curve, weighted by user impression volume w_u): \text{GAUC}... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/recsys-offline-metrics

Q: Compare MRR (Mean Reciprocal Rank) vs MAP (Mean Average Precision) across single-positive vs multi-positive recommendation tasks?

【RecSys Metrics: GAUC & NDCG 核心考点 4】Compare MRR (Mean Reciprocal Rank) vs MAP (Mean Average Precision) across single-positive vs multi-positive recommendation tasks?。技术解析：Definition: RecSys Offline Evaluation Metrics quantify ranking quality, personalization accuracy, and relevance fidelity before pushing model candidates online; the two flagship metrics are: 1) GAUC (Group Area Under ROC Curve, weighted by user impression volume w_u): \text{GAUC}... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/recsys-offline-metrics

Q: Why is a +0.005 GAUC improvement considered a massive production milestone directly driving millions in revenue?

【RecSys Metrics: GAUC & NDCG 核心考点 5】Why is a +0.005 GAUC improvement considered a massive production milestone directly driving millions in revenue?。技术解析：Definition: RecSys Offline Evaluation Metrics quantify ranking quality, personalization accuracy, and relevance fidelity before pushing model candidates online; the two flagship metrics are: 1) GAUC (Group Area Under ROC Curve, weighted by user impression volume w_u): \text{GAUC}... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/recsys-offline-metrics

推荐离线指标 GAUC 与 NDCG

🎯Core Definition

RecSys Offline Evaluation Metrics quantify ranking quality, personalization accuracy, and relevance fidelity before pushing model candidates online; the two flagship metrics are: 1) GAUC (Group Area Under ROC Curve, weighted by user impression volume

w_u

\text{GAUC} = \frac{\sum_{u} w_u \cdot \text{AUC}_u}{\sum_{u} w_u}

, measuring intra-user personalization ranking ability; 2) NDCG@K (Normalized Discounted Cumulative Gain):

\text{NDCG@K} = \frac{\text{DCG@K}}{\text{IDCG@K}}

, where

\text{DCG@K} = \sum_{i=1}^K \frac{2^{r_i} - 1}{\log_2(i + 1)}

, rewarding highly relevant items positioned near top ranking slots via logarithmic position discounting.

💡Use Cases

Offline model validation, hyperparameter tuning, and checkpoint candidate screening for online deployment.

⚡Key Problems Solved

Global AUC is heavily distorted by cross-user baseline CTR disparities (models can score high Global AUC purely by separating active users from inactive users while failing intra-user ranking completely); GAUC isolates true per-user personalization power.

🎯5 High-Frequency Exam Points

Derive mathematically why Global AUC suffers cross-user bias and demonstrate cases where Global AUC rises while GAUC drops?

Derive NDCG@K, DCG@K, and IDCG@K formulas, explaining the psychological grounding of logarithmic position discount

\log_2(i+1)

How should users with only positive (all-click) or only negative (zero-click) labels be filtered/handled when aggregating GAUC?

Compare MRR (Mean Reciprocal Rank) vs MAP (Mean Average Precision) across single-positive vs multi-positive recommendation tasks?

Why is a +0.005 GAUC improvement considered a massive production milestone directly driving millions in revenue?

📖 In-depth Guide:📄 recommendation-system-design →

Updated 2026-08-14

🎯

Test Your Knowledge: Practice Questions for "RecSys Metrics: GAUC & NDCG"

Single choice pitfall questions with instant feedback and mistake tracking.

🚀 Start Card Practice ➔

← Previous CardReranking Diversity & DPP Process Next Card →Online A/B Testing & Interleaving

🔗 More System Design Knowledge Cards

RecSys Multi-Stage Funnel & 50ms SLA DSSM Two-Tower Retrieval YouTube DNN Candidate Generation Pre-Ranking Lightweight Architecture