RecSys Metrics: GAUC & NDCG

RecSys Offline Evaluation Metrics quantify ranking quality, personalization accuracy, and relevance fidelity before pushing model candidates online; the two flagship metrics are: 1) GAUC (Group Area Under ROC Curve, weighted by user impression volume $w_u$): $\text{GAUC} = \frac{\sum_{u} w_u \cdot \text{AUC}_u}{\sum_{u} w_u}$, measuring intra-user personalization ranking ability; 2) NDCG@K (Normalized Discounted Cumulative Gain): $\text{NDCG@K} = \frac{\text{DCG@K}}{\text{IDCG@K}}$, where $\text{DCG@K} = \sum_{i=1}^K \frac{2^{r_i} - 1}{\log_2(i + 1)}$, rewarding highly relevant items positioned near top ranking slots via logarithmic position discounting.

Offline model validation, hyperparameter tuning, and checkpoint candidate screening for online deployment.

Global AUC is heavily distorted by cross-user baseline CTR disparities (models can score high Global AUC purely by separating active users from inactive users while failing intra-user ranking completely); GAUC isolates true per-user personalization power.

Derive mathematically why Global AUC suffers cross-user bias and demonstrate cases where Global AUC rises while GAUC drops?

Derive NDCG@K, DCG@K, and IDCG@K formulas, explaining the psychological grounding of logarithmic position discount $\log_2(i+1)$?

How should users with only positive (all-click) or only negative (zero-click) labels be filtered/handled when aggregating GAUC?

Compare MRR (Mean Reciprocal Rank) vs MAP (Mean Average Precision) across single-positive vs multi-positive recommendation tasks?

Why is a +0.005 GAUC improvement considered a massive production milestone directly driving millions in revenue?