MRR & GAUC

two industry-grade ranking/recommendation metrics. Mean reciprocal rank $MRR = \frac{1}{|Q|} \sum_{q \in Q} \frac{1}{rank_q}$, where $rank_q$ is the position of the first relevant item in the result list of query $q$, contributing 0 when nothing relevant exists; MRR lies in $(0, 1]$. Numerical example: for 3 queries whose first relevant item is at ranks 1, 3 and 5, $MRR = \frac{1}{3}(1 + \frac{1}{3} + \frac{1}{5}) = \frac{1}{3} \times 1.533 \approx 0.511$.

GAUC (Group AUC): compute an AUC per user, then average weighted by each user's sample size, $GAUC = \frac{\sum_u w_u \cdot AUC_u}{\sum_u w_u}$, typically $w_u$ = number of impressions/samples of user $u$. Numerical example: user A has 80 samples with $AUC_A = 0.7$, user B has 20 samples with $AUC_B = 0.6$, so $GAUC = \frac{80 \times 0.7 + 20 \times 0.6}{100} = \frac{56 + 12}{100} = 0.68$, whereas a simple average over users would give $(0.7 + 0.6)/2 = 0.65$.

search/QA/knowledge-retrieval “first hit” evaluation uses MRR; recsys fine-ranking (Alibaba's DIN paper) uses GAUC for per-user ranking quality; interviews often probe the difference, the hand computation, and industrial choice.

MRR cares only about the position of the first relevant result — in search, users barely scroll past the first screen, so extra relevant items and later positions matter little, and MRR focuses attention on “how fast the first hit appears”; global AUC is dominated by heavy active users in recommendation (long-tailed user distribution), so a few power users with good rankings inflate the global AUC and hide poor experience for most users; GAUC evaluates at user granularity and weights each user by sample size, mirroring the “one model per user” personalization view and treating long-tail users more fairly.

Test Your Knowledge: Practice Questions for "MRR & GAUC"

Single choice pitfall questions with instant feedback and mistake tracking.