NDCG

a ranking quality metric measuring how well highly relevant documents are placed at the top. Discounted cumulative gain $DCG@k = \sum_{i=1}^{k} \frac{2^{rel_i} - 1}{\log_2(i+1)}$, where $rel_i$ is the graded relevance at position $i$ (typically 0-4) and $\log_2(i+1)$ is the position discount (1 at rank 1, $\log_2 3 \approx 1.585$ at rank 2, 2 at rank 3, $\log_2 5 \approx 2.322$ at rank 4). IDCG is the DCG of the ideal ranking (relevance descending), and $NDCG@k = \frac{DCG@k}{IDCG@k} \in [0,1]$. Numerical example: true relevance [3, 2, 1, 0], model output order [2, 1, 3, 0], then $DCG@4 = \frac{2^2-1}{1} + \frac{2^1-1}{\log_2 3} + \frac{2^3-1}{2} + \frac{2^0-1}{\log_2 5} = 3 + 0.63 + 3.5 + 0 = 7.13$; IDCG with the ideal order [3, 2, 1, 0]: $IDCG@4 = \frac{2^3-1}{1} + \frac{2^2-1}{\log_2 3} + \frac{2^1-1}{2} + 0 = 7 + 1.89 + 0.5 = 9.39$, so $NDCG@4 \approx 7.13 / 9.39 \approx 0.76$.

search ranking and recsys fine-ranking (LambdaRank / LTR optimized for NDCG); hand computation is an interview staple; the truncation k means “only the top k positions count”, usually k = 5 or 10.

precision/recall treat all relevant documents equally, ignoring position and degree of relevance; $2^{rel}-1$ exponentially amplifies the gain of highly relevant documents (a relevance-4 item contributes ~2.3× a relevance-3 item) and $\log_2(i+1)$ penalizes lower positions, encoding “more relevant and higher up = more credit”; dividing by IDCG decouples NDCG from candidate-list size and relevance distribution so scores are comparable across queries, and NDCG is 0 when nothing relevant is ranked — avoiding the inflation of raw DCG as k grows.

Hand computation: relevance [3, 2, 1, 0] ranked as [2, 1, 3, 0]; compute DCG@4, IDCG@4 and NDCG@4.

Why $2^{rel_i}-1$ instead of $rel_i$? What is the intuition behind the position discount $\log_2(i+1)$?

Why divide DCG by IDCG? What problems does the unnormalized DCG have?

How NDCG differs from binary metrics: graded 0-4 relevance vs relevant/irrelevant; how does the truncation k affect results?

Relation between NDCG and MRR: what does NDCG@1 correspond to under binary relevance?