M7-063M7: Retrieval, Ranking & RecSysDeep Recommendation ModelsHard
⚡Flashcard Drill➔Mastery:
Explain How the Entire Space Multi-Task Model (ESMM) Eliminates Sample Selection Bias (SSB) and Data Sparsity in CVR
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
ESMM models post-click conversion rate (CVR) across the entire impression space by mathematically decomposing Click-Through and Conversion Rate (CTCVR) as pCTCVR = pCTR * pCVR, training on unselected impression logs to eliminate sample selection bias.
📌 Key Takeaways
- •Sample Selection Bias (SSB): Traditional CVR models are trained only on clicked samples, but are evaluated across all impressed candidates at serving time.
- •Data Sparsity (DS): Conversion events are 100x rarer than clicks; training CVR in isolation leads to severe parameter underfitting.
- •Multiplicative probability decomposition: p(Click, Conversion | x) = p(Click | x) * p(Conversion | Click, x); enables joint training over all impression logs.
- •Shared embedding table: The CVR network and CTR network share input embedding layers, transferring rich click signals into the sparse conversion domain.
📐 Mathematical Derivations
Mathematical & Probabilistic Formulation: ESMM Architecture (Ma et al., Alibaba, 2018).
(1) The Two Fundamental Pathologies in CVR Modeling:
- Sample Selection Bias (SSB):
Conversion rate is defined mathematically as the conditional probability of conversion given click:
where is click, is conversion, and is candidate features.
Traditional models train exclusively on clicked samples . However, during online serving, the ranking model scores all impressed candidates . Because , the training distribution diverges drastically from the serving distribution.
- Data Sparsity (DS): Conversion samples account for only of total traffic, causing standalone CVR embeddings to overfit.
(2) The Multiplicative Probability Decomposition:
Define Click-Through & Conversion Rate ():
Notice that both and can be observed and trained directly over the entire impression space $\mathcal{D}_{\text{impressed}}$:
- indicates clicked impression; indicates unclicked impression.
- indicates converted impression; indicates non-converted impression.
(3) Loss Function Formulation:
ESMM constructs two neural networks: a CTR tower predicting and a CVR tower predicting . Both share the same feature embedding table.
At test time, the model directly outputs as the predicted conversion probability.
(1) The Two Fundamental Pathologies in CVR Modeling:
- Sample Selection Bias (SSB):
Conversion rate is defined mathematically as the conditional probability of conversion given click:
where is click, is conversion, and is candidate features.
Traditional models train exclusively on clicked samples . However, during online serving, the ranking model scores all impressed candidates . Because , the training distribution diverges drastically from the serving distribution.
- Data Sparsity (DS): Conversion samples account for only of total traffic, causing standalone CVR embeddings to overfit.
(2) The Multiplicative Probability Decomposition:
Define Click-Through & Conversion Rate ():
Notice that both and can be observed and trained directly over the entire impression space $\mathcal{D}_{\text{impressed}}$:
- indicates clicked impression; indicates unclicked impression.
- indicates converted impression; indicates non-converted impression.
(3) Loss Function Formulation:
ESMM constructs two neural networks: a CTR tower predicting and a CVR tower predicting . Both share the same feature embedding table.
At test time, the model directly outputs as the predicted conversion probability.
🏭 Production Trade-offs
In-Depth Analysis & Engineering Trade-offs: ① Why not train CVR by predicting CTCVR directly and dividing by CTR?—If , whenever an item has a very low predicted click probability (), numerical division leads to extreme volatility ( or infinity); ESMM models as a bounded neural output in and multiplies forward, maintaining complete numerical stability. ② Shared embedding representation transfer—because the CTR tower receives gradients from hundreds of millions of click events, the shared embedding representations are exceptionally rich; this directly benefits the CVR tower, solving the Data Sparsity dilemma. ③ Handling delayed conversions (Attribution Window)—conversions often occur hours or days after the initial click; closing attribution windows too early creates false negative labels in ; production pipelines implement delayed feedback modeling or importance weighting on conversion logs. ④ Multi-stage conversion funnels (ESM2)—e-commerce funnels span multiple actions: Impression Click Add-to-Cart Purchase; ESM2 extends ESMM to a directed acyclic graph (DAG) of conditional probabilities, modeling intermediate conversion steps simultaneously. ⑤ Inference speed—online ad auctions require evaluating ; both towers evaluate concurrently on GPU in . ⑥ Interview takeaway—formulate the two pathologies (SSB and DS), write down the probability identity , detail the joint loss function over the entire space, and explain the numerical division failure mode.
⚠️ Common Interview Pitfalls
- ✕Attempting to compute CVR at inference time by dividing pCTCVR by pCTR, resulting in catastrophic numerical instability and division-by-zero errors when pCTR is small.
- ✕Training standalone CVR models purely on clicked samples without recognizing that online serving encounters unclicked impression distributions (SSB).
- ✕Failing to share embedding parameters between the CTR and CVR networks, forfeiting the primary solution to the Data Sparsity problem.
🎯 Interviewer Follow-up Inquiries
- ?Why is multiplying pCTR and pCVR during training numerically superior to estimating pCTCVR and dividing by pCTR?
- ?How does the ESM2 model extend ESMM to multi-step sequential conversion funnels (Impression -> Click -> Cart -> Buy)?
- ?What delayed feedback modeling strategies handle post-click conversion attribution windows that span several days?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚