M7-063M7: Retrieval, Ranking & RecSysDeep Recommendation ModelsHard
⚡Flashcard Drill➔Mastery:

Explain How the Entire Space Multi-Task Model (ESMM) Eliminates Sample Selection Bias (SSB) and Data Sparsity in CVR

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M7-064 →
📐 Mathematical Definition
p(CTCVR)⏟entire space=p(CTR)⏟entire space×p(CVR)⏟entire space;train on all samples\underbrace{p(\text{CTCVR})}_{\text{entire space}}=\underbrace{p(\text{CTR})}_{\text{entire space}}\times\underbrace{p(\text{CVR})}_{\text{entire space}};\qquad \text{train on all samples}
⚡ One-Sentence Core Answer
ESMM models post-click conversion rate (CVR) across the entire impression space by mathematically decomposing Click-Through and Conversion Rate (CTCVR) as pCTCVR = pCTR * pCVR, training on unselected impression logs to eliminate sample selection bias.

📌 Key Takeaways

  • •
    Sample Selection Bias (SSB): Traditional CVR models are trained only on clicked samples, but are evaluated across all impressed candidates at serving time.
  • •
    Data Sparsity (DS): Conversion events are 100x rarer than clicks; training CVR in isolation leads to severe parameter underfitting.
  • •
    Multiplicative probability decomposition: p(Click, Conversion | x) = p(Click | x) * p(Conversion | Click, x); enables joint training over all impression logs.
  • •
    Shared embedding table: The CVR network and CTR network share input embedding layers, transferring rich click signals into the sparse conversion domain.

📐 Mathematical Derivations

Mathematical & Probabilistic Formulation: ESMM Architecture (Ma et al., Alibaba, 2018).

(1) The Two Fundamental Pathologies in CVR Modeling:
- Sample Selection Bias (SSB):
Conversion rate is defined mathematically as the conditional probability of conversion given click:
pCVR=P(y=1∣z=1,x)p\text{CVR} = P(y=1 \mid z=1, x)
where z∈{0,1}z \in \{0, 1\} is click, y∈{0,1}y \in \{0, 1\} is conversion, and xx is candidate features.
Traditional models train exclusively on clicked samples Dclicked={(x,y)∣z=1}\mathcal{D}_{\text{clicked}} = \{(x, y) \mid z=1\}. However, during online serving, the ranking model scores all impressed candidates Dimpressed={x}\mathcal{D}_{\text{impressed}} = \{x\}. Because P(x∣z=1)≠P(x)P(x \mid z=1) \neq P(x), the training distribution diverges drastically from the serving distribution.
- Data Sparsity (DS): Conversion samples account for only 0.01%–0.1%0.01\%\text{--}0.1\% of total traffic, causing standalone CVR embeddings to overfit.

(2) The Multiplicative Probability Decomposition:
Define Click-Through & Conversion Rate (pCTCVRp\text{CTCVR}):
pCTCVR=P(z=1,y=1∣x)=P(z=1∣x)×P(y=1∣z=1,x)=pCTR×pCVRp\text{CTCVR} = P(z=1, y=1 \mid x) = P(z=1 \mid x) \times P(y=1 \mid z=1, x) = p\text{CTR} \times p\text{CVR}
Notice that both pCTRp\text{CTR} and pCTCVRp\text{CTCVR} can be observed and trained directly over the entire impression space $\mathcal{D}_{\text{impressed}}$:
- z=1z=1 indicates clicked impression; z=0z=0 indicates unclicked impression.
- y=1y=1 indicates converted impression; y=0y=0 indicates non-converted impression.

(3) Loss Function Formulation:
ESMM constructs two neural networks: a CTR tower predicting fCTR(x)f_{\text{CTR}}(x) and a CVR tower predicting fCVR(x)f_{\text{CVR}}(x). Both share the same feature embedding table.
L(x,z,y)=LBCE(z,fCTR(x))+LBCE(z⋅y, fCTR(x)×fCVR(x))\mathcal{L}(x, z, y) = \mathcal{L}_{\text{BCE}}(z, f_{\text{CTR}}(x)) + \mathcal{L}_{\text{BCE}}\big( z \cdot y, \, f_{\text{CTR}}(x) \times f_{\text{CVR}}(x) \big)
L=−zln⁡fCTR(x)−(1−z)ln⁡(1−fCTR(x))−(z⋅y)ln⁡(fCTRfCVR)−(1−z⋅y)ln⁡(1−fCTRfCVR)\mathcal{L} = - z \ln f_{\text{CTR}}(x) - (1 - z) \ln(1 - f_{\text{CTR}}(x)) - (z \cdot y) \ln(f_{\text{CTR}} f_{\text{CVR}}) - (1 - z \cdot y) \ln(1 - f_{\text{CTR}} f_{\text{CVR}})
At test time, the model directly outputs fCVR(x)f_{\text{CVR}}(x) as the predicted conversion probability.

🏭 Production Trade-offs

In-Depth Analysis & Engineering Trade-offs: ① Why not train CVR by predicting CTCVR directly and dividing by CTR?—If pCVR=pCTCVRpCTRp\text{CVR} = \frac{p\text{CTCVR}}{p\text{CTR}}, whenever an item has a very low predicted click probability (pCTR→0p\text{CTR} \to 0), numerical division leads to extreme volatility (pCVR>1p\text{CVR} > 1 or infinity); ESMM models pCVRp\text{CVR} as a bounded neural output in [0,1][0, 1] and multiplies forward, maintaining complete numerical stability. ② Shared embedding representation transfer—because the CTR tower receives gradients from hundreds of millions of click events, the shared embedding representations are exceptionally rich; this directly benefits the CVR tower, solving the Data Sparsity dilemma. ③ Handling delayed conversions (Attribution Window)—conversions often occur hours or days after the initial click; closing attribution windows too early creates false negative labels in pCTCVRp\text{CTCVR}; production pipelines implement delayed feedback modeling or importance weighting on conversion logs. ④ Multi-stage conversion funnels (ESM2)—e-commerce funnels span multiple actions: Impression →\to Click →\to Add-to-Cart →\to Purchase; ESM2 extends ESMM to a directed acyclic graph (DAG) of conditional probabilities, modeling intermediate conversion steps simultaneously. ⑤ Inference speed—online ad auctions require evaluating eCPM=Bid×pCTR×pCVReCPM = \text{Bid} \times p\text{CTR} \times p\text{CVR}; both towers evaluate concurrently on GPU in <2 ms< 2\text{ ms}. ⑥ Interview takeaway—formulate the two pathologies (SSB and DS), write down the probability identity P(z=1,y=1∣x)=P(z=1∣x)P(y=1∣z=1,x)P(z=1, y=1 \mid x) = P(z=1 \mid x) P(y=1 \mid z=1, x), detail the joint loss function over the entire space, and explain the numerical division failure mode.
⚠️ Common Interview Pitfalls
  • ✕
    Attempting to compute CVR at inference time by dividing pCTCVR by pCTR, resulting in catastrophic numerical instability and division-by-zero errors when pCTR is small.
  • ✕
    Training standalone CVR models purely on clicked samples without recognizing that online serving encounters unclicked impression distributions (SSB).
  • ✕
    Failing to share embedding parameters between the CTR and CVR networks, forfeiting the primary solution to the Data Sparsity problem.
🎯 Interviewer Follow-up Inquiries
  • ?
    Why is multiplying pCTR and pCVR during training numerically superior to estimating pCTCVR and dividing by pCTR?
  • ?
    How does the ESM2 model extend ESMM to multi-step sequential conversion funnels (Impression -> Click -> Cart -> Buy)?
  • ?
    What delayed feedback modeling strategies handle post-click conversion attribution windows that span several days?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM7-062: Explain Multi-gate Mixture-of-Experts (MMoE) and Multi-Task Learning Architectures in Recommendation Systems📋 Back to BankNext →M7-064: Explain Key Principles of Feature Engineering and Embedding Design in Deep Recommendation