Long-Term Retention RL

Q: Why do delayed rewards make credit assignment hard, and what role does the discount $\gamma$ play?

【Long-Term Retention RL 核心考点 1】Why do delayed rewards make credit assignment hard, and what role does the discount $\gamma$ play?。技术解析：Definition: Long-term retention RL models D1/D7 retention and LTV as delayed rewards: the true return of today's action (e.g., a recommended item) is observed days or weeks later, so a discount factor \gamma is introduced to model cross-period value, with long-term objective J = ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/RL/card/long-term-retention

Q: Formula: $J = \mathbb{E}[\sum_t \gamma^t r_t]$ — what business objectives do $\gamma \to 0$ and $\gamma \to 1$ correspond to?

【Long-Term Retention RL 核心考点 2】Formula: $J = \mathbb{E}[\sum_t \gamma^t r_t]$ — what business objectives do $\gamma \to 0$ and $\gamma \to 1$ correspond to?。技术解析：Definition: Long-term retention RL models D1/D7 retention and LTV as delayed rewards: the true return of today's action (e.g., a recommended item) is observed days or weeks later, so a discount factor \gamma is introduced to model cross-period value, with long-term objective J = ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/RL/card/long-term-retention

Q: Why does optimizing CTR alone hurt retention, and how do you design rewards for both short- and long-term?

【Long-Term Retention RL 核心考点 3】Why does optimizing CTR alone hurt retention, and how do you design rewards for both short- and long-term?。技术解析：Definition: Long-term retention RL models D1/D7 retention and LTV as delayed rewards: the true return of today's action (e.g., a recommended item) is observed days or weeks later, so a discount factor \gamma is introduced to model cross-period value, with long-term objective J = ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/RL/card/long-term-retention

Q: Attributing delayed feedback: TD bootstrapping vs MC, and how surrogate rewards speed up learning?

【Long-Term Retention RL 核心考点 4】Attributing delayed feedback: TD bootstrapping vs MC, and how surrogate rewards speed up learning?。技术解析：Definition: Long-term retention RL models D1/D7 retention and LTV as delayed rewards: the true return of today's action (e.g., a recommended item) is observed days or weeks later, so a discount factor \gamma is introduced to model cross-period value, with long-term objective J = ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/RL/card/long-term-retention

Q: How do you plug retention as delayed feedback into bandits (delayed-reward bandits) or RL?

【Long-Term Retention RL 核心考点 5】How do you plug retention as delayed feedback into bandits (delayed-reward bandits) or RL?。技术解析：Definition: Long-term retention RL models D1/D7 retention and LTV as delayed rewards: the true return of today's action (e.g., a recommended item) is observed days or weeks later, so a discount factor \gamma is introduced to model cross-period value, with long-term objective J = ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/RL/card/long-term-retention

长效留存延迟奖励

🎯Core Definition

Long-term retention RL models D1/D7 retention and LTV as delayed rewards: the true return of today's action (e.g., a recommended item) is observed days or weeks later, so a discount factor

\gamma

is introduced to model cross-period value, with long-term objective

J = \mathbb{E}[\sum_{t} \gamma^t r_t]

— small

\gamma

favors the near term, large

\gamma

the far term, trading off against short-term metrics (e.g., instant CTR). Delayed rewards create a credit-assignment problem: returns are hard to attribute to specific rounds/actions; common fixes include temporal decomposition, surrogate rewards, and sequential-decision frameworks.

💡Use Cases

retention optimization for short-video/social/gaming apps, LTV modeling for subscriptions and ads, fixing the "CTR up but retention down" paradox in recsys; interviews ask why optimizing CTR alone hurts the long-term experience and how discounting/delayed rewards fix it.

⚡Key Problems Solved

purely short-term objectives (CTR) invite clickbait that destroys long-term retention; treating retention as a delayed reward discounted by

\gamma

makes the policy trade some instant metrics for higher long-term value, while TD (bootstrapped) or Monte-Carlo attribution reduces the variance of delayed feedback — aligning short- and long-term metrics.

🎯5 High-Frequency Exam Points

Why do delayed rewards make credit assignment hard, and what role does the discount

\gamma

play?

Formula:

J = \mathbb{E}[\sum_t \gamma^t r_t]

— what business objectives do

\gamma \to 0

and

\gamma \to 1

correspond to?

Why does optimizing CTR alone hurt retention, and how do you design rewards for both short- and long-term?

Attributing delayed feedback: TD bootstrapping vs MC, and how surrogate rewards speed up learning?

How do you plug retention as delayed feedback into bandits (delayed-reward bandits) or RL?

📖 In-depth Guide:📄 bandits-and-online-decision →

Updated 2026-08-12

🎯

Test Your Knowledge: Practice Questions for "Long-Term Retention RL"

Single choice pitfall questions with instant feedback and mistake tracking.

🚀 Start Card Practice ➔

← Previous CardSlate / Combinatorial Bandit Next Card →RL with Verifiable Rewards

🔗 More Reinforcement Learning Knowledge Cards

Actor-Critic Behavioral Cloning Contextual Bandit CoT & Reasoning RL