Back to Reinforcement Learning Mind Map
中文·English
🎮 Reinforcement LearningID: slate-bandit

Slate / Combinatorial Bandit

Slate/组合老虎机
🎯Core Definition
A Slate Bandit (combinatorial bandit) extends actions from single items to a whole page-level list: each round pick KK items out of NN candidates to form a slate SS with S=K|S| = K, i.e. (NK)\binom{N}{K} combinations; rewards may be item-level (per-item clicks) or slate-level (e.g., at least one click, dwell time). The typical solution is Top-K combinatorial exploration: apply UCB/TS-style confidence scores at the item level — sort by μ^a+2lntna\hat{\mu}_a + \sqrt{\frac{2\ln t}{n_a}} and take the top KK, guaranteeing each candidate gets explored, then adjust for position/diversity.
💡Use Cases
home feed, search result pages, email recommendations, ad layout combinations; interviews contrast "one action vs a set of actions" with MAB, hitting combinatorial complexity (NK)\binom{N}{K} and greedy approximation.
Key Problems Solved
enumerating (NK)\binom{N}{K} super-arms explodes combinatorially and cannot be estimated individually; slate bandits exploit the combinatorial structure (per-item contribution + position effects) to reduce the estimation dimension from combinations to items, keeping O(lnT)O(\ln T)-scale regret over large candidate pools while explicitly modeling item dependencies (diversity penalties, mutual exclusions).
🎯5 High-Frequency Exam Points
1
Combinatorics: how many slates for N=10,K=3N=10, K=3? Why can't we estimate every super-arm individually?
2
Top-K combinatorial exploration: how do item-level UCB/TS scores with top-K selection guarantee exploration?
3
Difference between item-level and slate-level reward modeling, and when to use each?
4
How does position bias distort estimates, and how do you correct for it?
5
Modeling dependencies within a slate: diversity, mutual exclusion, and click co-occurrence constraints?
Updated 2026-08-12
🎯
Test Your Knowledge: Practice Questions for "Slate / Combinatorial Bandit"
Single choice pitfall questions with instant feedback and mistake tracking.
🚀 Start Card Practice
Previous CardContextual BanditNext CardLong-Term Retention RL

🔗 More Reinforcement Learning Knowledge Cards

Actor-CriticBehavioral CloningCoT & Reasoning RLConservative Q-Learning