← Back to Reinforcement Learning Mind Map
🎮 Reinforcement LearningID: slate-bandit

Slate / Combinatorial Bandit

Slate/组合老虎机
🎯Core Definition
A Slate Bandit (combinatorial bandit) extends actions from single items to a whole page-level list: each round pick KK items out of NN candidates to form a slate SS with ∣S∣=K|S| = K, i.e. (NK)\binom{N}{K} combinations; rewards may be item-level (per-item clicks) or slate-level (e.g., at least one click, dwell time). The typical solution is Top-K combinatorial exploration: apply UCB/TS-style confidence scores at the item level — sort by μ^a+2ln⁡tna\hat{\mu}_a + \sqrt{\frac{2\ln t}{n_a}} and take the top KK, guaranteeing each candidate gets explored, then adjust for position/diversity.
💡Use Cases
home feed, search result pages, email recommendations, ad layout combinations; interviews contrast "one action vs a set of actions" with MAB, hitting combinatorial complexity (NK)\binom{N}{K} and greedy approximation.
⚡Key Problems Solved
enumerating (NK)\binom{N}{K} super-arms explodes combinatorially and cannot be estimated individually; slate bandits exploit the combinatorial structure (per-item contribution + position effects) to reduce the estimation dimension from combinations to items, keeping O(ln⁡T)O(\ln T)-scale regret over large candidate pools while explicitly modeling item dependencies (diversity penalties, mutual exclusions).
🎯Core Interview Exam Points
Fundamentals · Production Scenarios · Deep Dive
Core Def
Explain the core definition, underlying mechanism/math, and key principles of Slate / Combinatorial Bandit.
Use Cases
In production engineering, what are the primary use cases for Slate / Combinatorial Bandit and what core pain points does it solve?
🔥 High-Frequency Deep-Dive Questions
1
Combinatorics: how many slates for N=10,K=3N=10, K=3? Why can't we estimate every super-arm individually?
2
Top-K combinatorial exploration: how do item-level UCB/TS scores with top-K selection guarantee exploration?
3
Difference between item-level and slate-level reward modeling, and when to use each?
4
How does position bias distort estimates, and how do you correct for it?
5
Modeling dependencies within a slate: diversity, mutual exclusion, and click co-occurrence constraints?
Updated 2026-08-12
🎯
Verify Mastery: Practice Exam Questions for "Slate / Combinatorial Bandit"
Take pitfall multiple-choice questions with derivation options; mistakes are automatically logged.
🚀 Start Card Practice ➔
← Previous CardContextual BanditNext Card →Long-Term Retention RL

🔗 More Reinforcement Learning Knowledge Cards

引用出处: TalentMe 450+ 工业级 AI 架构考点知识库 (Tech Vault) · 遵循 CC-BY-4.0 知识共享协议
URL: https://talentme.airsota.com/resources/tech/roadmap/mindmap/RL/card/slate-bandit
中文速览与面试要点

Slate/组合老虎机 (Slate / Combinatorial Bandit)

定义: Slate Bandit (组合老虎机) 把动作从单个 item 扩展到整页推荐列表:每轮从 N 个候选中选出 K 个组成 slate S,要求 |S| = K,共 NK 种组合;奖励既可以是 item 级(逐条点击),也可以是 slate 级(如"至少一次点击"、页面停留时长)。典型解法是 Top-K 组合探索:把 UCB/TS 的置信公式作用到 item 级别——按 _a + 2 tn_a 排序取前 K,保证每个候选都有被探索的机会,再结合位置/多样性修正。 使用场景: 首页信息流、搜索结果页、邮件推荐、广告组合样式;面试从"一个动作 vs 一组动作"的维度对比 MAB,引出组合复杂度 NK 与贪心近似。 解决的问题: 直接枚举 NK 个超臂 (super-arm) 会组合爆炸、无法逐一估计;slate bandit 利用组合结构(每个 item 的独立贡献 + 位置效应)把估计维度从组合级降到 item 级,在大候选池下保持 O( T) 量级的 regret,同时能显式建模 item 间依赖(多样性惩罚、互斥内容)。

核心要点

  • 组合复杂度: N=10, K=3 时有多少种 slate?为什么不能逐一估计每个超臂?
  • Top-K 组合探索的公式: 如何在 item 级做 UCB/TS 并取前 K, 探索如何保证?
  • item 级奖励与 slate 级奖励建模的差别?各自适用场景?
  • 位置偏差 (position bias) 如何影响估计?如何修正?
  • slate 内 item 间的依赖如何建模: 多样性/互斥/协同点击的约束?