M3-007Medium方差缩减 · Variance reduction
什么是 triggered analysis?为什么它能提高实验灵敏度?
What is triggered analysis and why does it increase sensitivity?
💡 考场高分一句话答案:
只分析“真正会被改动影响”的用户子集;它把被稀释的效应恢复到真实量级,从而在不增加样本的情况下提高功效。
It analyses only the users the change could actually affect. That restores the diluted effect to its true magnitude and raises power without more sample.
📌 核心考察要点 (Key Points)
- 前提:对照组也要记录“本会触发”的标志(counterfactual logging),否则无法构造可比子集
- 效果:若触发比例 $q$,全量口径效应为 $q\tau$,triggered 口径为 $\tau$;功效提升约 $1/q^2$
- 局限:结论只对触发人群成立,不能直接外推到全量
- 与稀释的区别:稀释是分母问题,triggered 是修正分母;不是“筛选好结果”
- Precondition: the control arm must also log the 'would have triggered' flag (counterfactual logging), otherwise no comparable subset exists
- Effect: if the trigger share is $q$, the all-users effect is $q\tau$ and the triggered effect is $\tau$; power improves roughly as $1/q^2$
- Limitation: the conclusion holds only for the triggered population, not for everyone
- Relation to dilution: dilution is a denominator problem; triggered analysis fixes the denominator rather than cherry-picking results
📐 第一性原理与数理推导机理
设全量口径下效应估计为 $\hat\tau_{\text{all}}=q\hat\tau$,方差近似不变(因为未触发用户贡献 0 效应但贡献方差)。信噪比因此下降为原来的 $q$。triggered 口径剔除未触发用户,$q\to 1$,信噪比恢复。
⚖️ 业务权衡与取舍 (Trade-offs & Calibration)
triggered analysis 的正确用法是双层报告:triggered 子集给效应大小(用于判断机制是否成立),全量给业务影响(用于决策)。只报 triggered 会高估整体收益,只报全量会误杀有效改动。
⚠️ 常见高频面试陷阱与避坑指南
- 事后按“是否点击了新功能”划分 triggered——那是结果变量,产生选择偏差
- 只报 triggered 结果,向业务方暗示这是全局收益
🔍 大厂面试官连环追问 (Follow-ups)
- trigger 比例很低(如 2%)时,实验要跑多久?
- 如何设计 trigger 条件才算合理?
嫌字多看太累?进入苏格拉底式对话推演实验室
零打字负担,以大厂考官真实困境切入,通过点选直觉选项暴露思维盲区,手把手推导 Why。
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.