M3-017Medium结果解读 · Interpretation
贝叶斯 A/B 测试与频率派有什么区别?什么时候用哪个?
How does a Bayesian A/B test differ from a frequentist one, and when would you use each?
💡 考场高分一句话答案:
贝叶斯给“效应大于 0 的概率”,更贴合决策语言;频率派控制长期错误率,更贴合合规与科学标准。
Bayesian methods give the probability the effect is positive, which matches decision language; frequentist methods control long-run error rates, which matches scientific and regulatory standards.
📌 核心考察要点 (Key Points)
- 贝叶斯:需要先验;输出 $P(\tau>0\mid \text{data})$ 与后验区间;天然支持序贯查看(但需注意先验敏感性)
- 频率派:输出 p 值与置信区间;控制 Type I error;不依赖先验
- 实践选择:产品决策常用贝叶斯(易解释、可早期停止);监管/科学场景用频率派
- 共同陷阱:两者都不能替代“效应量是否值得”的判断
- Bayesian: requires a prior; outputs $P(\tau>0\mid \text{data})$ and credible intervals; supports sequential viewing but is prior-sensitive
- Frequentist: outputs p-values and confidence intervals; controls Type I error; no prior required
- Practical split: product decisions often use Bayesian for interpretability and early stopping; regulated or scientific settings use frequentist
- Shared trap: neither replaces the judgement of whether the effect size is worth having
📐 第一性原理与数理推导机理
贝叶斯后验 $p(\tau\mid \text{data})\propto p(\text{data}\mid\tau)p(\tau)$。若用无信息先验,后验区间与置信区间在大样本下近似一致;差异主要体现在小样本与多次查看时的行为。
⚖️ 业务权衡与取舍 (Trade-offs & Calibration)
面试里最实用的说法是:“我用频率派做严谨性检验(p 值、CI),用贝叶斯语言向业务方沟通(‘有 92% 的概率这个改动是正向的’)。两者在大样本下结论一致,冲突时说明效应接近 0。”这句话既显示理解,又显示沟通意识。
⚠️ 常见高频面试陷阱与避坑指南
- 用贝叶斯后验概率替代“效应量大小”的判断
- 认为贝叶斯可以随便 peeking(先验与停止规则仍会影响结论)
🔍 大厂面试官连环追问 (Follow-ups)
- 如何选先验?先验敏感怎么办?
- 贝叶斯方法如何做多重比较?
嫌字多看太累?进入苏格拉底式对话推演实验室
零打字负担,以大厂考官真实困境切入,通过点选直觉选项暴露思维盲区,手把手推导 Why。
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.