M1-002M1: Mathematics & Statistics FundamentalsProbability FoundationsEasy
⚡Flashcard Drill➔Mastery:
Formulate Bayes' Theorem and Detail the Respective Roles of Prior, Likelihood, and Posterior
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Posterior is proportional to Likelihood times Prior; the prior encodes existing beliefs, the likelihood is provided by empirical data, and the posterior reflects updated knowledge.
📌 Key Takeaways
- •The denominator serves as the normalizing constant (marginal likelihood / evidence).
- •Conjugate priors ensure the posterior resides in the same parametric family as the prior, enabling closed-form algebraic updates.
- •MAP selects the posterior mode, whereas MLE selects the likelihood mode (equivalent to MAP under a uniform prior).
📐 Mathematical Derivations
Bayes' Theorem states . Here, denotes the prior distribution representing epistemic uncertainty before observing data. The likelihood function quantifies how plausibly the observed sample was generated under parameter configuration . The evidence guarantees that the posterior density integrates to 1. In logarithmic form, , illustrating that Bayesian updating is an additive log-evidence accumulation process.
🏭 Production Trade-offs
In industrial production: (1) Under small-sample regimes (e.g., cold-start CTR prediction in ad auctions), an empirical Bayes prior prevents severe overfitting by pulling noisy estimates toward category averages (shrinkage). (2) As sample size , the likelihood dominates by the Bernstein-von Mises theorem, causing Bayesian and frequentist MLE estimators to asymptotically converge. Conjugacy offers streaming updates, while non-conjugate complex models require compute-heavy approximations like MCMC or SVI.
⚠️ Common Interview Pitfalls
- ✕Overconfidence caused by excessively rigid priors (e.g., Dirac delta or zero variance) preventing data from shifting belief.
- ✕Ignoring the computational bottleneck of the evidence denominator in continuous high-dimensional parameter spaces.
🎯 Interviewer Follow-up Inquiries
- ?Under what exact conditions do MAP and MLE produce mathematically identical point estimates?
- ?How do conjugate priors differ in multi-parameter exponential families versus 1D distributions?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚
Associated Knowledge Base Guides & Mindmaps
Explore the comprehensive technical article, exam cards, and global architecture tree.
中文速览与面试要点
写出贝叶斯定理,并说明先验、似然、后验各自的角色。
后验 ∝ 似然 × 先验;先验编码已有信念,似然由数据提供,后验是更新后的信念。
核心要点
- 贝叶斯定理把'学习'形式化为信念更新:posterior ∝ likelihood × prior。三个量的角色清晰可分—— 先验 p(θ) 编码在看到数据前对参数的信念(正则化/领域知识的载体); 似然 p(D|θ) 由数据与生成模型假设决定,是唯一与数据相关的项; 后验 p(θ|D) 是二者的乘积再归一化。分母 p(D)=∫p(D|θ)p(θ)dθ 与 θ 无关,因此常被省略为比例式。关键性质:数据量增加时,似然项的对数随 N 线性增长而先验固定,故先验影响被逐渐稀释,后验趋向 MLE——这是'数据够多时正则化不重要'的形式化表述。共轭先验(Beta-Bernoulli、Dirichlet-Multinomial、Normal-Normal)使后验与先验同族,可解析更新,是 A/B 测试中 Thompson Sampling 的基础。
- 两个易被追问的区分:① MAP 不是'贝叶斯最优' ——MAP 只取后验众数一个点,丢弃了后验的不确定性;贝叶斯决策理论要求对后验做积分得到后验预测分布 p(y|x,D)=∫p(y|x,θ)p(θ|D)dθ,这在高维下不可解,故工程上退化为 MAP(等价于加正则的 MLE)。② 先验的选择是主观的但可检验 ——弱信息先验(如 Jeffreys 先验)尽量减少主观影响;若后验对先验敏感,说明数据信息不足,应报告先验敏感性分析。
- 分母 P(D) 是归一化常数(证据)
- 共轭先验使后验与先验同族,可解析更新
- MAP = 取后验众数;MLE = 取似然众数(等价于均匀先验)
常见陷阱
- 把后验分布与后验预测分布混淆(前者关于参数,后者关于新数据)
- 认为 MAP 就是贝叶斯方法——它丢失了不确定性量化