M2-042M2: Classical Machine LearningNaive BayesEasy
⚡Flashcard Drill➔Mastery:
Explain Laplace (Additive) Smoothing and Why It Prevents Zero-Probability Catastrophes in Naive Bayes
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Laplace smoothing adds pseudo-counts to frequency estimators: , preventing unobserved feature-class pairs from zeroing out the entire posterior probability.
📌 Key Takeaways
- •Zero-Frequency Problem: If a word never appeared in class in training data, empirical frequency ; multiplying probabilities causes , completely blinding the model to all other strong signals.
- •Laplace Smoothing Formula: , where is vocabulary size.
- •Bayesian Interpretation: Equivalent to MAP estimation under a symmetric Dirichlet conjugate prior .
📐 Mathematical Derivations
Derivation via Bayesian Conjugacy: Let word counts in class follow a Multinomial distribution with parameters where . The conjugate prior over the simplex is the symmetric Dirichlet distribution: . When observing empirical word counts with total tokens , the posterior is: . This is . The Bayesian expected posterior probability (posterior mean) is: , directly deriving the Laplace smoothing formula.
🏭 Production Trade-offs
Tuning smoothing parameter : (1) is standard Laplace smoothing. (2) (e.g. , Lidstone smoothing) is preferred when vocabulary size is massive (), preventing the denominator from overwhelming small-count signals.
⚠️ Common Interview Pitfalls
- ✕Adding to the numerator without adding to the denominator (probabilities must sum to 1).
- ✕Treating as an untunable constant (Lidstone should be cross-validated).
🎯 Interviewer Follow-up Inquiries
- ?How does Lidstone smoothing generalize Laplace smoothing for fractional ?
- ?Why does Good-Turing frequency estimation provide superior smoothing for very rare events?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚