M2-043M2: Classical Machine LearningNaive BayesMedium
⚡Flashcard Drill➔Mastery:

Compare Gaussian, Multinomial, and Bernoulli Naive Bayes Models and Their Respective Likelihoods

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M2-044 →
📐 Mathematical Definition
Gaussian: P(xj∣y)=N(μjy,σjy2)\text{Gaussian}:\ P(x_j\mid y)=\mathcal N(\mu_{jy},\sigma_{jy}^2)
⚡ One-Sentence Core Answer
Gaussian Naive Bayes models continuous features via normal distributions; Multinomial Naive Bayes models word count frequencies; Bernoulli Naive Bayes models binary feature presence/absence while explicitly penalizing non-occurrences.

📌 Key Takeaways

  • •
    Gaussian NB: P(xj∣y=k)=12πσjk2exp⁡(−(xj−μjk)22σjk2)P(x_j\mid y=k) = \frac{1}{\sqrt{2\pi\sigma_{jk}^2}} \exp\left(-\frac{(x_j - \mu_{jk})^2}{2\sigma_{jk}^2}\right); for continuous features.
  • •
    Multinomial NB: P(x∣y=k)∝∏j=1dpjkxjP(x\mid y=k) \propto \prod_{j=1}^d p_{jk}^{x_j}; models discrete counts/frequencies (e.g. Word count vectors, TF-IDF).
  • •
    Bernoulli NB: P(x∣y=k)=∏j=1d[pjkxj+(1−pjk)(1−xj)]P(x\mid y=k) = \prod_{j=1}^d [p_{jk} x_j + (1 - p_{jk})(1 - x_j)]; models binary flags (presence vs absence); explicitly incorporates the absence of words.

📐 Mathematical Derivations

Contrast between Multinomial and Bernoulli likelihoods: In text classification, let document dd have binary indicator xj∈{0,1}x_j \in \{0, 1\} and word counts cjc_j. In Multinomial NB, log-likelihood is ∑j:cj>0cjlog⁡pjk\sum_{j : c_j > 0} c_j \log p_{jk}. Words that do not appear in the document (cj=0c_j = 0) contribute 0log⁡pjk=00 \log p_{jk} = 0, exerting zero influence on classification. In Bernoulli NB, log-likelihood is ∑j:xj=1log⁡pjk+∑j:xj=0log⁡(1−pjk)\sum_{j : x_j=1} \log p_{jk} + \sum_{j : x_j=0} \log(1 - p_{jk}). Crucially, words that do NOT appear in the document explicitly contribute log⁡(1−pjk)\log(1 - p_{jk}), penalizing the class if that word was expected to appear!

🏭 Production Trade-offs

When to use which: (1) Bernoulli NB: Short texts (e.g. Tweets, SMS spam, search queries) where word presence/absence matters more than frequency. (2) Multinomial NB: Long documents where word frequency carries rich topic signal. (3) Gaussian NB: Physical sensor measurements, biomedical continuous features (requires checking whether features are unimodal).
⚠️ Common Interview Pitfalls
  • ✕
    Using Multinomial NB on negative feature values (Multinomial requires non-negative counts or scaled TF-IDF).
  • ✕
    Using Bernoulli NB on long documents (Bernoulli over-penalizes long documents due to non-occurrence of rare words).
🎯 Interviewer Follow-up Inquiries
  • ?
    Why does Bernoulli Naive Bayes inherently penalize document length differences?
  • ?
    How does Complement Naive Bayes (CNB) address severe class imbalance in text categorization?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM2-042: Explain Laplace (Additive) Smoothing and Why It Prevents Zero-Probability Catastrophes in Naive Bayes📋 Back to BankNext →M2-044: Explain Why Naive Bayes Operates in the Log-Probability Domain to Prevent Underflow