M2-043M2: Classical Machine LearningNaive BayesMedium
⚡Flashcard Drill➔Mastery:
Compare Gaussian, Multinomial, and Bernoulli Naive Bayes Models and Their Respective Likelihoods
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Gaussian Naive Bayes models continuous features via normal distributions; Multinomial Naive Bayes models word count frequencies; Bernoulli Naive Bayes models binary feature presence/absence while explicitly penalizing non-occurrences.
📌 Key Takeaways
- •Gaussian NB: ; for continuous features.
- •Multinomial NB: ; models discrete counts/frequencies (e.g. Word count vectors, TF-IDF).
- •Bernoulli NB: ; models binary flags (presence vs absence); explicitly incorporates the absence of words.
📐 Mathematical Derivations
Contrast between Multinomial and Bernoulli likelihoods: In text classification, let document have binary indicator and word counts . In Multinomial NB, log-likelihood is . Words that do not appear in the document () contribute , exerting zero influence on classification. In Bernoulli NB, log-likelihood is . Crucially, words that do NOT appear in the document explicitly contribute , penalizing the class if that word was expected to appear!
🏭 Production Trade-offs
When to use which: (1) Bernoulli NB: Short texts (e.g. Tweets, SMS spam, search queries) where word presence/absence matters more than frequency. (2) Multinomial NB: Long documents where word frequency carries rich topic signal. (3) Gaussian NB: Physical sensor measurements, biomedical continuous features (requires checking whether features are unimodal).
⚠️ Common Interview Pitfalls
- ✕Using Multinomial NB on negative feature values (Multinomial requires non-negative counts or scaled TF-IDF).
- ✕Using Bernoulli NB on long documents (Bernoulli over-penalizes long documents due to non-occurrence of rare words).
🎯 Interviewer Follow-up Inquiries
- ?Why does Bernoulli Naive Bayes inherently penalize document length differences?
- ?How does Complement Naive Bayes (CNB) address severe class imbalance in text categorization?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚