M2-045M2: Classical Machine LearningNaive BayesHard
⚡Flashcard Drill➔Mastery:
Contrast Naive Bayes and Logistic Regression as a Generative-Discriminative Model Pair (Ng & Jordan, 2001)
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Naive Bayes (generative) models joint distribution and converges faster ( data); Logistic Regression (discriminative) directly models conditional and reaches a strictly lower asymptotic error ceiling ( data).
📌 Key Takeaways
- •Generative vs Discriminative: Naive Bayes models ; Logistic Regression directly models .
- •Identical Functional Form: Under binary features, Naive Bayes posterior has the EXACT mathematical functional form of Logistic Regression: .
- •Ng & Jordan (2001) Findings: Naive Bayes reaches its asymptotic error rate with only samples, beating Logistic Regression on tiny datasets; Logistic Regression converges with samples to a lower asymptotic error floor.
📐 Mathematical Derivations
Proof of identical parametric form: Consider binary features . In Naive Bayes: , where log-odds . This is identical to the linear predictor in Logistic Regression! The only difference is parameter estimation: Naive Bayes estimates by maximizing joint likelihood (generative), while Logistic Regression optimizes conditional likelihood directly (discriminative).
🏭 Production Trade-offs
When each dominates: (1) Naive Bayes wins: Extreme small-sample regimes (), streaming online updates with microsecond latency requirements, missing feature resilience. (2) Logistic Regression wins: Correlated features (e.g. In ad auctions with overlapping user tags), large training datasets, and scenarios requiring calibrated probability outputs.
⚠️ Common Interview Pitfalls
- ✕Believing Naive Bayes and Logistic Regression learn different decision boundary geometries (both produce linear hyperplanes).
- ✕Using Naive Bayes when features have strong multicollinearity without feature selection.
🎯 Interviewer Follow-up Inquiries
- ?Why does Logistic Regression achieve lower asymptotic error than Naive Bayes when the conditional independence assumption is false?
- ?How does generative-discriminative pair duality generalize to HMMs vs Linear-Chain CRFs?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚