Naive Bayes

Naive Bayes is a generative classifier built on Bayes' theorem; its core assumption is that features are conditionally independent given the class $C$. By Bayes' theorem

the prior $P(C)$ times the feature-likelihood product yields the posterior, and the class with maximum posterior is predicted — the normalizer $P(x)$ is the same for all classes and cancels under argmax. The parameterization defines three variants: Multinomial NB (count features, multinomial distribution, for word counts in text), Gaussian NB (continuous features fit as $P(x_i \mid C) = \mathcal{N}(x_i; \mu_{C,i}, \sigma_{C,i}^2)$), and Bernoulli NB (binary 0/1 features, Bernoulli distribution). If a feature-class pair never appears in training, the posterior product collapses to zero, hence Laplace smoothing

with $N_y$ the number of class-$y$ samples, $N_{y,x_i}$ the count of feature $x_i$ in class $y$, $|V|$ the vocabulary size, and $\alpha = 1$ giving add-one smoothing.

text classification (spam, sentiment), fast baselines on high-dimensional sparse features, online incremental learning; interviews often ask to derive the conditional-independence posterior and compare NB with logistic regression.

a generative view over high-dimensional discrete features with parameters growing linearly, no iterative training, and streaming updates; the cost is the strong conditional-independence assumption — correlated features bias probability estimates, though classification boundaries usually remain usable and NB stays robust to noisy features.

Test Your Knowledge: Practice Questions for "Naive Bayes"

Single choice pitfall questions with instant feedback and mistake tracking.