Common Distributions

Common distribution families with parameter-mean-variance table: ① Normal $\mathcal{N}(\mu,\sigma^2)$: pdf $f(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$, mean $\mu$, variance $\sigma^2$;② Bernoulli $\text{Bern}(p)$: $P(X=1) = p$, mean $p$, variance $p(1-p)$;③ Binomial $\text{Bin}(n,p)$: $P(X=k) = \binom{n}{k}p^k(1-p)^{n-k}$, mean $np$, variance $np(1-p)$;④ Poisson $\text{Poisson}(\lambda)$: $P(X=k) = \frac{\lambda^k e^{-\lambda}}{k!}$, mean $\lambda$, variance $\lambda$;⑤ Exponential $\text{Exp}(\lambda)$: pdf $f(x) = \lambda e^{-\lambda x}$, mean $1/\lambda$, variance $1/\lambda^2$.

pick the likelihood and output layer from the data generating process — Bernoulli (cross-entropy) for binary classification, Normal (MSE) for regression, Poisson for counts, Exponential for waiting times; interviews often open with "choose a distribution for this data and write the likelihood".

a mismatched distribution assumption systematically corrupts likelihoods and variance estimates — Poisson requires mean = variance, so over-dispersed count data needs the negative binomial; the memoryless Exponential is the right model for waiting times without decay memory. Three relations form the skeleton: Binomial is a sum of $n$ independent Bernoullis, Poisson is its $n \to \infty$, $p \to 0$, $np = \lambda$ limit, and Normal is the large-$n$ limit of both (CLT).

Test Your Knowledge: Practice Questions for "Common Distributions"

Single choice pitfall questions with instant feedback and mistake tracking.