Common Distributions

Common distribution families with parameter-mean-variance table: ① Normal $\mathcal{N}(\mu,\sigma^2)$: pdf $f(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$, mean $\mu$, variance $\sigma^2$;② Bernoulli $\text{Bern}(p)$: $P(X=1) = p$, mean $p$, variance $p(1-p)$;③ Binomial $\text{Bin}(n,p)$: $P(X=k) = \binom{n}{k}p^k(1-p)^{n-k}$, mean $np$, variance $np(1-p)$;④ Poisson $\text{Poisson}(\lambda)$: $P(X=k) = \frac{\lambda^k e^{-\lambda}}{k!}$, mean $\lambda$, variance $\lambda$;⑤ Exponential $\text{Exp}(\lambda)$: pdf $f(x) = \lambda e^{-\lambda x}$, mean $1/\lambda$, variance $1/\lambda^2$.

pick the likelihood and output layer from the data generating process — Bernoulli (cross-entropy) for binary classification, Normal (MSE) for regression, Poisson for counts, Exponential for waiting times; interviews often open with "choose a distribution for this data and write the likelihood".

a mismatched distribution assumption systematically corrupts likelihoods and variance estimates — Poisson requires mean = variance, so over-dispersed count data needs the negative binomial; the memoryless Exponential is the right model for waiting times without decay memory. Three relations form the skeleton: Binomial is a sum of $n$ independent Bernoullis, Poisson is its $n \to \infty$, $p \to 0$, $np = \lambda$ limit, and Normal is the large-$n$ limit of both (CLT).

Write the parameters, mean and variance of Normal, Bernoulli, Binomial, Poisson and Exponential?

Relations among Binomial, Poisson and Normal (limits and approximations)?

What is the memoryless property of the Exponential? How does it model waiting times?

Why Bernoulli (cross-entropy) for classification and Normal (MSE) for regression — from the likelihood perspective?

Poisson has mean = variance; what do you do with over-dispersed count data?