DDPM Forward/Reverse Process

DDPM (Denoising Diffusion Probabilistic Models) builds generation on two Markov chains: a forward process $q$ that gradually adds noise to an image until it becomes standard Gaussian over $T$ steps, and a reverse process $p_\theta$ that learns to denoise step by step. The forward step is $q(x_t \mid x_{t-1}) = \mathcal{N}(x_t;\ \sqrt{1-\beta_t}x_{t-1},\ \beta_t I)$; since each step injects independent Gaussian noise, the accumulated noise has a closed form at any time $t$:

with $\alpha_t = 1 - \beta_t$ and the cumulative schedule $\bar\alpha_t = \prod_{s=1}^{t}\alpha_s$ (the SNR $\bar\alpha_t/(1-\bar\alpha_t)$ decreases with $t$); the standard linear schedule runs $\beta_1 = 10^{-4}$ up to $\beta_T = 0.02$. The training objective follows from maximizing the ELBO: each term is a Gaussian KL between the posterior $q(x_{t-1} \mid x_t, x_0)$ and $p_\theta(x_{t-1} \mid x_t)$, which with known variances reduces to the L2 distance between means; since the mean is a linear combination of $x_t$ and the noise $\epsilon$, the loss collapses to regressing the injected noise:

Sampling starts from $x_T \sim \mathcal{N}(0, I)$ and iterates backward $t = T, \dots, 1$: $x_{t-1} = \dfrac{1}{\sqrt{\alpha_t}}\left(x_t - \dfrac{\beta_t}{\sqrt{1-\bar\alpha_t}}\epsilon_\theta(x_t, t)\right) + \sigma_t z$, where $\sigma_t^2 = \tilde\beta_t$ (posterior variance) and $z = 0$ at the final step $t=1$.

an interview staple — “write the DDPM training/sampling formulas”; the backbone of text-to-image models (Stable Diffusion) and video/audio generation, often followed by “why regress the noise with L2?”.

GANs suffer unstable adversarial training, mode collapse and difficult tuning; DDPM's loss is plain MSE noise regression — stable first-order optimization and full-distribution coverage (good diversity) — at the cost of ~1000 sampling steps, which motivates DDIM/LCM acceleration.

Test Your Knowledge: Practice Questions for "DDPM Forward/Reverse Process"

Single choice pitfall questions with instant feedback and mistake tracking.