K2HardPart K · Generative Models & Diffusion

DDPM Reverse Denoising Sampling Step

⚡Flashcard Drill➔

DDPM reverse posterior denoising step estimating mu_theta from predicted noise eps_theta.

⏱️ Time: O(T * B * C * H * W) 推理需串行循环 T 次
💾 Space: O(B * C * H * W)
🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
def ddpm_p_sample_step(
x_t: np.ndarray, # 当前时间步带噪图 (B, C, H, W)
t_idx: int, # 当前时间步整数 t
predicted_noise: np.ndarray, # 神经网络预测的噪声 epsilon (B, C, H, W)
alphas: np.ndarray, # (T,)
alphas_cumprod: np.ndarray, # (T,)
betas: np.ndarray # (T,)
) -> np.ndarray:
alpha_t = alphas[t_idx]
beta_t = betas[t_idx]
alpha_bar_t = alphas_cumprod[t_idx]
# 1. 估计逆向均值 mu_t
coeff =
mean = (1.0 / np.sqrt(alpha_t)) * (x_t - coeff * predicted_noise)
# 2. 如果是最后一步 t=0,直接返回均值,不再注入扰动
if t_idx == 0:
return mean
# 3. 计算扰动方差 sigma_t (DDPM 原文常用 beta_t 或后验方差)
alpha_bar_prev = alphas_cumprod[t_idx - 1] if t_idx > 0 else 1.0
posterior_variance = beta_t * (1.0 - alpha_bar_prev) / (1.0 - alpha_bar_t)
sigma_t = np.sqrt(posterior_variance)
# 4. 注入随机高斯扰动
z = np.random.randn(*x_t.shape)
return
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: How does DDIM compress diffusion sampling from ~1000 steps down to 20–50 steps?
Next: Classifier-Free Guidance (CFG) →
💡

Core Mental Anchor / Mnemonic

Subtract scaled predicted noise to get posterior mean; inject Gaussian noise only for t > 0.

📐 Mathematical Derivation & Core Formula

xt−1=1αt(xt−βt1−αˉtϵθ(xt,t))+σtz,z∼N(0,I)x_{t-1} = \frac{1}{\sqrt{\alpha_t}}\left(x_t - \frac{\beta_t}{\sqrt{1 - \bar{\alpha}_t}} \epsilon_\theta(x_t, t)\right) + \sigma_t z, \quad z \sim \mathcal{N}(0, \mathbf{I})
### Tractable Reverse Posterior Mean Parameterization
Conditioned on x0x_0, the forward posterior q(xt−1∣xt,x0)q(x_{t-1} \mid x_t, x_0) is Gaussian with mean:
μ~t(xt,x0)=αˉt−1βt1−αˉtx0+αt(1−αˉt−1)1−αˉtxt\tilde{\mu}_t(x_t, x_0) = \frac{\sqrt{\bar{\alpha}_{t-1}}\beta_t}{1 - \bar{\alpha}_t} x_0 + \frac{\sqrt{\alpha_t}(1 - \bar{\alpha}_{t-1})}{1 - \bar{\alpha}_t} x_t
Substituting x0=xt−1−αˉtϵθ(xt,t)αˉtx_0 = \frac{x_t - \sqrt{1 - \bar{\alpha}_t} \epsilon_\theta(x_t, t)}{\sqrt{\bar{\alpha}_t}} gives the DDPM reverse sampling step xt−1=μθ(xt,t)+σtzx_{t-1} = \mu_\theta(x_t, t) + \sigma_t z (z=0z=0 at t=0t=0).

🔄 Tensor Dimensions & Shape Flow

x_t, eps_theta -> compute posterior mean -> add sigma_t * z if t > 0 -> output x_{t-1}

🛡️ Numerical Stability & Pitfalls Checklist

  • Set stochastic noise z = 0 strictly at the final step (t == 0) to avoid residual speckle.
  • DDIM skips Markov noise (sigma_t = 0) to trace deterministic ODE steps in 20–50 iterations.

💻 Reference Code Implementation

import numpy as np

def ddpm_p_sample_step(
    x_t: np.ndarray,              # 当前时间步带噪图 (B, C, H, W)
    t_idx: int,                   # 当前时间步整数 t
    predicted_noise: np.ndarray,  # 神经网络预测的噪声 epsilon (B, C, H, W)
    alphas: np.ndarray,           # (T,)
    alphas_cumprod: np.ndarray,   # (T,)
    betas: np.ndarray             # (T,)
) -> np.ndarray:
    alpha_t = alphas[t_idx]
    beta_t = betas[t_idx]
    alpha_bar_t = alphas_cumprod[t_idx]
    
    # 1. 估计逆向均值 mu_t
    coeff = beta_t / np.sqrt(1.0 - alpha_bar_t)
    mean = (1.0 / np.sqrt(alpha_t)) * (x_t - coeff * predicted_noise)
    
    # 2. 如果是最后一步 t=0,直接返回均值,不再注入扰动
    if t_idx == 0:
        return mean
        
    # 3. 计算扰动方差 sigma_t (DDPM 原文常用 beta_t 或后验方差)
    alpha_bar_prev = alphas_cumprod[t_idx - 1] if t_idx > 0 else 1.0
    posterior_variance = beta_t * (1.0 - alpha_bar_prev) / (1.0 - alpha_bar_t)
    sigma_t = np.sqrt(posterior_variance)
    
    # 4. 注入随机高斯扰动
    z = np.random.randn(*x_t.shape)
    return mean + sigma_t * z

🧪 Runnable Assertions & Validation

Copy and run directly in Python / Jupyter:

import numpy as np
betas = np.linspace(1e-4, 0.02, 10)
alphas = 1.0 - betas
alphas_cumprod = np.cumprod(alphas)
x_t = np.random.randn(1, 3, 8, 8)
pred_eps = np.zeros_like(x_t)
x_next = ddpm_p_sample_step(x_t, 9, pred_eps, alphas, alphas_cumprod, betas)
assert x_next.shape == (1, 3, 8, 8)
print("✓ DDPM 单步反向去噪模拟自测通过")

🎯 Core Follow-up Inquiries & Reference Answers

Q1:How does DDIM compress diffusion sampling from ~1000 steps down to 20–50 steps?
DDIM generalizes the forward chain to a non-Markovian family sharing the same marginals; setting stochastic noise σt=0\sigma_t = 0 turns reverse generation into a deterministic ODE trajectory solvable with large step sizes.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
← Prev: DDPM Forward Diffusion ProcessBack to Part K & All 105Next: Classifier-Free Guidance (CFG) →