K2HardPart K · Generative Models & Diffusion
DDPM Reverse Denoising Sampling Step
DDPM reverse posterior denoising step estimating mu_theta from predicted noise eps_theta.
⏱️ Time:
O(T * B * C * H * W) 推理需串行循环 T 次💾 Space:
O(B * C * H * W)🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
def ddpm_p_sample_step(
x_t: np.ndarray, # 当前时间步带噪图 (B, C, H, W)
t_idx: int, # 当前时间步整数 t
predicted_noise: np.ndarray, # 神经网络预测的噪声 epsilon (B, C, H, W)
alphas: np.ndarray, # (T,)
alphas_cumprod: np.ndarray, # (T,)
betas: np.ndarray # (T,)
) -> np.ndarray:
alpha_t = alphas[t_idx]
beta_t = betas[t_idx]
alpha_bar_t = alphas_cumprod[t_idx]
# 1. 估计逆向均值 mu_t
coeff =
mean = (1.0 / np.sqrt(alpha_t)) * (x_t - coeff * predicted_noise)
# 2. 如果是最后一步 t=0,直接返回均值,不再注入扰动
if t_idx == 0:
return mean
# 3. 计算扰动方差 sigma_t (DDPM 原文常用 beta_t 或后验方差)
alpha_bar_prev = alphas_cumprod[t_idx - 1] if t_idx > 0 else 1.0
posterior_variance = beta_t * (1.0 - alpha_bar_prev) / (1.0 - alpha_bar_t)
sigma_t = np.sqrt(posterior_variance)
# 4. 注入随机高斯扰动
z = np.random.randn(*x_t.shape)
return
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: How does DDIM compress diffusion sampling from ~1000 steps down to 20–50 steps?
💡
Core Mental Anchor / Mnemonic
Subtract scaled predicted noise to get posterior mean; inject Gaussian noise only for t > 0.
📐 Mathematical Derivation & Core Formula
### Tractable Reverse Posterior Mean Parameterization
Conditioned on , the forward posterior is Gaussian with mean:
Substituting gives the DDPM reverse sampling step ( at ).
Conditioned on , the forward posterior is Gaussian with mean:
Substituting gives the DDPM reverse sampling step ( at ).
🔄 Tensor Dimensions & Shape Flow
x_t, eps_theta -> compute posterior mean -> add sigma_t * z if t > 0 -> output x_{t-1}
🛡️ Numerical Stability & Pitfalls Checklist
- Set stochastic noise
z = 0strictly at the final step (t == 0) to avoid residual speckle. - DDIM skips Markov noise (
sigma_t = 0) to trace deterministic ODE steps in 20–50 iterations.
💻 Reference Code Implementation
import numpy as np
def ddpm_p_sample_step(
x_t: np.ndarray, # 当前时间步带噪图 (B, C, H, W)
t_idx: int, # 当前时间步整数 t
predicted_noise: np.ndarray, # 神经网络预测的噪声 epsilon (B, C, H, W)
alphas: np.ndarray, # (T,)
alphas_cumprod: np.ndarray, # (T,)
betas: np.ndarray # (T,)
) -> np.ndarray:
alpha_t = alphas[t_idx]
beta_t = betas[t_idx]
alpha_bar_t = alphas_cumprod[t_idx]
# 1. 估计逆向均值 mu_t
coeff = beta_t / np.sqrt(1.0 - alpha_bar_t)
mean = (1.0 / np.sqrt(alpha_t)) * (x_t - coeff * predicted_noise)
# 2. 如果是最后一步 t=0,直接返回均值,不再注入扰动
if t_idx == 0:
return mean
# 3. 计算扰动方差 sigma_t (DDPM 原文常用 beta_t 或后验方差)
alpha_bar_prev = alphas_cumprod[t_idx - 1] if t_idx > 0 else 1.0
posterior_variance = beta_t * (1.0 - alpha_bar_prev) / (1.0 - alpha_bar_t)
sigma_t = np.sqrt(posterior_variance)
# 4. 注入随机高斯扰动
z = np.random.randn(*x_t.shape)
return mean + sigma_t * z
🧪 Runnable Assertions & Validation
Copy and run directly in Python / Jupyter:
import numpy as np
betas = np.linspace(1e-4, 0.02, 10)
alphas = 1.0 - betas
alphas_cumprod = np.cumprod(alphas)
x_t = np.random.randn(1, 3, 8, 8)
pred_eps = np.zeros_like(x_t)
x_next = ddpm_p_sample_step(x_t, 9, pred_eps, alphas, alphas_cumprod, betas)
assert x_next.shape == (1, 3, 8, 8)
print("✓ DDPM 单步反向去噪模拟自测通过")🎯 Core Follow-up Inquiries & Reference Answers
Q1:How does DDIM compress diffusion sampling from ~1000 steps down to 20–50 steps?
DDIM generalizes the forward chain to a non-Markovian family sharing the same marginals; setting stochastic noise turns reverse generation into a deterministic ODE trajectory solvable with large step sizes.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.