K1Medium★ Core Essential · Industrial BedrockPart K · Generative Models & Diffusion

DDPM Forward Diffusion Process

⚡Flashcard Drill➔

Closed-form DDPM forward noising using cumulative alpha products to sample any timestep t in O(1).

⏱️ Time: O(B * C * H * W) 单步闭式计算
💾 Space: O(T) 预计算调度缓存
🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
class DDPMForwardDiffusion:
def __init__(self, timesteps: int = 1000, beta_start: float = 1e-4, beta_end: float = 0.02):
self.timesteps = timesteps
# 1. 线性噪声调度 (Linear Beta Schedule)
self.betas =
self.alphas = 1.0 - self.betas
# 2. 累乘 alpha_bar
self.alphas_cumprod = np.cumprod(self.alphas, axis=0)
self.sqrt_alphas_cumprod = np.sqrt(self.alphas_cumprod)
self.sqrt_one_minus_alphas_cumprod = np.sqrt(1.0 - self.alphas_cumprod)
def q_sample(self, x_0: np.ndarray, t: np.ndarray, noise: np.ndarray = None) -> np.ndarray:
"""
闭式加噪:一步直接获得 x_t
参数:
x_0: (B, C, H, W) 原始清晰输入图像
t: (B,) 随机时间步索引 (0 <= t < timesteps)
noise: (B, C, H, W) 标准高斯噪声,若 None 则自动采样
"""
if noise is None:
noise = np.random.randn(*x_0.shape)
# 提取当前时间步的系数并重塑为 (B, 1, 1, 1) 供广播
sqrt_alpha_bar = self.sqrt_alphas_cumprod[t][:, np.newaxis, np.newaxis, np.newaxis]
sqrt_one_minus_alpha_bar = self.sqrt_one_minus_alphas_cumprod[t][:, np.newaxis, np.newaxis, np.newaxis]
# 核心闭式公式: sqrt(alpha_bar) * x_0 + sqrt(1 - alpha_bar) * noise
return
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: What does a standard DDPM network predict in its training objective: x_0 or the added noise epsilon?
Next: DDPM Reverse Denoising Sampling Step →
💡

Core Mental Anchor / Mnemonic

Multiply alphas into alpha_bar_t and sample x_t directly from x_0 in one closed-form step.

📐 Mathematical Derivation & Core Formula

q(xt∣x0)=N(xt; αˉtx0, (1−αˉt)I)  ⟹  xt=αˉtx0+1−αˉtϵq(x_t \mid x_0) = \mathcal{N}\left(x_t; \, \sqrt{\bar{\alpha}_t} x_0, \, (1 - \bar{\alpha}_t) \mathbf{I}\right) \implies x_t = \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1 - \bar{\alpha}_t} \epsilon
### Markov Noise Schedule Closed-Form Marginals
Starting from single-step corruption xt=αtxt−1+1−αtϵt−1x_t = \sqrt{\alpha_t} x_{t-1} + \sqrt{1 - \alpha_t} \epsilon_{t-1} (αt=1−βt\alpha_t = 1 - \beta_t) and recursively composing independent Gaussians yields the closed-form marginal with αˉt=∏s=1tαs\bar{\alpha}_t = \prod_{s=1}^t \alpha_s:
xt=αˉtx0+1−αˉtϵ,ϵ∼N(0,I)x_t = \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1 - \bar{\alpha}_t} \epsilon, \quad \epsilon \sim \mathcal{N}(0, \mathbf{I})
Training therefore samples any timestep tt directly in O(1)O(1) without simulating steps 1…t−11 \dots t-1.

🔄 Tensor Dimensions & Shape Flow

x_0: (B, C, H, W), t: (B,) -> gather sqrt_alpha_bar -> mix x_0 + noise -> x_t: (B, C, H, W)

🛡️ Numerical Stability & Pitfalls Checklist

  • Precompute cumulative schedule products cumprod in float64 to prevent cumulative drift across 1000+ steps.
  • Ensure discrete timesteps t stay within [0, timesteps - 1].
  • Verify alpha_bar approaches 0 at max t so x_T matches isotropic Gaussian noise.

💻 Reference Code Implementation

import numpy as np

class DDPMForwardDiffusion:
    def __init__(self, timesteps: int = 1000, beta_start: float = 1e-4, beta_end: float = 0.02):
        self.timesteps = timesteps
        # 1. 线性噪声调度 (Linear Beta Schedule)
        self.betas = np.linspace(beta_start, beta_end, timesteps, dtype=np.float64)
        self.alphas = 1.0 - self.betas
        # 2. 累乘 alpha_bar
        self.alphas_cumprod = np.cumprod(self.alphas, axis=0)
        self.sqrt_alphas_cumprod = np.sqrt(self.alphas_cumprod)
        self.sqrt_one_minus_alphas_cumprod = np.sqrt(1.0 - self.alphas_cumprod)
        
    def q_sample(self, x_0: np.ndarray, t: np.ndarray, noise: np.ndarray = None) -> np.ndarray:
        """
        闭式加噪:一步直接获得 x_t
        参数:
            x_0: (B, C, H, W) 原始清晰输入图像
            t: (B,) 随机时间步索引 (0 <= t < timesteps)
            noise: (B, C, H, W) 标准高斯噪声,若 None 则自动采样
        """
        if noise is None:
            noise = np.random.randn(*x_0.shape)
            
        # 提取当前时间步的系数并重塑为 (B, 1, 1, 1) 供广播
        sqrt_alpha_bar = self.sqrt_alphas_cumprod[t][:, np.newaxis, np.newaxis, np.newaxis]
        sqrt_one_minus_alpha_bar = self.sqrt_one_minus_alphas_cumprod[t][:, np.newaxis, np.newaxis, np.newaxis]
        
        # 核心闭式公式: sqrt(alpha_bar) * x_0 + sqrt(1 - alpha_bar) * noise
        return sqrt_alpha_bar * x_0 + sqrt_one_minus_alpha_bar * noise

🧪 Runnable Assertions & Validation

Copy and run directly in Python / Jupyter:

import numpy as np
diff = DDPMForwardDiffusion(timesteps=100)
x_0 = np.ones((2, 3, 16, 16))
t = np.array([0, 99])
x_t = diff.q_sample(x_0, t)
assert x_t.shape == (2, 3, 16, 16)
# t=0 时 alpha_bar 接近 1,应非常接近 x_0
assert np.isclose(x_t[0].mean(), 1.0, atol=0.1)
print("✓ DDPM 前向加噪闭式计算自测通过")

🎯 Core Follow-up Inquiries & Reference Answers

Q1:What does a standard DDPM network predict in its training objective: x_0 or the added noise epsilon?
Typically the injected standard Gaussian noise ϵ\epsilon, minimizing Lsimple=Et,x0,ϵ[∥ϵ−ϵθ(xt,t)∥2]\mathcal{L}_{\text{simple}} = \mathbb{E}_{t, x_0, \epsilon}[\|\epsilon - \epsilon_\theta(x_t, t)\|^2], which Ho et al. showed balances gradient magnitudes cleanly across noise levels.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
← Prev: Pre-Filtered Hybrid Search with RRF FusionBack to Part K & All 105Next: DDPM Reverse Denoising Sampling Step →