K4HardPart K · Generative Models & Diffusion
Flow Matching Velocity Field Objective
Regress constant velocity fields along straight optimal-transport paths between data and Gaussian noise.
⏱️ Time:
O(B * C * H * W)💾 Space:
O(B * C * H * W)🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
def flow_matching_loss_and_sample(
x_data: np.ndarray, # x_0: 清晰样本 (B, C, H, W)
x_noise: np.ndarray, # x_1: 纯高斯白噪声 (B, C, H, W)
t: np.ndarray # 时间步浮点数 t 属于 [0, 1] (B,)
) -> tuple:
# 广播维度 (B, 1, 1, 1)
t_broad =
# 1. 线性最优传输插值路径
x_t = (1.0 - t_broad) * x_data + t_broad * x_noise
# 2. 真实目标速度场向量: u_t = x_1 - x_0
target_velocity =
return x_t, target_velocity
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: How does Flow Matching simplify generative modeling conceptually compared to DDPM?
💡
Core Mental Anchor / Mnemonic
Interpolate linearly between x0 and noise x1; train network to predict constant velocity (x1 - x0).
📐 Mathematical Derivation & Core Formula
### Optimal-Transport Straight Velocity Trajectories
Flow Matching (Lipman et al., 2022) learns a deterministic velocity field along straight conditional optimal-transport paths (, data and noise ).
The target velocity is constant along each straight path (), giving the regression loss and allowing fast, low-step ODE integration.
Flow Matching (Lipman et al., 2022) learns a deterministic velocity field along straight conditional optimal-transport paths (, data and noise ).
The target velocity is constant along each straight path (), giving the regression loss and allowing fast, low-step ODE integration.
🔄 Tensor Dimensions & Shape Flow
x_data, x_noise, t -> interpolate x_t -> target velocity (x_noise - x_data) -> MSE loss
🛡️ Numerical Stability & Pitfalls Checklist
- Sample continuous timesteps t in
[0, 1]rather than discrete diffusion step indices. - Modern models (Flux/SD3) bias t sampling with Logit-Normal distributions toward mid-trajectory steps.
💻 Reference Code Implementation
import numpy as np
def flow_matching_loss_and_sample(
x_data: np.ndarray, # x_0: 清晰样本 (B, C, H, W)
x_noise: np.ndarray, # x_1: 纯高斯白噪声 (B, C, H, W)
t: np.ndarray # 时间步浮点数 t 属于 [0, 1] (B,)
) -> tuple:
# 广播维度 (B, 1, 1, 1)
t_broad = t[:, np.newaxis, np.newaxis, np.newaxis]
# 1. 线性最优传输插值路径
x_t = (1.0 - t_broad) * x_data + t_broad * x_noise
# 2. 真实目标速度场向量: u_t = x_1 - x_0
target_velocity = x_noise - x_data
return x_t, target_velocity
🧪 Runnable Assertions & Validation
Copy and run directly in Python / Jupyter:
import numpy as np
x0 = np.zeros((2, 2))
x1 = np.ones((2, 2))
t = np.array([0.5, 0.5])
xt, v = flow_matching_loss_and_sample(x0, x1, t)
assert np.allclose(xt, 0.5), "t=0.5 时插值应恰为中点 0.5"
assert np.allclose(v, 1.0), "速度场向量应为恒定 1.0"
print("✓ Flow Matching 速度场计算自测通过")🎯 Core Follow-up Inquiries & Reference Answers
Q1:How does Flow Matching simplify generative modeling conceptually compared to DDPM?
Instead of stochastic SDE noise schedules and reverse variance parameterizations, Flow Matching frames generation as regressing a deterministic ODE velocity vector field along straight interpolation paths between noise and data.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.