M6-067M6: Multimodal & Generative ModelsFlow Matching & Rectified FlowEasy
⚡Flashcard Drill➔Mastery:

Flow Matching Foundations, Vector Field Regression, and CNF Objectives

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M6-068 →
📐 Mathematical Definition
LFM=Et,x0,x1∥vθ(xt,t)−(x1−x0)∥2,xt=(1−t)x0+tx1\mathcal{L}_{\text{FM}}=\mathbb{E}_{t,x_0,x_1}\left\|v_\theta(x_t,t)-(x_1-x_0)\right\|^2,\quad x_t=(1-t)x_0+tx_1
⚡ One-Sentence Core Answer
Flow Matching trains a neural network to regress a continuous time-dependent vector field that transports a simple Gaussian prior distribution to the data distribution along straight probability trajectories.

📌 Key Takeaways

  • •
    Continuous Normalizing Flows (CNF): defines generative transport via ordinary differential equation  dxdt=vθ(x,t) \,\frac{dx}{dt} = v_\theta(x, t)\,, pushing simple noise p0p_0 into empirical data p1p_1
  • •
    Conditional Flow Matching (CFM): makes training tractable by regressing per-sample conditional vector fields  ut(x∣x1) \,u_t(x \mid x_1)\,, eliminating intractable marginal integration
  • •
    Straight-path trajectory advantage: linear interpolation paths  xt=(1−t)x0+tx1 \,x_t = (1-t) x_0 + t x_1\, yield straight constant-velocity flows, enabling stable few-step numerical ODE integration

📐 Mathematical Derivations

Mathematical Mechanism: 1. Continuous Normalizing Flow (CNF) ODE: A time-dependent vector field vt:Rd→Rdv_t: \mathbb{R}^d \to \mathbb{R}^d defines probability flow ϕt(x)\phi_t(x) via ODE: ddtϕt(x)=vt(ϕt(x)),ϕ0(x)=x0∼p0(x)=N(0,I)\frac{d}{dt} \phi_t(x) = v_t(\phi_t(x)), \quad \phi_0(x) = x_0 \sim p_0(x) = \mathcal{N}(0, I) The probability density pt(x)p_t(x) satisfies the continuity equation: ∂pt(x)∂t+∇x⋅(pt(x)vt(x))=0\frac{\partial p_t(x)}{\partial t} + \nabla_x \cdot \big( p_t(x) v_t(x) \big) = 0 2. The Flow Matching Objective (Lipman et al., 2023): Directly optimizing vector field vθv_\theta to match marginal target ut(x)u_t(x): LFM(θ)=Et∼U[0,1],  x∼pt(x)[∥vθ(x,t)−ut(x)∥2]\mathcal{L}_{\text{FM}}(\theta) = \mathbb{E}_{t \sim \mathcal{U}[0, 1], \; x \sim p_t(x)} \big[ \| v_\theta(x, t) - u_t(x) \|^2 \big] is intractable because marginal velocity ut(x)u_t(x) requires integrating over all data points. 3. Conditional Flow Matching (CFM) Theorem: Lipman et al. proved that conditioning on individual data points x1∼q(x1)x_1 \sim q(x_1) and prior noise x0∼p0(x0)x_0 \sim p_0(x_0) yields an identical gradient: ∇θLFM(θ)=∇θLCFM(θ)\nabla_\theta \mathcal{L}_{\text{FM}}(\theta) = \nabla_\theta \mathcal{L}_{\text{CFM}}(\theta) where: LCFM(θ)=Et∼U[0,1],  x0∼p0,  x1∼q(x1),  xt∼pt(x∣x0,x1)[∥vθ(xt,t)−ut(xt∣x0,x1)∥2]\mathcal{L}_{\text{CFM}}(\theta) = \mathbb{E}_{t \sim \mathcal{U}[0, 1], \; x_0 \sim p_0, \; x_1 \sim q(x_1), \; x_t \sim p_t(x \mid x_0, x_1)} \Big[ \big\| v_\theta(x_t, t) - u_t(x_t \mid x_0, x_1) \big\|^2 \Big] 4. Optimal Transport (OT) Displacement Interpolation: Define the straight probability path: xt=(1−t)x0+tx1,t∈[0,1]x_t = (1 - t) x_0 + t x_1, \quad t \in [0, 1] The true conditional velocity vector is constant over time: ut(xt∣x0,x1)=dxtdt=x1−x0u_t(x_t \mid x_0, x_1) = \frac{d x_t}{dt} = x_1 - x_0 Yielding the remarkably simple regression objective: LCFM(θ)=Et,x0,x1[∥vθ((1−t)x0+tx1,  t)−(x1−x0)∥2]\mathcal{L}_{\text{CFM}}(\theta) = \mathbb{E}_{t, x_0, x_1} \Big[ \big\| v_\theta\big( (1-t)x_0 + t x_1, \; t \big) - (x_1 - x_0) \big\|^2 \Big]

🏭 Production Trade-offs

Deep Dive & Engineering Trade-offs: ① Straight Paths vs Curved Diffusion Trajectories: Standard diffusion models follow curved stochastic Brownian paths in latent space due to variance-preserving trigonometric schedules. Curved trajectories require small step sizes and high-order solvers to avoid drifting off the manifold. Flow Matching with linear interpolation connects noise directly to data via straight lines with constant velocity u=x1−x0u = x_1 - x_0. Straight trajectories can be integrated with basic first-order Euler solvers in 10-20 steps with minimal truncation error. ② Simplicity and Training Stability: The Flow Matching loss requires no noise schedule tuning (no βt\beta_t, no αt\alpha_t, no cosine schedule offsets). Time t∈[0,1]t \in [0, 1] is uniform, and target x1−x0x_1 - x_0 is scale-invariant, eliminating gradient spikes in extreme noise regimes. ③ Modern Foundation Adoption: Flow Matching has completely superseded traditional DDPM in modern state-of-the-art vision and audio foundation models (Stable Diffusion 3, Flux.1, Voicebox). ⑤ Interview Strategy: Define CNF via ODE dxdt=vt(x)\frac{dx}{dt} = v_t(x), state the continuity equation, prove that Conditional Flow Matching gradients equal marginal Flow Matching gradients, derive the straight-path velocity ut=x1−x0u_t = x_1 - x_0, and contrast straight paths against curved diffusion paths.
⚠️ Common Interview Pitfalls
  • ✕
    Attempting to integrate the marginal vector field directly during training rather than using Conditional Flow Matching (CFM)
  • ✕
    Confusing time indexing: in Flow Matching, t=0t=0 is pure noise and t=1t=1 is clean data (or vice versa depending on notation)
  • ✕
    Assuming Flow Matching requires complex noise variance schedules; straight-path CFM uses linear interpolation xt=(1−t)x0+tx1x_t = (1-t)x_0 + tx_1
🎯 Interviewer Follow-up Inquiries
  • ?
    Why is the Conditional Flow Matching (CFM) objective mathematically equivalent in expectation to the intractable marginal Flow Matching objective?
  • ?
    How do straight-line probability paths in Flow Matching reduce ODE solver truncation errors compared to curved diffusion trajectories?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM6-066: Latent Autoencoder Selection: Continuous KL-Regularized VAE vs Discrete VQ-VAE📋 Back to BankNext →M6-068: Rectified Flow, Straight Trajectories, and the Reflow Iteration Protocol