M6-075M6: Multimodal & Generative ModelsFlow Matching & Rectified FlowHard
⚡Flashcard Drill➔Mastery:
Unified Framework of Continuous Diffusion SDEs, Probability Flow ODEs, and Flow Matching
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Continuous diffusion SDEs, deterministic Probability Flow ODEs, and Flow Matching form a unified mathematical continuum of continuous-time generative transport, differing only in trajectory curvature and stochastic noise injection.
📌 Key Takeaways
- •The unifying differential equation: all three paradigms transport a prior noise distribution to the data distribution via continuous vector fields governed by the Fokker-Planck continuity equation
- •Stochastic vs deterministic duality: SDEs inject continuous Brownian perturbations during reverse sampling, while ODEs and Flow Matching follow deterministic paths with identical marginals
- •Trajectory geometry evolution: progresses from curved stochastic trajectories (DDPM SDE) to curved deterministic trajectories (DDIM / Probability Flow ODE) to straight optimal transport trajectories (Flow Matching)
📐 Mathematical Derivations
Mathematical Mechanism: 1. The Master SDE Formulation (Song et al., 2021): Forward transport: Generative reverse transport: 2. The Generalized Probability Flow ODE Family: Parameterize sampling via stochasticity parameter : (a) Standard Reverse SDE (DDPM). (b) Deterministic Probability Flow ODE (DDIM): 3. The Flow Matching Generalization: Flow Matching replaces the diffusion drift and score terms with a direct velocity vector field : Under linear displacement interpolation : Transforming curved Brownian diffusion paths into straight-line optimal transport flows. 4. Paradigm Comparison Matrix: \begin{array}{l|c|c|c} \textbf{Feature} & \textbf{Diffusion SDE (DDPM)} & \textbf{Probability Flow ODE (DDIM)} & \textbf{Flow Matching (SD3/Flux)} \\ \hline \text{Trajectory Type} & \text{Stochastic / Curved} & \text{Deterministic / Curved} & \textbf{Deterministic / Straight} \\ \text{Target Variable} & \text{Noise } \epsilon_\theta \text{ or Score } s_\theta & \text{Noise } \epsilon_\theta & \textbf{Velocity } v_\theta = x_1 - x_0 \\ \text{Optimal Solvers} & \text{Ancestral SDE (Euler-a)} & \text{Semi-linear ODE (DPM-Solver)} & \textbf{Standard Euler / Midpoint} \\ \text{Exact Likelihood} & \text{Intractable (variational bound)} & \textbf{Exact (change of variables)} & \textbf{Exact (change of variables)} \\ \text{Inversion Ability} & \text{No (stochastic noise)} & \textbf{Yes (deterministic ODE)} & \textbf{Yes (deterministic ODE)} \end{array}
🏭 Production Trade-offs
Deep Dive & Engineering Trade-offs: ① The Trajectory Curvature Continuum: The fundamental evolutionary progression in generative modeling has been straightening the transport path. DDPM's path is highly curved and noisy, requiring 1,000 steps. DDIM straightens the path into a deterministic curve, reducing steps to 50. Flow Matching straightens the trajectory into Euclidean lines, enabling sampling in 15-20 steps. Reflow straightens paths further, enabling 1-2 step generation. ② Likelihood Computation and Inversion: Both Probability Flow ODEs and Flow Matching allow exact continuous log-likelihood computation via the instantaneous change-of-variables formula: , where the Jacobian trace is computed using the Hutchinson estimator. ③ Stochastic Self-Correction vs Speed: Stochastic SDEs retain an advantage in noisy, imperfect models: injecting Gaussian noise acts as a Langevin MCMC stabilizer that knocks drifting samples back into high-probability manifolds. Deterministic ODEs and Flow Matching lack this noise cushion; model approximation errors accumulate monotonically along the trajectory. ⑤ Interview Strategy: Write the master SDE equation with parameter , show how gives DDPM and gives the Probability Flow ODE, show how Flow Matching sets with straight velocity , and articulate the historical trajectory straightening progression.
⚠️ Common Interview Pitfalls
- ✕Viewing diffusion models and flow matching as competing mutually exclusive paradigms rather than continuous-time siblings
- ✕Assuming stochastic SDEs have no advantages; SDE noise injection provides valuable trajectory error-correction in low-capacity models
- ✕Forgetting that Probability Flow ODEs and Flow Matching both support exact data log-likelihood computation via Jacobian trace integration
🎯 Interviewer Follow-up Inquiries
- ?How does the stochasticity parameter continuously interpolate between the deterministic Probability Flow ODE and the stochastic reverse SDE?
- ?What mathematical progression connects DDPM's 1000-step diffusion to Flow Matching's 20-step straight transport?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚