M6-070M6: Multimodal & Generative ModelsFlow Matching & Rectified FlowMedium
⚡Flashcard Drill➔Mastery:
Interconnection Between Flow Matching and Score-Based Denoising
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Flow Matching and Score Matching represent dual parameterizations of the same continuous probability transport, connected via a linear transformation linking velocity vector fields to the score of marginal densities.
📌 Key Takeaways
- •Conceptual duality: Score Matching estimates the spatial gradient of log-density , while Flow Matching estimates the temporal drift velocity
- •Linear bridge identity: along probability flow trajectories, velocity relates directly to score: , proving structural equivalence
- •Training advantages: Flow Matching with straight paths eliminates the extreme variance of score functions near , stabilizing training across all timesteps
📐 Mathematical Derivations
Mathematical Mechanism: 1. Velocity-to-Score Derivation: Recall the continuous-time Probability Flow ODE corresponding to a forward diffusion SDE : In Flow Matching, the ODE is defined directly by velocity field : Equating both ODE drift terms reveals the exact analytical relationship: 2. Variance-Preserving (VP) Diffusion Equivalence: For VP diffusion where and : Because , the velocity field is a direct linear reparameterization of predicted noise . 3. The Geometric Path Distinction: (a) Diffusion Path: Enforces curved trigonometric trajectories on the unit hypersphere: . (b) Flow Matching Optimal Transport Path: Connects endpoints via Euclidean straight lines: .
🏭 Production Trade-offs
Deep Dive & Engineering Trade-offs: ① Numerical Stability of Flow Matching over Score Matching: As in diffusion, data variance collapses (), causing the score function to explode towards infinity (). Network training requires aggressive loss weighting (e.g., , SNR clipping) to prevent gradient blowup. In Flow Matching with straight paths, velocity remains strictly bounded across all , providing inherent numerical stability without heuristic loss reweighting. ② Algorithmic Portability: Because Flow Matching is an ODE formulation, all advanced diffusion sampling tools—Classifier-Free Guidance (CFG), high-order numerical solvers (DPM-Solver, Heun), and deterministic latent inversion—port directly to Flow Matching models with zero modifications. ③ Model Migration: Existing diffusion checkpoints parameterizing velocity (such as SD 2.1 -prediction) can be sampled using Flow Matching ODE integrators by remapping time variables. ⑤ Interview Strategy: Formulate the Probability Flow ODE, derive the linear relationship between velocity and score , contrast curved diffusion paths against straight OT paths, and explain why Flow Matching avoids score explosion at low noise.
⚠️ Common Interview Pitfalls
- ✕Assuming Flow Matching is an entirely disconnected theory from diffusion; Flow Matching is a generalized continuous ODE framework containing diffusion as a curved-path special case
- ✕Failing to account for the drift term when converting between diffusion score networks and Flow Matching velocity fields
- ✕Expecting straight-path Flow Matching to follow Brownian motion; straight-path CFM is purely deterministic ODE transport
🎯 Interviewer Follow-up Inquiries
- ?How does the analytical transformation convert a pre-trained score model into a velocity field?
- ?Why does the score function explode near in diffusion models while Flow Matching velocity remains strictly bounded?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚