M6-067M6: Multimodal & Generative ModelsFlow Matching & Rectified FlowEasy
⚡Flashcard Drill➔Mastery:
Flow Matching Foundations, Vector Field Regression, and CNF Objectives
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Flow Matching trains a neural network to regress a continuous time-dependent vector field that transports a simple Gaussian prior distribution to the data distribution along straight probability trajectories.
📌 Key Takeaways
- •Continuous Normalizing Flows (CNF): defines generative transport via ordinary differential equation , pushing simple noise into empirical data
- •Conditional Flow Matching (CFM): makes training tractable by regressing per-sample conditional vector fields , eliminating intractable marginal integration
- •Straight-path trajectory advantage: linear interpolation paths yield straight constant-velocity flows, enabling stable few-step numerical ODE integration
📐 Mathematical Derivations
Mathematical Mechanism: 1. Continuous Normalizing Flow (CNF) ODE: A time-dependent vector field defines probability flow via ODE: The probability density satisfies the continuity equation: 2. The Flow Matching Objective (Lipman et al., 2023): Directly optimizing vector field to match marginal target : is intractable because marginal velocity requires integrating over all data points. 3. Conditional Flow Matching (CFM) Theorem: Lipman et al. proved that conditioning on individual data points and prior noise yields an identical gradient: where: 4. Optimal Transport (OT) Displacement Interpolation: Define the straight probability path: The true conditional velocity vector is constant over time: Yielding the remarkably simple regression objective:
🏭 Production Trade-offs
Deep Dive & Engineering Trade-offs: ① Straight Paths vs Curved Diffusion Trajectories: Standard diffusion models follow curved stochastic Brownian paths in latent space due to variance-preserving trigonometric schedules. Curved trajectories require small step sizes and high-order solvers to avoid drifting off the manifold. Flow Matching with linear interpolation connects noise directly to data via straight lines with constant velocity . Straight trajectories can be integrated with basic first-order Euler solvers in 10-20 steps with minimal truncation error. ② Simplicity and Training Stability: The Flow Matching loss requires no noise schedule tuning (no , no , no cosine schedule offsets). Time is uniform, and target is scale-invariant, eliminating gradient spikes in extreme noise regimes. ③ Modern Foundation Adoption: Flow Matching has completely superseded traditional DDPM in modern state-of-the-art vision and audio foundation models (Stable Diffusion 3, Flux.1, Voicebox). ⑤ Interview Strategy: Define CNF via ODE , state the continuity equation, prove that Conditional Flow Matching gradients equal marginal Flow Matching gradients, derive the straight-path velocity , and contrast straight paths against curved diffusion paths.
⚠️ Common Interview Pitfalls
- ✕Attempting to integrate the marginal vector field directly during training rather than using Conditional Flow Matching (CFM)
- ✕Confusing time indexing: in Flow Matching, is pure noise and is clean data (or vice versa depending on notation)
- ✕Assuming Flow Matching requires complex noise variance schedules; straight-path CFM uses linear interpolation
🎯 Interviewer Follow-up Inquiries
- ?Why is the Conditional Flow Matching (CFM) objective mathematically equivalent in expectation to the intractable marginal Flow Matching objective?
- ?How do straight-line probability paths in Flow Matching reduce ODE solver truncation errors compared to curved diffusion trajectories?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚