M6-069M6: Multimodal & Generative ModelsFlow Matching & Rectified FlowMedium
⚡Flashcard Drill➔Mastery:
Conditional Flow Matching vs Marginal Flow Matching: Tractability and Equivalence
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Conditional Flow Matching overcomes the intractability of marginal vector field integration by regressing sample-conditioned vector fields that share identical gradients in expectation.
📌 Key Takeaways
- •The marginal intractability barrier: training marginal vector field directly requires integrating over all training samples, which is computationally intractable in high dimensions
- •Conditional Flow Matching theorem: regressing tractable per-sample conditional velocity fields provably matches the marginal velocity field in expectation
- •Objective unification: establishes that simple point-to-point conditional regression optimizes the true continuous probability density transport path
📐 Mathematical Derivations
Mathematical Mechanism: 1. Continuous Normalizing Flow (CNF) & Marginal Field: A continuous vector field generates probability path via ODE . By the continuity equation: The ideal Marginal Flow Matching (FM) objective is: where is the marginal target velocity. Computing requires integrating over the full data distribution , which is intractable. 2. Conditional Flow Matching (CFM) Theorem (Lipman et al., 2023): Conditioning the probability path on individual data samples and base noise : The Conditional Flow Matching loss is: Expanding the squared norm: Proving that regressing simple conditional vector fields optimizes the intractable marginal transport field with zero approximation error.
🏭 Production Trade-offs
Deep Dive & Engineering Trade-offs: ① The Core Insight: Conditional Expectation Equals Marginal Target: Just as Denoising Score Matching uses tractable conditional scores to estimate the intractable marginal score , Conditional Flow Matching uses tractable conditional velocity to train the marginal velocity field. This structural insight transforms continuous normalizing flows from theoretical curiosities into practical generative foundation models. ② Path Flexibility: While diffusion models are locked into specific Gaussian Markov perturbation schedules, CFM supports arbitrary conditional probability paths . Practitioners can design paths that obey Optimal Transport, straight lines, or physical constraints (e.g., rigid body mechanics in molecular docking). ③ Uniform Scale Dynamics: In straight-path CFM (), target velocity is constant across all . The network output target has uniform scale throughout training, eliminating the gradient volatility and SNR weighting hacks required in diffusion models. ⑤ Interview Strategy: Formulate the continuity equation, write down both and , prove gradient equivalence via Fubini's theorem, and contrast CFM's constant velocity with score matching's volatile scales.
⚠️ Common Interview Pitfalls
- ✕Attempting to approximate the marginal vector field via Monte Carlo density estimation rather than using Conditional Flow Matching
- ✕Assuming Conditional Flow Matching generates samples via conditional generation; sampling integrates the marginal ODE unconditionally or with CFG
- ✕Confusing the conditional velocity field with the marginal ODE vector field
🎯 Interviewer Follow-up Inquiries
- ?How does Fubini's theorem prove that the gradients of the Conditional Flow Matching objective equal the gradients of the marginal Flow Matching loss?
- ?What mathematical advantages does Flow Matching's constant velocity target () provide over diffusion score targets across different timesteps?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚