M6-073M6: Multimodal & Generative ModelsFlow Matching & Rectified FlowHard
⚡Flashcard Drill➔Mastery:
Mathematical Derivation of the Vector Field Regression Objective in Flow Matching
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
The Flow Matching training objective is derived by establishing that regressing a constant conditional velocity field along linear probability paths provably minimizes the marginal continuous transport error in expectation.
📌 Key Takeaways
- •Linear displacement path: defines intermediate states via linear interpolation between noise and target data
- •Time-invariant conditional velocity: differentiating with respect to yields constant target velocity
- •Unbiased marginal regression: by the law of total expectation, minimizing conditional MSE directly optimizes the true marginal vector field
📐 Mathematical Derivations
Mathematical Mechanism: 1. Path Definition: Given base distribution and target data distribution , define the linear interpolation trajectory for : 2. Conditional Velocity Field Derivation: Differentiating the trajectory with respect to continuous time parameter : Crucially, this conditional velocity vector is completely independent of time $t$ and depends strictly on the endpoints . 3. Regression Objective Formulation: The conditional regression loss over parameter is: 4. Proof of Marginal Alignment: Expand the conditional expectation conditioned on : By the projection property of conditional expectation, the optimal velocity field satisfies: Which is exactly the true marginal vector field governing the continuity equation .
🏭 Production Trade-offs
Deep Dive & Engineering Trade-offs: ① The Unmatched Mathematical Elegance of CFM: In DDPM, training requires deriving ELBO telescoping sums, computing intractable Gaussian KL divergences, dropping prefactors, and tuning noise schedules . In Flow Matching, the entire foundation reduces to four lines of calculus: define , differentiate to get velocity , regress with MSE , and cite conditional expectation. This mathematical transparency is why the generative AI community has pivoted to Flow Matching. ② Time Convention Discrepancy: In standard physics and Flow Matching literature, represents pure noise (prior ) and represents clean data (). However, some diffusion-heritage implementations (Stable Diffusion 3) invert this convention: is noise and is clean data, where . Ensuring consistent sign conventions in codebases is critical. ③ Weighting Functions in CFM: While uniform weighting is standard, weighting the loss by time-dependent prefactors can focus model capacity on perceptually critical intermediate timesteps. ⑤ Interview Strategy: Write down the linear path equation, differentiate to derive the constant velocity , formulate the CFM regression loss, prove that conditional expectation yields the marginal field , and emphasize its training stability.
⚠️ Common Interview Pitfalls
- ✕Inverting the sign of the target velocity vector ( vs ) due to mismatched time boundary conventions ( vs )
- ✕Attempting to introduce complex non-linear trigonometric functions into when linear straight paths achieve optimal simplicity
- ✕Assuming predicts the clean image ; predicts the directional velocity vector field
🎯 Interviewer Follow-up Inquiries
- ?How does the projection property of conditional expectation prove that regressing yields the exact marginal velocity field ?
- ?What happens mathematically if the time boundary convention is flipped so that represents clean data and represents pure noise?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚