M6-052M6: Multimodal & Generative ModelsGuidance & Fast Sampling (CFG / DDIM)Easy
⚡Flashcard Drill➔Mastery:
Classifier-Free Guidance (CFG): Mathematical Formulation and Conditional Steering
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Classifier-Free Guidance steers diffusion generation toward prompt conditions by linearly extrapolating between conditional and unconditional score predictions, eliminating external classifier networks.
📌 Key Takeaways
- •Extrapolation formula: combines unconditional prediction and conditional prediction via guidance scale :
- •Joint training protocol: randomly drops the conditioning prompt with probability during training, replacing it with empty token
- •Score sharpening effect: setting amplifies conditional modes and suppresses unconditional background density, boosting text alignment and visual clarity at the expense of sample diversity
📐 Mathematical Derivations
Mathematical Mechanism: 1. Implicit Classifier Bayes Decomposition: By Bayes' rule, the conditional score decomposes into unconditional score plus classifier gradient: Ho & Salimans (2021) observed that the implicit classifier gradient can be expressed purely using conditional and unconditional generative models: 2. Guided Score Formulation: Substituting this implicit classifier gradient into guided score with temperature/scale multiplier : 3. Noise Predictor Parameterization: Since score relates to predicted noise via , the guided noise prediction is: When , generation reduces to standard conditional diffusion without guidance. When , the vector difference pushes the generation trajectory away from generic unconditional data toward the conditioned prompt manifold.
🏭 Production Trade-offs
Deep Dive & Engineering Trade-offs: ① The Dual Forward Pass Tax: Standard CFG requires evaluating the neural network twice per sampling step: once for and once for . In production serving, the batch size is doubled (), doubling latency and FLOPs per request. ② The Conditioning Dropout Rate ($p_{\text{uncond}}$): Setting too low () prevents the model from learning a robust unconditional prior , causing CFG extrapolation to produce erratic artifacts. Setting too high () degrades conditional text adherence. Industry standard consensus fixes . ③ Negative Prompting as Dynamic Unconditional Steering: In standard CFG, the unconditional branch uses an empty string . In creative text-to-image tools (Automatic1111, Midjourney), users provide a 'negative prompt' ('blurry, distorted, extra limbs'). Replacing with actively repels generated samples from the negative semantic manifold. ④ Interview Strategy: Formulate the Bayesian score decomposition, derive the linear combination , explain how training with condition dropout () enables single-model dual capability, and explain the negative prompting mechanism.
⚠️ Common Interview Pitfalls
- ✕Setting guidance scale and expecting strong text prompt adherence; without guidance (), complex prompt composition collapses
- ✕Forgetting to train with unconditional condition dropout (), making CFG extrapolation impossible at inference time
- ✕Assuming CFG adds no serving compute; CFG doubles the effective forward-pass batch size at every sampling step
🎯 Interviewer Follow-up Inquiries
- ?How does substituting a negative text prompt in place of the empty condition in CFG mathematically repel unwanted visual features?
- ?Why is conditioning dropout () during training mandatory for enabling Classifier-Free Guidance at inference time?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚