M5-026M5: NLP & Large Language ModelsInstruction Tuning & Supervised Fine-TuningMedium
⚡Flashcard Drill➔Mastery:
Choosing Learning Rates and Epochs in SFT
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Supervised fine-tuning requires conservative learning rates ( to smaller than pre-training) and low epoch counts (1 to 3 epochs) with cosine decay to adapt conversational formatting without destroying pre-trained knowledge representations.
📌 Key Takeaways
- •Learning rate range: typically for full fine-tuning, and for parameter-efficient LoRA; an order of magnitude smaller than pre-training ()
- •Epoch discipline: 1 to 3 epochs maximum; training beyond 3 epochs almost universally causes memorization, repetition, and catastrophic forgetting
- •Warmup and decay: 3-5% linear warmup followed by cosine decay down to of peak learning rate
📐 Mathematical Derivations
Mathematical Mechanism: 1. Gradient Scale and Optimization Landscape: In pre-training, data is vast and diverse ( tokens); stochastic gradient noise acts as an implicit regularizer. In SFT, data is small ( tokens) and highly curated. The gradient updates are concentrated in a tiny subspace: If learning rate is large, the parameter vector jumps out of the pre-trained loss basin: , destroying pre-trained feature detectors. 2. Overfitting & Memorization Dynamics: Validation loss on held-out SFT data typically reaches its minimum after 1.5 to 2.5 epochs. Beyond 3 epochs, training loss continues to plunge while validation cross-entropy explodes, indicating that the model is no longer learning general instruction schemas but is memorizing specific benchmark responses verbatim.
🏭 Production Trade-offs
Deep Dive & Engineering Trade-offs: ① Full Fine-Tuning vs LoRA Learning Rates: LoRA updates only a tiny fraction of parameters (); consequently, it requires a to higher learning rate () to move the adapter parameters effectively compared to full fine-tuning (). ② Batch Size Selection: SFT typically uses moderate effective batch sizes: sequences (achieved via gradient accumulation). Very large batch sizes () reduce stochasticity and lead to poor generalization on conversational edge cases. ③ Weight Decay Tuning: Set weight decay conservatively (); aggressive weight decay pulls weights toward zero, interfering with pre-trained parameter norms. ④ Early Stopping by Validation PPL: Monitor validation loss on a held-out set of 1,000 diverse multi-turn dialogues; stop training at the very first inflection point of validation perplexity. ⑤ Interview Strategy: Contrast pre-training vs SFT learning rates with numerical orders of magnitude, explain why 1-3 epochs is the empirical ceiling, and distinguish between full fine-tuning and LoRA learning rate dynamics.
⚠️ Common Interview Pitfalls
- ✕Using pre-training learning rates () for full SFT (causes immediate gradient blowup and perplexity destruction)
- ✕Training SFT for 5 to 10 epochs on small datasets (causes severe verbatim memorization and repetitive generation)
- ✕Omitting learning rate warmup, which destabilizes attention layers on the first few batches
🎯 Interviewer Follow-up Inquiries
- ?Why does LoRA require a significantly higher learning rate than full fine-tuning during SFT?
- ?How do you use validation loss inflection to prevent overfitting during instruction tuning?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚