M5-026M5: NLP & Large Language ModelsInstruction Tuning & Supervised Fine-TuningMedium
⚡Flashcard Drill➔Mastery:

Choosing Learning Rates and Epochs in SFT

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M5-027 →
📐 Mathematical Definition
ηSFT≈10−5∼2×10−5≪ηpretrain;epochs=1∼3\eta_{\text{SFT}}\approx10^{-5}\sim2\times10^{-5}\ll\eta_{\text{pretrain}};\qquad \text{epochs}=1\sim3
⚡ One-Sentence Core Answer
Supervised fine-tuning requires conservative learning rates (10×10\times to 50×50\times smaller than pre-training) and low epoch counts (1 to 3 epochs) with cosine decay to adapt conversational formatting without destroying pre-trained knowledge representations.

📌 Key Takeaways

  • •
    Learning rate range: typically 1–2×10−51\text{--}2 \times 10^{-5} for full fine-tuning, and 1–2×10−41\text{--}2 \times 10^{-4} for parameter-efficient LoRA; an order of magnitude smaller than pre-training (3×10−43\times 10^{-4})
  • •
    Epoch discipline: 1 to 3 epochs maximum; training beyond 3 epochs almost universally causes memorization, repetition, and catastrophic forgetting
  • •
    Warmup and decay: 3-5% linear warmup followed by cosine decay down to 10%10\% of peak learning rate

📐 Mathematical Derivations

Mathematical Mechanism: 1. Gradient Scale and Optimization Landscape: In pre-training, data is vast and diverse (101210^{12} tokens); stochastic gradient noise acts as an implicit regularizer. In SFT, data is small (107–10810^7\text{--}10^8 tokens) and highly curated. The gradient updates are concentrated in a tiny subspace: Δθ=−η∑t=1T∇θLSFT(θt)\Delta \theta = -\eta \sum_{t=1}^T \nabla_\theta \mathcal{L}_{\text{SFT}}(\theta_t) If learning rate η\eta is large, the parameter vector jumps out of the pre-trained loss basin: ∥θ−θpre∥2≫0\|\theta - \theta_{\text{pre}}\|_2 \gg 0, destroying pre-trained feature detectors. 2. Overfitting & Memorization Dynamics: Validation loss on held-out SFT data typically reaches its minimum after 1.5 to 2.5 epochs. Beyond 3 epochs, training loss continues to plunge while validation cross-entropy explodes, indicating that the model is no longer learning general instruction schemas but is memorizing specific benchmark responses verbatim.

🏭 Production Trade-offs

Deep Dive & Engineering Trade-offs: ① Full Fine-Tuning vs LoRA Learning Rates: LoRA updates only a tiny fraction of parameters (<1%<1\%); consequently, it requires a 5×5\times to 10×10\times higher learning rate (10−410^{-4}) to move the adapter parameters effectively compared to full fine-tuning (10−510^{-5}). ② Batch Size Selection: SFT typically uses moderate effective batch sizes: B∈[32,128]B \in [32, 128] sequences (achieved via gradient accumulation). Very large batch sizes (>512>512) reduce stochasticity and lead to poor generalization on conversational edge cases. ③ Weight Decay Tuning: Set weight decay conservatively (0.01–0.10.01\text{--}0.1); aggressive weight decay pulls weights toward zero, interfering with pre-trained parameter norms. ④ Early Stopping by Validation PPL: Monitor validation loss on a held-out set of 1,000 diverse multi-turn dialogues; stop training at the very first inflection point of validation perplexity. ⑤ Interview Strategy: Contrast pre-training vs SFT learning rates with numerical orders of magnitude, explain why 1-3 epochs is the empirical ceiling, and distinguish between full fine-tuning and LoRA learning rate dynamics.
⚠️ Common Interview Pitfalls
  • ✕
    Using pre-training learning rates (3×10−43 \times 10^{-4}) for full SFT (causes immediate gradient blowup and perplexity destruction)
  • ✕
    Training SFT for 5 to 10 epochs on small datasets (causes severe verbatim memorization and repetitive generation)
  • ✕
    Omitting learning rate warmup, which destabilizes attention layers on the first few batches
🎯 Interviewer Follow-up Inquiries
  • ?
    Why does LoRA require a significantly higher learning rate than full fine-tuning during SFT?
  • ?
    How do you use validation loss inflection to prevent overfitting during instruction tuning?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM5-025: Division of Labor Between SFT and RLHF/DPO📋 Back to BankNext →M5-027: Chat Template Formatting and Multi-Turn Conversation Construction