M5-024M5: NLP & Large Language ModelsInstruction Tuning & Supervised Fine-TuningHard
⚡Flashcard Drill➔Mastery:
SFT Data Mixture and Multi-Task Balancing
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
SFT multi-task balancing allocates data proportions across dialogue, coding, mathematics, reasoning, and safety to prevent over-fitting to single modalities while maintaining generalist conversational capabilities.
📌 Key Takeaways
- •Core objective: prevent task competition and cross-domain interference while maximizing general capabilities across diverse user intents
- •Typical industrial mixture: general chat (30-40%), coding and technical tasks (20-30%), mathematical reasoning (15-20%), structured extraction/tools (10-15%), safety & refusal (5%)
- •Temperature-based rebalancing: prevents dominant high-volume datasets from drowning out small, critical datasets (like safety refusals or multi-turn tool calling)
📐 Mathematical Derivations
Mathematical Mechanism: 1. Multi-Task SFT Loss: Given distinct task datasets with dataset sizes : 2. Temperature Sampling Rebalancing: If data is sampled proportional to raw size , a massive chat dataset ( examples) completely starves a small safety dataset ( examples). Temperature rebalancing scales sampling probability by temperature : - When : sampling is proportional to raw dataset size. - When : sampling is completely uniform across all tasks (). - Standard practice uses to boost the presence of high-value, low-volume datasets without extreme duplication.
🏭 Production Trade-offs
Deep Dive & Engineering Trade-offs: ① Task Conflict Dynamics: Certain tasks exhibit negative gradient transfer: aggressive safety training degrades coding and math performance (causing false refusal of benign code exploits); aggressive short-answer training degrades detailed multi-step reasoning. Domain weights must be tuned to keep refusal rates bounded. ② Maximum Sample Capping: A simple, robust alternative to temperature sampling is hard capping: set a maximum threshold samples per task category, ensuring no single dataset dominates the training run. ③ Role of System Prompts: Prepend task-specific system prompts (e.g., 'You are an expert mathematician', 'You are a helpful coding assistant') to help the Transformer's attention heads condition on distinct behavioral modes, minimizing cross-task interference. ④ Multi-Turn vs Single-Turn Balancing: Maintain at least multi-turn conversational data; models trained exclusively on single-turn Q&A fail to track context across extended dialogues. ⑤ Interview Strategy: Present a realistic industrial data mixture breakdown, derive the temperature sampling formula , and discuss how system prompts and sample caps mitigate negative task transfer.
⚠️ Common Interview Pitfalls
- ✕Training on raw dataset proportions where a large chat dataset starves critical small datasets like tool-calling or safety
- ✕Over-indexing on safety data, leading to severe false-refusal rates on programming and security queries
- ✕Omitting multi-turn conversation data during SFT (leads to context amnesia in chat applications)
🎯 Interviewer Follow-up Inquiries
- ?How does temperature parameter balance sample diversity against overfitting on small datasets in multi-task SFT?
- ?What causes negative transfer between safety refusal training and code vulnerability analysis?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚