Supervised Fine-Tuning (SFT)

SFT (Supervised Fine-Tuning) fine-tunes the pretrained model on high-quality instruction–response pairs to turn a text-continuation engine into an instruction follower. Mechanism: the objective stays cross-entropy $L = -\sum \log P(y|x)$ but on instruction data; typically 1–3 epochs at a low LR (~1e-5, far below pretraining) with global-batch packing. The core lesson is 'quality beats quantity': Alpaca replicated ChatGPT-level instruction following with only 52K GPT-generated samples.

giving general models conversational/instruction-following ability, domain adaptation (medical/legal/finance), and as the initial policy for RLHF.

pretrained models only continue text and ignore instructions. A few thousand to tens of thousands of high-quality pairs align behavior — far better than blindly scaling data; more than 3 epochs overfits instructions and hurts general ability (catastrophic forgetting). Compared with ~1.4T pretraining tokens, SFT uses ~7 orders of magnitude less data (52K vs 1.4T), slashing cost, while providing the starting policy for RLHF.

Test Your Knowledge: Practice Questions for "Supervised Fine-Tuning (SFT)"

Single choice pitfall questions with instant feedback and mistake tracking.