⚡ LLMsID: sft-basics

Supervised Fine-Tuning (SFT)

Q: SFT vs pretraining: data, objective, epochs?

【Supervised Fine-Tuning (SFT) 核心考点 1】SFT vs pretraining: data, objective, epochs?。技术解析：Definition: SFT (Supervised Fine-Tuning) fine-tunes the pretrained model on high-quality instruction–response pairs to turn a text-continuation engine into an instruction follower. Mechanism: the objective stays cross-entropy L = -\sum \log P(y|x) but on instruction data; typical... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/sft-basics

Q: Why does quality beat quantity in SFT data?

【Supervised Fine-Tuning (SFT) 核心考点 2】Why does quality beat quantity in SFT data?。技术解析：Definition: SFT (Supervised Fine-Tuning) fine-tunes the pretrained model on high-quality instruction–response pairs to turn a text-continuation engine into an instruction follower. Mechanism: the objective stays cross-entropy L = -\sum \log P(y|x) but on instruction data; typical... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/sft-basics

Q: Can SFT overfit? How to choose epochs and LR?

【Supervised Fine-Tuning (SFT) 核心考点 3】Can SFT overfit? How to choose epochs and LR?。技术解析：Definition: SFT (Supervised Fine-Tuning) fine-tunes the pretrained model on high-quality instruction–response pairs to turn a text-continuation engine into an instruction follower. Mechanism: the objective stays cross-entropy L = -\sum \log P(y|x) but on instruction data; typical... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/sft-basics

Q: What is the relationship between SFT and RLHF/DPO?

【Supervised Fine-Tuning (SFT) 核心考点 4】What is the relationship between SFT and RLHF/DPO?。技术解析：Definition: SFT (Supervised Fine-Tuning) fine-tunes the pretrained model on high-quality instruction–response pairs to turn a text-continuation engine into an instruction follower. Mechanism: the objective stays cross-entropy L = -\sum \log P(y|x) but on instruction data; typical... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/sft-basics

Q: How are instruction datasets built and filtered (human/LLM-generated)?

【Supervised Fine-Tuning (SFT) 核心考点 5】How are instruction datasets built and filtered (human/LLM-generated)?。技术解析：Definition: SFT (Supervised Fine-Tuning) fine-tunes the pretrained model on high-quality instruction–response pairs to turn a text-continuation engine into an instruction follower. Mechanism: the objective stays cross-entropy L = -\sum \log P(y|x) but on instruction data; typical... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/sft-basics

指令微调 SFT

🎯Core Definition

SFT (Supervised Fine-Tuning) fine-tunes the pretrained model on high-quality instruction–response pairs to turn a text-continuation engine into an instruction follower. Mechanism: the objective stays cross-entropy

L = -\sum \log P(y|x)

but on instruction data; typically 1–3 epochs at a low LR (~1e-5, far below pretraining) with global-batch packing. The core lesson is 'quality beats quantity': Alpaca replicated ChatGPT-level instruction following with only 52K GPT-generated samples.

💡Use Cases

giving general models conversational/instruction-following ability, domain adaptation (medical/legal/finance), and as the initial policy for RLHF.

⚡Key Problems Solved

pretrained models only continue text and ignore instructions. A few thousand to tens of thousands of high-quality pairs align behavior — far better than blindly scaling data; more than 3 epochs overfits instructions and hurts general ability (catastrophic forgetting). Compared with ~1.4T pretraining tokens, SFT uses ~7 orders of magnitude less data (52K vs 1.4T), slashing cost, while providing the starting policy for RLHF.

🎯Core Interview Exam Points

Fundamentals · Production Scenarios · Deep Dive

Core Def

Explain the core definition, underlying mechanism/math, and key principles of Supervised Fine-Tuning (SFT).

Use Cases

In production engineering, what are the primary use cases for Supervised Fine-Tuning (SFT) and what core pain points does it solve?

🔥 High-Frequency Deep-Dive Questions

SFT vs pretraining: data, objective, epochs?

Why does quality beat quantity in SFT data?

Can SFT overfit? How to choose epochs and LR?

What is the relationship between SFT and RLHF/DPO?

How are instruction datasets built and filtered (human/LLM-generated)?