05. Training, Post-Training & Alignment
NVIDIA
DeepSpeed
Meta (Llama)
Google DeepMind
OpenAI
Anthropic
Hugging Face (PEFT)
DatabricksMeta (Llama)Hugging FaceMeta (Llama)Training raw base models into reasoning-dense, instruction-following, human-aligned, and verifiable agents.
5.1 Large-Scale Training Systems
3D Parallelism (TP/PP/DP), Context Parallelism (CP), PyTorch FSDP, DeepSpeed ZeRO memory sharding, long-horizon fault tolerance, and async checkpoint recovery.
3D Parallelism (TP/PP/DP), FSDP & ZeRO Memory Sharding
Megatron-LM TP/PP communication scheduling, Context/Sequence Parallelism (CP/SP) for long context, PyTorch FSDP, DeepSpeed ZeRO-1/2/3 parameter/optimizer sharding, and activation checkpointing memory trade-offs.
NVIDIADeepSpeedMeta (Llama)Google DeepMindOpenAITraining Fault Tolerance, Checkpointing & Observability
10K+ GPU async checkpoint persistence and rapid recovery, straggler and silent hang detection, FP8/BF16 mixed-precision numerical stability, automated loss spike remediation, and W&B telemetry.
NVIDIAMeta (Llama)5.2 Supervised Fine-Tuning & Adaptation
LoRA/QLoRA/DoRA low-rank adaptation, loss masking, multi-task data mixtures, curriculum learning, and offline quality regression gates.
LoRA, QLoRA, DoRA & High-Throughput Fine-Tuning
LoRA low-rank decomposition math ($W = W_0 + \frac{\alpha}{r}BA$), QLoRA NormalFloat4 (NF4) quantized fine-tuning, DoRA magnitude-direction decoupling, loss masking, and Unsloth/PEFT fast kernels.
Hugging Face (PEFT)DatabricksInstruction Mixture, Curriculum Tuning & Regression Gates
Multi-domain instruction weighting, difficulty-graded curriculum fine-tuning, catastrophic forgetting mitigation, chat template format safety, and automated pre-deployment regression gates.
Hugging FaceDatabricks5.3 Preference Optimization, Alignment & Reasoning
DPO/IPO/SimPO offline preference optimization, Constitutional AI, PPO online RL, DeepSeek-R1 GRPO group relative sampling, and rule-based verifiable rewards (RLVR).
DPO / IPO Offline Preference Optimization & Data Curation
DPO closed-form optimization bypassing separate reward models, IPO/KTO/SimPO variants, Constitutional AI, preference dataset curation, and out-of-distribution (OOD) degeneration mitigation.
AnthropicOpenAIMeta (Llama)Google DeepMindPPO Online RL, DeepSeek GRPO & Rule-Verifiable Rewards (RLVR)
PPO online policy gradient with Actor-Critic/Reward models, DeepSeek-R1 GRPO group relative reward sampling (bypassing separate Critic networks), rule-based verifiable rewards (RLVR) for math/code reasoning, and Process Reward Models (PRM).
OpenAIAnthropicGoogle DeepMind