Roadmap

LLMs Mind Map

48 knowledge points · click a node for its structured card

Same color = same guide topic. Hover a node for its dependency chain. Drag and zoom canvas. Export PNG or go fullscreen from the top-right.

Pre-Training Data EngineeringTokenization BPE/WordPieceSpeculative Decoding (cross-module)Attention & KVArchitecture BlocksSupervised Fine-TuningRLHF 3-Stage PipelineQuantization PTQ/QATMoE Routing Top-kChain-of-ThoughtHallucination Types & CausesAgent & Tool Calling (cross-module)Auto Metrics vs LLM-as-JudgeContext Engineering (cross-module)Pre-Training Data EngineeringPre-Training Data EngineeringClick for definition & exam pointsObjectives & Optimizer DetailsObjectives & Optimizer DetailsClick for definition & exam pointsLong-Context Eval Needle/Lost-in-MiddleLong-Context Eval Needle/Lost-in-MiddleClick for definition & exam pointsBenchmarks MMLU/GSM8KBenchmarks MMLU/GSM8KClick for definition & exam pointsScaling LawsScaling LawsClick for definition & exam pointsModel Families & TrendsModel Families & TrendsClick for definition & exam pointsLong-Context ExtensionLong-Context ExtensionClick for definition & exam pointsTokenization BPE/WordPieceTokenization BPE/WordPieceClick for definition & exam pointsText EmbeddingsText EmbeddingsClick for definition & exam pointsPadding & PackingPadding & PackingClick for definition & exam pointsDecoding StrategiesDecoding StrategiesClick for definition & exam pointsSpeculative Decoding (cross-module)Speculative Decoding (cross-module)Click for definition & exam pointsKV Cache Management (cross-module)KV Cache Management (cross-module)Click for definition & exam pointsScaled Dot-Product AttentionScaled Dot-Product AttentionClick for definition & exam pointsAttention Variants MHA/MQA/GQAAttention Variants MHA/MQA/GQAClick for definition & exam pointsMLA Low-Rank KV CompressionMLA Low-Rank KV CompressionClick for definition & exam pointsFlashAttentionFlashAttentionClick for definition & exam pointsSSM & Hybrid ArchitecturesSSM & Hybrid ArchitecturesClick for definition & exam pointsPositional Encoding RoPE/ALiBiPositional Encoding RoPE/ALiBiClick for definition & exam pointsRoPE Extrapolation PI/NTK/YaRNRoPE Extrapolation PI/NTK/YaRNClick for definition & exam pointsFFN & SwiGLUFFN & SwiGLUClick for definition & exam pointsPre-LN & RMSNormPre-LN & RMSNormClick for definition & exam pointsSupervised Fine-TuningSupervised Fine-TuningClick for definition & exam pointsLoRA / QLoRALoRA / QLoRAClick for definition & exam pointsSoft Prompts / AdaptersSoft Prompts / AdaptersClick for definition & exam pointsRLHF 3-Stage PipelineRLHF 3-Stage PipelineClick for definition & exam pointsPPO Clipped ObjectivePPO Clipped ObjectiveClick for definition & exam pointsDirect Preference OptimizationDirect Preference OptimizationClick for definition & exam pointsReference-Free Alignment SimPO/ORPO/KTOReference-Free Alignment SimPO/ORPO/KTOClick for definition & exam pointsGRPO Group Relative POGRPO Group Relative POClick for definition & exam pointsAlignment Tax & Preference DataAlignment Tax & Preference DataClick for definition & exam pointsQuantization PTQ/QATQuantization PTQ/QATClick for definition & exam pointsLow-Bit INT8/INT4Low-Bit INT8/INT4Click for definition & exam pointsSmoothQuant/AWQ & FP8SmoothQuant/AWQ & FP8Click for definition & exam pointsKnowledge DistillationKnowledge DistillationClick for definition & exam pointsMoE Routing Top-kMoE Routing Top-kClick for definition & exam pointsMoE Load BalancingMoE Load BalancingClick for definition & exam pointsDense vs MoEDense vs MoEClick for definition & exam pointsChain-of-ThoughtChain-of-ThoughtClick for definition & exam pointsTest-Time Compute ScalingTest-Time Compute ScalingClick for definition & exam pointsMCTS Reasoning SearchMCTS Reasoning SearchClick for definition & exam pointsPRM & RLVR (cross-module)PRM & RLVR (cross-module)Click for definition & exam pointsHallucination Types & CausesHallucination Types & CausesClick for definition & exam pointsHallucination MitigationHallucination MitigationClick for definition & exam pointsAgent & Tool Calling (cross-module)Agent & Tool Calling (cross-module)Click for definition & exam pointsAuto Metrics vs LLM-as-JudgeAuto Metrics vs LLM-as-JudgeClick for definition & exam pointsClassic NLP TasksClassic NLP TasksClick for definition & exam pointsContext Engineering (cross-module)Context Engineering (cross-module)Click for definition & exam points
100%
🖱️ Drag to pan · Hover for dependency chain · Click for card
Pre-Training Data EngineeringTokenization BPE/WordPieceSpeculative Decoding (cross-module)KV Cache Management (cross-module)Scaled Dot-Product AttentionSupervised Fine-TuningRLHF 3-Stage PipelineQuantization PTQ/QATMoE Routing Top-kChain-of-ThoughtHallucination Types & CausesAgent & Tool Calling (cross-module)Auto Metrics vs LLM-as-JudgeContext Engineering (cross-module)