Roadmap

👁️ Multimodal Mind Map

37 knowledge points · click a node for its structured card

Same color = same guide topic. Hover a node for its dependency chain. Drag and zoom canvas. Export PNG or go fullscreen from the top-right.

CLIP Two-TowerVLM ArchitectureVLM Ability & TrainingDiffusion CoreDiffusion ApplicationsDiffusion Training ObjectivesAudio RepresentationJEPA Joint-Embedding PredictionLLM Backbone (cross-module)Embodied AI (cross-module)Multimodal RAG (cross-module)CLIP Two-TowerCLIP Two-TowerClick for definition & exam pointsInfoNCE Contrastive LossInfoNCE Contrastive LossClick for definition & exam pointsCLIP ApplicationsCLIP ApplicationsClick for definition & exam pointsVLM ArchitectureVLM ArchitectureClick for definition & exam pointsVision EncoderVision EncoderClick for definition & exam pointsVLM Projector MLP/Q-FormerVLM Projector MLP/Q-FormerClick for definition & exam pointsHigh-Res & Dynamic TokensHigh-Res & Dynamic TokensClick for definition & exam pointsNative Any-to-Any MultimodalNative Any-to-Any MultimodalClick for definition & exam pointsInterleaved Training DataInterleaved Training DataClick for definition & exam pointsVLM CapabilitiesVLM CapabilitiesClick for definition & exam pointsMultimodal Reasoning & AgentsMultimodal Reasoning & AgentsClick for definition & exam pointsMultimodal HallucinationMultimodal HallucinationClick for definition & exam pointsMultimodal Eval BenchmarksMultimodal Eval BenchmarksClick for definition & exam pointsVLM 3-Stage PipelineVLM 3-Stage PipelineClick for definition & exam pointsMasked Cross-Entropy LossMasked Cross-Entropy LossClick for definition & exam pointsMultimodal DPO / RLHF-VMultimodal DPO / RLHF-VClick for definition & exam pointsDDPM Forward/ReverseDDPM Forward/ReverseClick for definition & exam pointsClassifier-Free GuidanceClassifier-Free GuidanceClick for definition & exam pointsLatent Diffusion (LDM)Latent Diffusion (LDM)Click for definition & exam pointsDiffusion vs GAN vs ARDiffusion vs GAN vs ARClick for definition & exam pointsSampler AccelerationSampler AccelerationClick for definition & exam pointsVideo GenerationVideo GenerationClick for definition & exam pointsDiT & 3D Causal VAEDiT & 3D Causal VAEClick for definition & exam pointsProgressive Video TrainingProgressive Video TrainingClick for definition & exam pointsDiffusion Training ObjectivesDiffusion Training ObjectivesClick for definition & exam pointsAudio RepresentationAudio RepresentationClick for definition & exam pointsASR (Whisper)ASR (Whisper)Click for definition & exam pointsTTSTTSClick for definition & exam pointsSpeech-Audio LLMSpeech-Audio LLMClick for definition & exam pointsRVQ Codebooks & TokensRVQ Codebooks & TokensClick for definition & exam pointsJEPA Joint-Embedding PredictionJEPA Joint-Embedding PredictionClick for definition & exam pointsWorld Model ApplicationsWorld Model ApplicationsClick for definition & exam pointsLLM Backbone (cross-module)LLM Backbone (cross-module)Click for definition & exam pointsEmbodied AI (cross-module)Embodied AI (cross-module)Click for definition & exam pointsVLA Models (cross-module)VLA Models (cross-module)Click for definition & exam pointsMultimodal RAG (cross-module)Multimodal RAG (cross-module)Click for definition & exam pointsColPali Visual RetrievalColPali Visual RetrievalClick for definition & exam points
100%
🖱️ Drag to pan · Hover for dependency chain · Click for card
CLIP Two-TowerVLM ArchitectureDDPM Forward/ReverseAudio RepresentationJEPA Joint-Embedding PredictionLLM Backbone (cross-module)Embodied AI (cross-module)Multimodal RAG (cross-module)