🔬 Applied Scientist & ML Specialist938 Problems · 127 Topics🌐 Linked to Tech Vault

TalentMe Science Depth: Universal Applied Science & ML Depth Problem Bank

938 end-to-end science depth, system design, transition bridge & BQ problems covering derivations, production trade-offs, pitfalls, and follow-ups.

🎯

Science Depth Mastery Tracker

0% Mastered

Locally synced in your browser. Track, review, and master all 938 scientific & design topics.

Mastered: 0
Reviewing: 0
Unstarted: 938
🚀 Start Next Unpracticed →
🌐
Linked to TalentMe Tech Vault & MindmapsGlobal System
All 938 questions are directly linked to TalentMe's Tech Vault guides, atomic cards, and interactive mindmaps.
🧭Math Foundations Mindmap & Cards→📖Bayesian Inference, Entropy & KL Divergence→🎯DS Track→
Found 24 in-depth questions
M4-100Sequences & TransformersModel Compression & Distillation
Easy

Three Granularities of Model Pruning: Unstructured, Semi-Structured 2:4, and Structured

⚡ One-Sentence Answer

Model pruning removes redundant parameters across three distinct granularities: unstructured pruning maximizes theoretical sparsity but requires sparse kernels, semi-structured 2:4 pruning matches NVIDIA Ampere Sparse Tensor Cores, and structured pruning removes entire channels or heads for immediate hardware acceleration.

Math & Tradeoffs→
M4-101Sequences & TransformersModel Compression & Distillation
Medium

On-Policy vs. Off-Policy Data in Knowledge Distillation for LLMs

⚡ One-Sentence Answer

Off-policy distillation trains the student on static teacher-generated text distributions, while on-policy distillation trains the student on its own self-generated responses evaluated by teacher feedback, eliminating distribution shift during autoregressive deployment.

Math & Tradeoffs→
M4-102Sequences & TransformersModel Compression & Distillation
Medium

Systematic Categorization of Inference Acceleration Pathways and Trade-offs

⚡ One-Sentence Answer

Inference acceleration optimizes across four complementary layers: algorithmic architecture (speculative decoding, GQA), model compression (quantization, pruning, distillation), runtime system engineering (continuous batching, PagedAttention), and hardware kernel fusion (FlashAttention, CUTLASS).

Math & Tradeoffs→
M4-103Sequences & TransformersModel Compression & Distillation
Hard

Quantization-Aware Training (QAT) vs. Post-Training Quantization (PTQ)

⚡ One-Sentence Answer

PTQ quantizes pre-trained weights post-hoc using a small calibration dataset without retraining, whereas QAT simulates quantization noise during training using the Straight-Through Estimator to recover accuracy on aggressive low-bit configurations.

Math & Tradeoffs→
M4-104Sequences & TransformersModel Compression & Distillation
Hard

Sources of Quantization Precision Loss and Mitigation Techniques

⚡ One-Sentence Answer

Quantization error stems from the fundamental conflict between rounding error on normal distributions and clipping error on extreme outliers, mitigated through non-uniform grids, outlier protection, and channel-wise scaling migration.

Math & Tradeoffs→
M4-105Sequences & TransformersModel Compression & Distillation
Hard

Trade-offs Between LoRA, QLoRA, and Full Fine-Tuning

⚡ One-Sentence Answer

Full fine-tuning updates all parameters for maximum performance on foundation models, LoRA freezes the base model and injects trainable low-rank adapters, and QLoRA quantizes the base model to 4-bit NormalFloat with paged optimizers to democratize fine-tuning on consumer GPUs.

Math & Tradeoffs→
M4-106Sequences & TransformersQuantization & Acceleration
Easy

Symmetric vs. Asymmetric Quantization Formulation and Overhead

⚡ One-Sentence Answer

Symmetric quantization restricts the zero-point to zero to enable fast zero-overhead integer Tensor Core multiplications, whereas asymmetric quantization uses an explicit zero-point offset to better fit skewed distributions at the cost of additive runtime correction terms.

Math & Tradeoffs→
M4-107Sequences & TransformersQuantization & Acceleration
Easy

Differences Between NormalFloat4 (NF4) and INT4 in QLoRA

⚡ One-Sentence Answer

NF4 constructs an information-theoretically optimal non-uniform quantile grid for normally distributed pre-trained weights, allocating equal probability mass to each bin and significantly outperforming uniform INT4 quantization.

Math & Tradeoffs→
M4-108Sequences & TransformersQuantization & Acceleration
Medium

Weight-Activation Quantization Trade-offs: W8A8 vs. W4A16

⚡ One-Sentence Answer

W8A8 quantizes both weights and activations to INT8 to achieve maximum compute speedups on Tensor Cores, whereas W4A16 quantizes only weights to 4-bit to maximize memory savings with high accuracy stability while retaining FP16 compute.

Math & Tradeoffs→
M4-109Sequences & TransformersQuantization & Acceleration
Medium

Benefits, Risks, and Outlier Sensitivity in KV Cache Quantization

⚡ One-Sentence Answer

KV cache quantization slashes generation memory by 2x to 4x to expand context lengths and serving concurrency, but requires asymmetric per-channel quantization for Keys and per-token quantization for Values to accommodate distinct outlier orientations.

Math & Tradeoffs→
M4-110Sequences & TransformersQuantization & Acceleration
Hard

Impact of Batching, Kernel Fusion, and Operator Optimization on Serving Throughput

⚡ One-Sentence Answer

Serving throughput is maximized by batching to amortize weight reading over multiple requests, kernel fusion to eliminate redundant HBM memory roundtrips, and operator optimization to maximize hardware Tensor Core utilization.

Math & Tradeoffs→
M4-111Sequences & TransformersQuantization & Acceleration
Hard

Systematic Metric Framework for Evaluating Inference Optimizations: TTFT, TPOT, MFU, and Cost

⚡ One-Sentence Answer

A rigorous inference optimization evaluation framework diagnoses hardware bottlenecks via profiling, verifies end-to-end latency SLAs and Goodput, validates downstream task accuracy, and quantifies unit cost economics.

Math & Tradeoffs→
M4-112Sequences & TransformersQuantization & Acceleration
Hard

Comparison of Post-Training Quantization Algorithms: GPTQ, AWQ, and SmoothQuant

⚡ One-Sentence Answer

GPTQ minimizes layer-wise quantization error via second-order inverse Hessian compensation, AWQ protects salient weight channels based on activation magnitudes, and SmoothQuant scales activations down to migrate quantization difficulty onto weights for W8A8 deployment.

Math & Tradeoffs→
M5-001NLP & Large Language ModelsTokenization (BPE / WordPiece / Unigram)
Easy

BPE Algorithm and Why It Is Widely Used in LLMs

⚡ One-Sentence Answer

Byte-Pair Encoding (BPE) iteratively merges the most frequent pairs of adjacent characters or subwords to construct a fixed-size subword vocabulary, eliminating out-of-vocabulary words while balancing sequence length and vocabulary size.

Math & Tradeoffs→
M5-002NLP & Large Language ModelsTokenization (BPE / WordPiece / Unigram)
Easy

Comparison of BPE, WordPiece, and Unigram Tokenization

⚡ One-Sentence Answer

BPE merges subwords based on raw co-occurrence frequency, WordPiece merges subwords based on mutual information (maximum likelihood), and Unigram prunes a large initial candidate pool based on sentence-level likelihood loss.

Math & Tradeoffs→
M5-003NLP & Large Language ModelsTokenization (BPE / WordPiece / Unigram)
Medium

Vocabulary Size Trade-offs in LLMs

⚡ One-Sentence Answer

Larger vocabularies compress text into fewer tokens to accelerate inference and expand the effective context window, but increase embedding parameter size and output softmax latency.

Math & Tradeoffs→
M5-004NLP & Large Language ModelsTokenization (BPE / WordPiece / Unigram)
Medium

Impact of Tokenizer on Downstream Metrics and Why Cross-Model Perplexity (PPL) Comparisons Are Unfair

⚡ One-Sentence Answer

Perplexity is normalized per token, making raw cross-model PPL comparisons invalid across different tokenizers because models with smaller vocabularies split text into more, easily predictable subwords that artificially deflate token-level loss.

Math & Tradeoffs→
M5-005NLP & Large Language ModelsTokenization (BPE / WordPiece / Unigram)
Hard

Tokenizer Challenges in Multilingual and Code Domains

⚡ One-Sentence Answer

Multilingual and code tokenization suffer from script fertility imbalances, high UTF-8 byte fragmentation, and loss of indentation and whitespace structure, requiring byte-fallback and dedicated syntactic pre-tokenization.

Math & Tradeoffs→
M5-006NLP & Large Language ModelsTokenization (BPE / WordPiece / Unigram)
Hard

Vocabulary Expansion Techniques and Considerations

⚡ One-Sentence Answer

Vocabulary expansion adds domain- or language-specific subwords to a pre-trained model by resizing the embedding and LM head matrices, requiring careful weight initialization and targeted continual pre-training to prevent catastrophic forgetting.

Math & Tradeoffs→
M5-007NLP & Large Language ModelsPretraining Objectives & Data Curation
Easy

Causal Language Modeling (CLM) vs. Masked Language Modeling (MLM)

⚡ One-Sentence Answer

CLM trains autoregressive decoders to predict the next token via unidirectional causal masking for text generation, whereas MLM trains bidirectional encoders to predict randomly masked tokens for discriminative representation learning.

Math & Tradeoffs→
M5-008NLP & Large Language ModelsPretraining Objectives & Data Curation
Easy

Impact of Data Quality and Mixture Ratios on Pre-training

⚡ One-Sentence Answer

Pre-training quality is dominated by aggressive data filtering, semantic deduplication, and deliberate domain mixture ratios, where small amounts of high-quality synthetic, mathematical, and code data disproportionately drive reasoning capabilities.

Math & Tradeoffs→
M5-009NLP & Large Language ModelsPretraining Objectives & Data Curation
Medium

Data Deduplication Methods and Why It Matters in LLM Pre-training

⚡ One-Sentence Answer

Data deduplication removes verbatim and near-duplicate text via exact hash matching and MinHash LSH, preventing model memorization, privacy leakage, and inefficient compute waste.

Math & Tradeoffs→
M5-010NLP & Large Language ModelsPretraining Objectives & Data Curation
Medium

Curriculum Learning and Data Ordering in LLM Pre-training

⚡ One-Sentence Answer

Curriculum learning structures pre-training by ordering data from broad, easy-to-learn web text to dense, high-quality code and academic reasoning, stabilizing early optimization and accelerating convergence.

Math & Tradeoffs→
M5-011NLP & Large Language ModelsPretraining Objectives & Data Curation
Hard

Synthetic Data in Pre-training and Post-Training: Roles and Risks

⚡ One-Sentence Answer

Synthetic data expands training volume in data-scarce domains like math and code via model generation and automated verification, but carries severe risks of model collapse, diversity loss, and error amplification if unvalidated.

Math & Tradeoffs→