🔬 Applied Scientist & ML Specialist938 Problems · 127 Topics🌐 Linked to Tech Vault

TalentMe Science Depth: Universal Applied Science & ML Depth Problem Bank

938 end-to-end science depth, system design, transition bridge & BQ problems covering derivations, production trade-offs, pitfalls, and follow-ups.

🎯

Science Depth Mastery Tracker

0% Mastered

Locally synced in your browser. Track, review, and master all 938 scientific & design topics.

Mastered: 0
Reviewing: 0
Unstarted: 136
🚀 Start Next Unpracticed →
🌐
Linked to TalentMe Tech Vault & MindmapsM5: NLP & Large Language Models
All 938 questions are directly linked to TalentMe's Tech Vault guides, atomic cards, and interactive mindmaps.
🧭LLM & Agent Pipelines Mindmap→📖RLHF, DPO & GRPO Reasoning→🎯AIE Fine-Tuning→
Found 24 in-depth questions
M5-049NLP & Large Language ModelsDPO Family (DPO / KTO / ORPO)
Hard

Robustness to Noisy Preferences: cDPO and RPO

⚡ One-Sentence Answer

cDPO incorporates label smoothing to account for human annotator error, while RPO adds an explicit negative log-ratio penalty to prevent extreme margin divergence and likelihood collapse.

Math & Tradeoffs→
M5-050NLP & Large Language ModelsDPO Family (DPO / KTO / ORPO)
Hard

DPO Implicit Reward and Its Use at Inference Time

⚡ One-Sentence Answer

A DPO-trained policy mathematically defines an implicit reward function $r(x, y) = \beta \log \frac{\pi_\theta(y|x)}{\pi_{\text{ref}}(y|x)}$, which can be evaluated at inference time for Best-of-$N$ re-ranking and quality filtering without training an external reward model.

Math & Tradeoffs→
M5-051NLP & Large Language ModelsDPO Family (DPO / KTO / ORPO)
Hard

Online / Iterative DPO Practices and Benefits

⚡ One-Sentence Answer

Online (Iterative) DPO closes the distribution gap of offline alignment by sampling completions dynamically from the current policy, scoring them with an automated judge, and updating the policy across rolling rounds.

Math & Tradeoffs→
M5-052NLP & Large Language ModelsDPO Family (DPO / KTO / ORPO)
Hard

Length Bias in DPO and Length Control Methods

⚡ One-Sentence Answer

DPO naturally suffers from length bias because cumulative log-probabilities grow with response length, requiring length-normalized rewards, margin penalties, or length-balanced pair filtering to prevent verbosity hacking.

Math & Tradeoffs→
M5-053NLP & Large Language ModelsDPO Family (DPO / KTO / ORPO)
Medium

SFT and DPO Hybrid Training Strategies

⚡ One-Sentence Answer

Co-training DPO with an auxiliary SFT loss over winning completions prevents likelihood displacement and language degeneration by anchoring the policy to high generative probability on positive demonstrations.

Math & Tradeoffs→
M5-054NLP & Large Language ModelsGRPO & Reasoning Models
Easy

Mathematical Formulation of GRPO Advantage Estimation and Loss

⚡ One-Sentence Answer

Group Relative Policy Optimization (GRPO) samples a group of candidate outputs for each prompt and estimates advantage using normalized intra-group reward z-scores, eliminating the Critic value network and slashing training VRAM.

Math & Tradeoffs→
M5-055NLP & Large Language ModelsGRPO & Reasoning Models
Easy

Reinforcement Learning with Verifiable Rewards (RLVR) and Its Advantages

⚡ One-Sentence Answer

RLVR trains models using deterministic binary or programmatic verifiers in formal domains like math and coding, completely eliminating reward model exploitation and enabling self-reinforcing test-time reasoning scaling.

Math & Tradeoffs→
M5-056NLP & Large Language ModelsGRPO & Reasoning Models
Medium

GSPO (Group Sequence Preference Optimization) Improvement Motivations

⚡ One-Sentence Answer

GSPO extends group relative optimization by evaluating preference margins at the full sequence trajectory level rather than decomposing into noisy per-token advantages, stabilizing policy updates over long reasoning chains.

Math & Tradeoffs→
M5-057NLP & Large Language ModelsGRPO & Reasoning Models
Medium

DAPO (Decoupled Advantage Policy Optimization) Improvement Highlights

⚡ One-Sentence Answer

DAPO decouples the optimization dynamics of positive and negative advantages, applying asymmetric clipping bounds and learning rates to prevent negative gradient updates from collapsing policy entropy.

Math & Tradeoffs→
M5-058NLP & Large Language ModelsGRPO & Reasoning Models
Hard

Reasoning Model Training Paradigm: Long CoT and Reinforcement Learning

⚡ One-Sentence Answer

Modern reasoning models are trained by pairing high-capacity base models with reinforcement learning over verifiable rewards, autonomously incentivizing extended Chain-of-Thought generation, self-correction, and systematic hypothesis testing.

Math & Tradeoffs→
M5-059NLP & Large Language ModelsGRPO & Reasoning Models
Hard

Evaluation and Cost Trade-offs of Reasoning Models

⚡ One-Sentence Answer

Reasoning models trade substantial inference compute and user latency for higher accuracy, requiring difficulty-aware dynamic routing and Goodput-oriented benchmarking to balance cost against performance.

Math & Tradeoffs→
M5-060NLP & Large Language ModelsGRPO & Reasoning Models
Hard

Group Size and Sampling Efficiency in GRPO

⚡ One-Sentence Answer

GRPO group size $G$ balances statistical advantage estimation accuracy against rollout generation compute, with $G \in [8, 16]$ providing the optimal trade-off for variance reduction and GPU scheduling efficiency.

Math & Tradeoffs→
M5-061NLP & Large Language ModelsGRPO & Reasoning Models
Hard

Cold Start Problem in Zero-RL / RLVR and Mitigation Strategies

⚡ One-Sentence Answer

The cold start problem occurs when a base model has a near-zero success rate on hard reasoning tasks, producing zero positive rewards and stalling RLVR exploration, solved through curated CoT seed demonstrations, prompt scaffolding, and difficulty curricula.

Math & Tradeoffs→
M5-062NLP & Large Language ModelsGRPO & Reasoning Models
Hard

Length Control and Length Penalties in Long CoT Reasoning

⚡ One-Sentence Answer

Long CoT models naturally drift toward extreme verbosity and repetitive looping during RL, requiring length penalties, repetition filters, and format constraints to ensure reasoning tokens reflect genuine cognitive work.

Math & Tradeoffs→
M5-063NLP & Large Language ModelsGRPO & Reasoning Models
Hard

Reasoning Model Distillation: Transferring Long CoT into Compact Models

⚡ One-Sentence Answer

Distilling reasoning capabilities by fine-tuning compact models on curated reasoning traces from frontier models is vastly more compute-efficient than training small models via RL from scratch, though bounded by the teacher's capability ceiling.

Math & Tradeoffs→
M5-064NLP & Large Language ModelsGRPO & Reasoning Models
Hard

Process Reward Models (PRM) vs. Outcome Reward Models (ORM): Usage and Annotation Cost

⚡ One-Sentence Answer

Outcome Reward Models (ORM) evaluate only final answer correctness with minimal annotation expense, whereas Process Reward Models (PRM) score every intermediate step to provide fine-grained credit assignment at the cost of massive human labeling overhead.

Math & Tradeoffs→
M5-065NLP & Large Language ModelsGRPO & Reasoning Models
Hard

Efficiency Challenges in Reasoning Models: Overthinking and Length Adaptation

⚡ One-Sentence Answer

Reasoning models frequently suffer from overthinking—spending thousands of reasoning tokens on trivial queries—requiring difficulty-adaptive budget allocation and early-exit mechanisms to control operational costs and prevent reasoning degradation.

Math & Tradeoffs→
M5-066NLP & Large Language ModelsPrompting & Reasoning Techniques
Easy

Why Chain-of-Thought (CoT) Works: A Computational Perspective

⚡ One-Sentence Answer

Chain-of-Thought works because it transforms serial computational depth from fixed network layers into autoregressive sequence length, providing dynamic external working memory that breaks the constant-depth expressivity limits of Transformers.

Math & Tradeoffs→
M5-067NLP & Large Language ModelsPrompting & Reasoning Techniques
Easy

Self-Consistency and Majority Voting in LLM Reasoning

⚡ One-Sentence Answer

Self-Consistency samples multiple diverse Chain-of-Thought reasoning paths at elevated temperatures and selects the final answer via majority voting, exploiting the principle that valid reasoning paths converge on identical answers while error paths disperse randomly.

Math & Tradeoffs→
M5-068NLP & Large Language ModelsPrompting & Reasoning Techniques
Medium

Tree-of-Thought (ToT) vs. ReAct: Internal Deliberation vs. External Interaction

⚡ One-Sentence Answer

Tree-of-Thought conducts deliberate internal tree search with backtracking and state evaluation without external tools, whereas ReAct interleaves internal reasoning with external tool actions and environmental observations.

Math & Tradeoffs→
M5-069NLP & Large Language ModelsPrompting & Reasoning Techniques
Medium

Few-Shot Example Selection and Ordering Effects in In-Context Learning

⚡ One-Sentence Answer

The performance of few-shot in-context learning is highly sensitive to example selection and order, with permutation changes causing up to 30% accuracy swings due to recency bias and majority class imbalance.

Math & Tradeoffs→
M5-070NLP & Large Language ModelsPrompting & Reasoning Techniques
Hard

Prompt Brittleness and Robustness in Large Language Models

⚡ One-Sentence Answer

Prompt brittleness describes sharp performance drops caused by trivial textual perturbations such as whitespace changes or synonym swaps, mitigated through prompt calibration, multi-prompt ensembling, and instruction fine-tuning.

Math & Tradeoffs→
M5-071NLP & Large Language ModelsPrompting & Reasoning Techniques
Hard

Self-Reflection and Self-Critique Mechanisms and Limitations in LLMs

⚡ One-Sentence Answer

Self-reflection improves multi-turn task success by prompting models to critique and refine past actions, but fundamentally fails when models attempt to verify factual claims without external ground-truth feedback.

Math & Tradeoffs→
M5-072NLP & Large Language ModelsPrompting & Reasoning Techniques
Medium

Prompt Formatting Sensitivity and the Impact of Chat Templates

⚡ One-Sentence Answer

Chat templates standardize the serialization of multi-turn dialogues into unambiguous token streams, preventing prompt injection and formatting mismatch that otherwise degrade model comprehension.

Math & Tradeoffs→