🔬 Applied Scientist & ML Specialist938 Problems · 127 Topics🌐 Linked to Tech Vault

TalentMe Science Depth: Universal Applied Science & ML Depth Problem Bank

938 end-to-end science depth, system design, transition bridge & BQ problems covering derivations, production trade-offs, pitfalls, and follow-ups.

🎯

Science Depth Mastery Tracker

0% Mastered

Locally synced in your browser. Track, review, and master all 938 scientific & design topics.

Mastered: 0
Reviewing: 0
Unstarted: 938
🚀 Start Next Unpracticed →
🌐
Linked to TalentMe Tech Vault & MindmapsGlobal System
All 938 questions are directly linked to TalentMe's Tech Vault guides, atomic cards, and interactive mindmaps.
🧭Math Foundations Mindmap & Cards→📖Bayesian Inference, Entropy & KL Divergence→🎯DS Track→
Found 24 in-depth questions
M3-081Deep Learning FoundationsDistributed Training Basics
Medium

Gradient Accumulation vs Data Parallelism: Mathematical Equivalence and Nuances

⚡ One-Sentence Answer

Both compute gradients across larger effective batches: Gradient Accumulation serializes micro-batches across time; Data Parallelism parallelizes micro-batches across space.

Math & Tradeoffs→
M3-082Deep Learning FoundationsDistributed Training Basics
Hard

Communication Bottlenecks in Large-Scale Training and Optimization Strategies

⚡ One-Sentence Answer

Communication bottlenecks arise from network bandwidth limits as models scale; mitigate via computation-communication overlap, gradient bucketing, FP8/FP16 compression, and topology-aware routing.

Math & Tradeoffs→
M3-083Deep Learning FoundationsDistributed Training Basics
Medium

FSDP vs DeepSpeed ZeRO: Architectural Relationship, Mechanics, and Differences

⚡ One-Sentence Answer

FSDP is PyTorch's native C++ implementation of ZeRO-3; both shard parameters, gradients, and optimizer states, but FSDP integrates directly with PyTorch autograd and Composable APIs.

Math & Tradeoffs→
M3-084Deep Learning FoundationsDistributed Training Basics
Hard

3D Parallelism and Sequence Parallelism (SP) in Large Foundation Models

⚡ One-Sentence Answer

3D Parallelism combines Data (DP), Tensor (TP), and Pipeline (PP) parallelism; Sequence Parallelism shards activation-heavy operations (LayerNorm, Dropout) along the sequence dimension.

Math & Tradeoffs→
M3-085Deep Learning FoundationsTraining Diagnostics & Debugging
Easy

Systematic Troubleshooting Workflow When Training Loss Fails to Decrease

⚡ One-Sentence Answer

First perform the single-batch overfitting sanity check; then systematically audit label correctness, learning rate schedules, initialization scale, gradient flow, and loss formulation.

Math & Tradeoffs→
M3-086Deep Learning FoundationsTraining Diagnostics & Debugging
Easy

Training Loss Decreases While Validation Loss Increases: Diagnosis and Remedies

⚡ One-Sentence Answer

This is textbook overfitting (high variance) or distribution drift; resolve by adding regularization, data augmentation, early stopping, or aligning domain shifts.

Math & Tradeoffs→
M3-087Deep Learning FoundationsTraining Diagnostics & Debugging
Medium

Three Common Loss Curve Anomalies and Their Underlying Root Causes

⚡ One-Sentence Answer

Loss curve anomalies include severe high-frequency oscillations (excessive LR), step-like plateau drops (scheduler/data pipeline shifts), and sudden vertical loss spikes (corrupted batches/numerical blowup).

Math & Tradeoffs→
M3-088Deep Learning FoundationsTraining Diagnostics & Debugging
Medium

How to Diagnose Underfitting vs Overfitting: Signals, Baselines, and Thresholds

⚡ One-Sentence Answer

Underfitting is diagnosed when training loss remains unacceptably high and close to validation loss; overfitting occurs when training loss is low while the generalization gap is wide.

Math & Tradeoffs→
M3-089Deep Learning FoundationsTraining Diagnostics & Debugging
Hard

Train-Serving Skew (Training-Inference Inconsistency): Root Causes and Mitigations

⚡ One-Sentence Answer

Train-serving skew occurs when models encounter divergent data distributions or execution behaviors between offline training and live production serving.

Math & Tradeoffs→
M3-090Deep Learning FoundationsTraining Diagnostics & Debugging
Medium

Statistical Diagnostics for Gradients and Activations: Distributions, Dead Units, and Logits

⚡ One-Sentence Answer

Inspect layer-wise activation means, standard deviations, dead neuron percentages, and logit dynamic ranges to detect vanishing, saturation, and representation collapse early.

Math & Tradeoffs→
M3-091Deep Learning FoundationsTraining Diagnostics & Debugging
Hard

Reproducibility Debugging in Deep Learning: RNG Seeds, Deterministic Algorithms, and Pipelines

⚡ One-Sentence Answer

Full reproducibility requires fixing random seeds across all libraries, enabling deterministic GPU algorithms, and isolating multi-processing data loader worker states.

Math & Tradeoffs→
M3-092Deep Learning FoundationsArchitecture Building Blocks
Easy

Why Residual Connections Enable Ultra-Deep Network Training

⚡ One-Sentence Answer

Residual connections break gradient vanishing by providing an additive identity pathway ($I$), creating an implicit ensemble of $2^L$ variable-depth paths and smoothing the loss landscape.

Math & Tradeoffs→
M3-093Deep Learning FoundationsArchitecture Building Blocks
Easy

The Bottleneck Architecture: Dimensionality Reduction, Expansion, and Efficiency

⚡ One-Sentence Answer

The bottleneck structure uses $1\times 1$ convolutions to compress channel dimensions before expensive $3\times 3$ spatial convolutions, slashing FLOPs and parameters while increasing non-linear depth.

Math & Tradeoffs→
M3-094Deep Learning FoundationsArchitecture Building Blocks
Medium

Depthwise Separable Convolutions: Mechanism and Computational Savings

⚡ One-Sentence Answer

It factorizes standard convolution into a channel-wise Depthwise convolution followed by a $1\times 1$ Pointwise convolution, reducing compute and parameters by factor $\frac{1}{C_{\text{out}}} + \frac{1}{K^2} \approx \frac{1}{K^2}$.

Math & Tradeoffs→
M3-095Deep Learning FoundationsArchitecture Building Blocks
Medium

Inductive Bias in Self-Attention vs Convolution

⚡ One-Sentence Answer

Convolution imposes rigid structural inductive biases (spatial locality, translation equivariance); Self-Attention is dynamic, content-based, and permutation-equivariant, trading prior structure for scaling capacity.

Math & Tradeoffs→
M3-096Deep Learning FoundationsArchitecture Building Blocks
Hard

Parameter Sharing in Deep Learning: Forms, Mechanisms, and Regularizing Impact

⚡ One-Sentence Answer

Parameter sharing constrains model capacity by reusing identical weights across space (CNNs), time (RNNs), layers (ALBERT), or modalities, injecting structural priors and slashing memory.

Math & Tradeoffs→
M3-097Deep Learning FoundationsConvolution & Vision Foundations
Easy

Calculating Convolutional Layer Parameter Count and Output Spatial Dimensions

⚡ One-Sentence Answer

Output spatial dimension is $\lfloor \frac{W - K + 2P}{S} \rfloor + 1$; total parameters equal $K^2 \cdot C_{\text{in}} \cdot C_{\text{out}} + C_{\text{out}}$ (with bias).

Math & Tradeoffs→
M3-098Deep Learning FoundationsConvolution & Vision Foundations
Easy

Recursive Calculation of Receptive Field in Convolutional Networks

⚡ One-Sentence Answer

A feature's receptive field expands recursively via $r_l = r_{l-1} + (k_l - 1) \cdot j_{l-1}$, where cumulative stride $j$ exponentially magnifies downsampled kernels.

Math & Tradeoffs→
M3-099Deep Learning FoundationsConvolution & Vision Foundations
Medium

Pooling vs Strided Convolutions: Structural and Functional Trade-offs

⚡ One-Sentence Answer

Pooling performs fixed non-parametric downsampling to inject local translation invariance; strided convolution learns adaptive feature transformations during downsampling.

Math & Tradeoffs→
M3-100Deep Learning FoundationsConvolution & Vision Foundations
Medium

Translation Equivariance vs Translation Invariance: Definitions and Roles in CNNs

⚡ One-Sentence Answer

Equivariance means shifting input shifts output feature maps identically ($f(T x) = T f(x)$); Invariance means shifting input leaves output completely unchanged ($f(T x) = f(x)$).

Math & Tradeoffs→
M3-101Deep Learning FoundationsConvolution & Vision Foundations
Hard

Architectural Design Philosophies: ViT vs Swin Transformer vs ConvNeXt

⚡ One-Sentence Answer

ViT applies minimal-bias global attention (data-hungry); Swin reintroduces hierarchical shifted windows; ConvNeXt modernizes pure CNNs with Transformer design principles.

Math & Tradeoffs→
M4-001Sequences & TransformersRecurrent Models (RNN/LSTM/GRU)
Easy

RNN Forward Dynamics, Backpropagation Through Time (BPTT), and Core Limitations

⚡ One-Sentence Answer

RNNs unroll recurrent transitions sequentially across time; BPTT multiplies repeated transition Jacobians, causing exponential gradient vanishing/explosion and preventing parallelization.

Math & Tradeoffs→
M4-002Sequences & TransformersRecurrent Models (RNN/LSTM/GRU)
Easy

LSTM: Three Gating Mechanisms and the Additive Cell State Highway

⚡ One-Sentence Answer

LSTM introduces forget, input, and output gates around a linear additive cell state $C_t = f_t \odot C_{t-1} + i_t \odot \tilde{C}_t$, providing an uninterrupted gradient highway.

Math & Tradeoffs→
M4-003Sequences & TransformersRecurrent Models (RNN/LSTM/GRU)
Medium

Comparing LSTM and GRU: Structural Simplifications and Trade-offs

⚡ One-Sentence Answer

GRU merges cell and hidden states into a single state and couples forget and input into an update gate, reducing parameter count by 25% while matching LSTM capacity.

Math & Tradeoffs→