⚡ DL & LLM Science Coding105 Code Drills · 17 Core Modules🛡️ 100% Numerically Stable

TalentMe Science Coding: Deep Learning & LLM Core Coding Engine

105 essential kernels covering activations, Transformer mechanics, alignment RL, distributed parallelism, inference serving, and eval calibration—with math derivations, assertions, and offline export.

🎯

Science Coding Mastery Tracker

0% Done

Locally synced in your browser. Track, review, and master all industrial kernels.

🟢Mastered:0
🟡Reviewing:0
⚪Unstarted:105
🚀 Start Next Unpracticed →
Showing 105 matching core algorithm kernels
A1★ Core Essential · Practice Foundation
Easy

Numerically Stable Softmax

Part A · Core Kernels & Activation Functions

Numerically stable implementation subtracting the axis maximum to prevent exponent overflow while preserving dimensional broadcasting.

💡 Subtract max to guard against overflow; keepdims ensures broadcasting; denominator has 1 to avoid zero-division
View Code & Math→
A2★ Core Essential · Industrial Bedrock
Easy

Numerically Stable Sigmoid & Log-Sigmoid

Part A · Core Kernels & Activation Functions

Piecewise formulation avoiding overflow on large negatives; Log-Sigmoid powered by logaddexp for robust BCE training.

💡 Standard form for positives, multiply exp(z) for negatives; use logaddexp for log-sigmoid
View Code & Math→
A3★ Core Essential · Industrial Bedrock
Medium

Gaussian Error Linear Unit (GELU)

Part A · Core Kernels & Activation Functions

Stochastic regularizer gating activation based on standard Gaussian CDF, standard in BERT and GPT-2.

💡 0.5x times (1 + tanh), cubic term scaled by 0.044715
View Code & Math→
A4★ Core Essential · Industrial Bedrock
Medium

SwiGLU Gated Feed-Forward Network

Part A · Core Kernels & Activation Functions

Flagship FFN operator powering LLaMA and DeepSeek, combining SiLU gated branch with up-projection and down-projection.

💡 Gate and up run in parallel; SiLU modulates via Hadamard product; down-projection restores dimension
View Code & Math→
A5
Easy

Softplus & LeakyReLU with Overflow Guards

Part A · Core Kernels & Activation Functions

Softplus with threshold clipping against overflow, alongside LeakyReLU negative slope leak.

💡 log1p for small inputs; threshold cutoff to identity for large values
View Code & Math→
B1★ Core Essential · Industrial Bedrock
Easy

Layer Normalization with Affine Transformation

Part B · Normalization Family

Per-token normalization across the feature dimension with consistent train/eval behavior and learned gamma/beta.

💡 Normalize along the last axis inside sqrt(var + eps), keep dims, then scale with gamma and shift with beta.
View Code & Math→
B2★ Core Essential · Industrial Bedrock
Easy

Root Mean Square Normalization (RMSNorm)

Part B · Normalization Family

Omit mean centering and normalize solely by root-mean-square to save memory passes in modern LLMs.

💡 Skip mean subtraction: square, average, sqrt(mean_sq + eps), divide, and multiply by gamma.
View Code & Math→
B3★ Core Essential · Industrial Bedrock
Medium

BatchNorm1d with Train vs Eval Distinction

Part B · Normalization Family

Track batch statistics with momentum into running stats during training; freeze and apply running stats at inference.

💡 Train on mini-batch stats with running EMA; freeze to running stats during inference.
View Code & Math→
B4
Medium

Group Normalization & Instance Normalization

Part B · Normalization Family

Partition channels into G groups and normalize independently per sample, removing batch-size dependence.

💡 Reshape channels into G groups, normalize across (C/G, H, W) per sample regardless of batch size.
View Code & Math→
B5★ Core Essential · Industrial Bedrock
Easy

Inverted Dropout with Train/Eval Scaling

Part B · Normalization Family

Scale kept activations by 1/(1-p) during training so expectations match without extra work at inference.

💡 Scale by 1/(1-p) during training so eval mode is an exact zero-cost identity pass.
View Code & Math→
C1★ Core Essential · Industrial Bedrock
Easy

Sinusoidal Positional Encoding

Part C · Positional Encoding Evolution

Classic Transformer fixed sine/cosine encodings across geometric wavelengths enabling linear relative shifts.

💡 Even dims take sin, odd dims take cos; log-exp frequencies map relative offsets to orthogonal rotations.
View Code & Math→
C2★ Core Essential · Industrial Bedrock
Medium

Rotary Position Embedding (RoPE)

Part C · Positional Encoding Evolution

Apply 2D pairwise rotations to Q and K so dot products depend strictly on relative token distance.

💡 Rotate Q/K pairs by m*theta: dot product depends strictly on relative offset (m - n).
View Code & Math→