Deep Learning & LLM Core Science Coding Engine
A comprehensive repository of 69 essential industrial kernels covering foundation activations, Transformer mechanics, alignment RL, parallel architectures, and inference engines. Complete with math derivations, tensor flows, stability checklists, and runnable test assertions.
Science Coding Mastery Tracker
0% DoneLocally synced in your browser. Track, review, and master all industrial kernels.
Numerically Stable Softmax
Industrial-grade implementation subtracting max to prevent exponent overflow while preserving dimensional broadcasting.
Numerically Stable Sigmoid & Log-Sigmoid
Piecewise formulation avoiding overflow on large negatives; Log-Sigmoid powered by logaddexp for robust BCE training.
Gaussian Error Linear Unit (GELU)
Stochastic regularizer gating activation based on standard Gaussian CDF, standard in BERT and GPT-2.
SwiGLU Gated Feed-Forward Network
Flagship FFN operator powering LLaMA and DeepSeek, combining SiLU gated branch with up-projection and down-projection.
Softplus & LeakyReLU with Overflow Guards
Softplus with threshold clipping against overflow, alongside LeakyReLU negative slope leak.
Layer Normalization with Affine Transformation
Industrial-grade implementation and mathematical foundations of Layer Normalization with Affine Transformation.
Root Mean Square Normalization (RMSNorm)
Industrial-grade implementation and mathematical foundations of Root Mean Square Normalization (RMSNorm).
BatchNorm1d with Train vs Eval Distinction
Industrial-grade implementation and mathematical foundations of BatchNorm1d with Train vs Eval Distinction.
Group Normalization & Instance Normalization
Industrial-grade implementation and mathematical foundations of Group Normalization & Instance Normalization.
Inverted Dropout with Train/Eval Scaling
Industrial-grade implementation and mathematical foundations of Inverted Dropout with Train/Eval Scaling.
Sinusoidal Positional Encoding
Industrial-grade implementation and mathematical foundations of Sinusoidal Positional Encoding.
Rotary Position Embedding (RoPE)
Industrial-grade implementation and mathematical foundations of Rotary Position Embedding (RoPE).