M3-074M3: Deep Learning FoundationsTraining Stability & Mixed PrecisionHard
⚡Flashcard Drill➔Mastery:

Operators Most Prone to Instability Under Low-Precision (FP16/BF16)

📐 Mathematical Definition
risky={exp⁡,log⁡,÷,sum-reduction, θ←θ+ηΔ},ηΔ≪θ\text{risky}=\{\exp,\log,\div,\text{sum-reduction},\ \theta\leftarrow\theta+\eta\Delta\},\quad \eta\Delta\ll\theta
⚡ One-Sentence Core Answer
Operators involving exponentials (Softmax, GELU), logarithms (Cross-Entropy), division (LayerNorm), and massive summations (large-dim matrix multiplications) are most vulnerable.

📌 Key Takeaways

  • •
    Exponentials (exp): Softmax, Sigmoid, and GELU overflow in FP16 when input exceeds 11.0911.09
  • •
    Divisions & Inversions: LayerNorm, RMSNorm, and batch variance calculation amplify small precision errors
  • •
    Accumulation reduction: FP16 matrix multiplications across large hidden dimensions (d=8192d=8192) suffer from accumulator underflow/swamping

📐 Mathematical Derivations

Vulnerability Analysis:
① Transcendental Exponentials: exp⁡(x)\exp(x) overflows in FP16 at x>ln⁡(65504)≈11.09x > \ln(65504) \approx 11.09. In self-attention, if qTk/dk>11.09q^T k / \sqrt{d_k} > 11.09, standard FP16 softmax immediately produces Inf. Fused FP32 accumulation and max-subtraction are mandatory.
② Variance and Standard Deviation: Computing σ2=1d∑(xi−μ)2\sigma^2 = \frac{1}{d} \sum (x_i - \mu)^2. In low precision, catastrophic cancellation occurs when subtracting large near-equal numbers (xi2−μ2x_i^2 - \mu^2), occasionally producing negative variance σ2<0\sigma^2 < 0. Taking σ2\sqrt{\sigma^2} produces NaN.
③ Catastrophic Swamping in Reductions:
When accumulating N=100,000N=100,000 small values into a large sum: St+1=St+xiS_{t+1} = S_t + x_i. When St>211xiS_t > 2^{11} x_i, low mantissa bits in FP16 are rounded off, causing small gradient increments to be completely ignored (swamping).
PyTorch Autocast Defense: torch.cuda.amp.autocast automatically leaves Softmax, LayerNorm, and Cross-Entropy in FP32 while executing large MatMuls and Convs in FP16/BF16.

🏭 Production Trade-offs

Autocast whitelist/blacklist: Always maintain FP32 master weights and execute loss computation, softmax, and normalization in FP32, casting back to FP16/BF16 only for linear/GEMM operations.
⚠️ Common Interview Pitfalls
  • ✕
    Manually casting LayerNorm or Softmax to FP16 inside custom model definitions, bypassing autocast safeguards
  • ✕
    Computing Softmax\text{Softmax} over long sequence lengths in pure FP16 without online FP32 accumulation
🎯 Interviewer Follow-up Inquiries
  • ?
    Why does PyTorch's autocast selectively keep LayerNorm and Softmax in FP32 during mixed precision execution?
  • ?
    What is 'catastrophic cancellation' in floating-point subtraction, and how does Welford's algorithm avoid it?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

Explore the comprehensive technical article, exam cards, and global architecture tree.

← PreviousM3-073: Systematic Troubleshooting Workflow When Training Loss Becomes NaN📋Back to BankNext →M3-075: Gradient Checkpointing (Activation Rematerialization): Memory vs Compute Trade-offs
中文速览与面试要点

哪些算子/操作在低精度下最容易出问题?

含 exp/log/除法的算子(softmax、LN、CE)、累加归约(attention、matmul)、以及需精确表示小增量的权重更新。

核心要点

  • 数学机理:低精度风险来自' 范围 '与' 精度 '两类。 (1) 范围风险(上溢/下溢) ——含 exp 的算子:softmax、GELU、sigmoid;FP16 下 exp(x) 在 x>11 时即超 65504 上溢。含 log 的算子:CE、KL、log-softmax;log(0) 得 −∞。含除法的算子:LN/BN(除方差)、attention 归一化;分母为 0 或极小则爆炸。 (2) 精度风险(累加误差) —— 归约运算 (sum/mean)在 FP16 下逐元素累加会累积舍入误差:累加 n 个数的误差约 O(n·u)(u 为机器精度);attention 的 log-sum-exp 要对序列长度 L 个元素求和、LN 要对 d 个元素求和,L/d 大时误差显著。故实践中这些归约用 FP32 累加('accumulate in FP32')。 (3) 表示风险 ——权重更新 θ←θ+ηΔθ:若 ηΔθ 相对于 θ 极小(如 η=1e-5、θ≈0.1、Δθ≈1),则 ηΔθ≈1e-6,而 FP16 在 0.1 附近的分辨率约 6e-5—— 更新量被舍入掉、参数根本不动 ;故必须用 FP32 master weights 做更新。这三类风险对应三条对策: 减 max / 加 eps / FP32 累加 + FP32 master 。
  • 深度剖析与工程权衡:① autocast 的白名单 ——PyTorch 的 autocast 会自动把 matmul/conv 等'安全'算子转 FP16(吞吐收益大),而把 softmax、LN、loss、归约等'敏感'算子保持 FP32(autocast 内置白/黑名单);理解这一点比手工管理 dtype 更可靠。② Flash Attention 的贡献 ——它在 kernel 内部用 FP32 做在线 softmax(维护 running max 与 running sum),既保证数值稳定又避免物化 L×L 的注意力矩阵;这是'数值稳定 + 高效'结合的典范。③ LayerNorm 的 ε ——低精度下方差可能因舍入略小于真实值,ε 太小则除法放大误差;故 LN 的 ε 常取 1e-5~1e-6,且计算在 FP32 下进行。④ matmul 的累加 ——张量核心的 FP16 matmul 在内部用 FP32 累加(这是硬件特性),故 matmul 本身相对安全;风险主要在非 matmul 的逐元素归约。⑤ 量化推理的对应问题 ——INT8 推理时,激活的离群值导致 per-tensor 量化误差大,故用 per-channel/per-token 量化(同'范围'问题的另一种解法)。⑥ 面试要点 ——被问'低精度训练要注意什么',应能按' exp/log/除法(范围)+ 归约累加(精度)+ 权重更新(表示) '三类回答,并提到 autocast 白名单与 FP32 master weights 两个具体机制;这体现'从原理到工程'的完整链条。
  • softmax/log-sum-exp:exp 上溢、减 max 必须做
  • LayerNorm/BN:方差计算与除法、ε 需调大
  • 损失函数:log(0) 下溢
  • 权重更新:FP16 下 η·g 被 θ 吞掉(需 FP32 master)

常见陷阱

  • 对所有算子一刀切用 FP16(敏感算子失稳)
  • LN 的 ε 设得过小(低精度下除法放大误差)