SmoothQuant/AWQ & FP8

Three advanced quantization schemes: SmoothQuant migrates activation outliers into weights via a mathematically equivalent transform — rewriting as $Y = (X \cdot \text{diag}(s)) \cdot (\text{diag}(s)^{-1} W)$ keeps the output unchanged but smooths the activation distribution so activations can go INT8 (W8A8); AWQ (Activation-aware Weight Quantization) identifies sensitive channels from activation statistics and protects the top 1% of important weight channels using only scale scaling (no training), achieving high-accuracy W4A16; FP8 offers two formats — E4M3 (4-bit exponent, 3-bit mantissa, higher precision) for forward/weights and E5M2 (5-bit exponent, 2-bit mantissa, wider range) for gradients/backward.

SmoothQuant for W8A8 inference when activations carry strong outliers (LLaMA-family models); AWQ for W4A16 weight quantization when serving very large models; FP8 as the mainstream precision for pretraining/fine-tuning on H100/B200, with forward and backward using different formats.

activation outliers are the main obstacle to LLM quantization — activations are usually harder to quantize than weights; SmoothQuant removes outliers, AWQ protects critical channels, and FP8 splits precision-vs-range duties across two formats.

Why does SmoothQuant work — how are outliers migrated to weights?

How does AWQ find sensitive channels, and why protect the top 1%?

E4M3 vs E5M2 — differences and where each is used?

What are W8A8 and W4A16, and what do they buy?

Why are activations harder to quantize than weights?