I1Medium★ Core Essential · Industrial BedrockPart I · Inference, Decoding & Quantization

INT8 Affine & Symmetric Quantization

⚡Flashcard Drill➔

Map FP32 tensors to INT8 and dequantize back using scale S and zero-point Z.

⏱️ Time: O(N)
💾 Space: 显存压缩至原先的 1/4
🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
def quantize_int8_symmetric(x: np.ndarray) -> tuple:
"""对称量化 (零点 Z=0)"""
max_val =
scale = max_val / 127.0 if max_val > 0 else 1.0
q = np.clip(np.round(x / scale), -127, 127).astype(np.int8)
return q, float(scale)
def dequantize_int8_symmetric(q: np.ndarray, scale: float) -> np.ndarray:
"""对称反量化"""
return q.astype(np.float32) * scale
def quantize_int8_affine(x: np.ndarray) -> tuple:
"""非对称仿射量化 (适用于激活值全为正数如 ReLU/GELU 场景)"""
x_min, x_max = np.min(x), np.max(x)
if x_min == x_max:
return np.zeros_like(x, dtype=np.int8), 1.0, 0
scale = (x_max - x_min) / 255.0
zero_point = int(np.round(-x_min / scale) - 128)
q =
return q, float(scale), int(zero_point)
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: How does SmoothQuant handle activation outliers in W8A8 LLM quantization?
Next: LLM Generation Sampler (Temperature, Top-K & Top-P) →
💡

Core Mental Anchor / Mnemonic

Fit dynamic range via scale S, anchor real 0.0 via zero-point Z (or Z=0 for symmetric).

📐 Mathematical Derivation & Core Formula

q=clip(round(xS)+Z,−128,127),x^=S⋅(q−Z)q = \mathrm{clip}\left(\mathrm{round}\left(\frac{x}{S}\right) + Z, -128, 127\right), \quad \hat{x} = S \cdot (q - Z)
### Linear Affine & Symmetric Quantization Mapping
Quantizing a real tensor x∈[α,β]x \in [\alpha, \beta] into integers [qmin⁡,qmax⁡][q_{\min}, q_{\max}] (e.g., INT8 [−128,127][-128, 127]):
1. Scale ($S$): S=β−αqmax⁡−qmin⁡S = \frac{\beta - \alpha}{q_{\max} - q_{\min}};
2. Zero-Point ($Z$): Z=round(−α/S)+qmin⁡Z = \mathrm{round}(-\alpha / S) + q_{\min}, ensuring exact 0.0 representation for padding;
3. Symmetric quantization: Sets α=−β=−max⁡(∣x∣)\alpha = -\beta = -\max(|x|) so Z=0Z = 0, avoiding zero-point offset terms during integer matmuls.

🔄 Tensor Dimensions & Shape Flow

x (FP32) -> min/max -> compute S, Z -> round(x/S) + Z -> clip[-128, 127] -> q (INT8)

🛡️ Numerical Stability & Pitfalls Checklist

  • Guard scale = 1.0 when max == min (constant tensors) to prevent zero division.
  • Cast back to FP32 before applying (q - Z) * S during dequantization to prevent int8 wraparound.

💻 Reference Code Implementation

import numpy as np

def quantize_int8_symmetric(x: np.ndarray) -> tuple:
    """对称量化 (零点 Z=0)"""
    max_val = np.max(np.abs(x))
    scale = max_val / 127.0 if max_val > 0 else 1.0
    q = np.clip(np.round(x / scale), -127, 127).astype(np.int8)
    return q, float(scale)

def dequantize_int8_symmetric(q: np.ndarray, scale: float) -> np.ndarray:
    """对称反量化"""
    return q.astype(np.float32) * scale

def quantize_int8_affine(x: np.ndarray) -> tuple:
    """非对称仿射量化 (适用于激活值全为正数如 ReLU/GELU 场景)"""
    x_min, x_max = np.min(x), np.max(x)
    if x_min == x_max:
        return np.zeros_like(x, dtype=np.int8), 1.0, 0
    scale = (x_max - x_min) / 255.0
    zero_point = int(np.round(-x_min / scale) - 128)
    q = np.clip(np.round(x / scale) + zero_point, -128, 127).astype(np.int8)
    return q, float(scale), int(zero_point)

🧪 Runnable Assertions & Validation

Copy and run directly in Python / Jupyter:

import numpy as np
x = np.linspace(-10.0, 10.0, 100)
q, scale = quantize_int8_symmetric(x)
x_recon = dequantize_int8_symmetric(q, scale)
# 量化误差最大不应超过半个量化台阶 scale/2
err = np.max(np.abs(x - x_recon))
assert err <= (scale / 2.0 + 1e-4)
print("✓ INT8 仿射与对称量化自测通过")

🎯 Core Follow-up Inquiries & Reference Answers

Q1:How does SmoothQuant handle activation outliers in W8A8 LLM quantization?
SmoothQuant migrates quantization difficulty from spiky activation channels into smoother weights via per-channel scaling ss: Y=(X⋅diag(s)−1)(diag(s)⋅W)Y = (X \cdot \mathrm{diag}(s)^{-1})(\mathrm{diag}(s) \cdot W), making both activations and weights easier to quantize cleanly to INT8.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
← Prev: ZeRO-1/2/3 Model-State Memory CalculatorBack to Part I & All 105Next: LLM Generation Sampler (Temperature, Top-K & Top-P) →