I1Medium★ Core Essential · Industrial BedrockPart I · Inference, Decoding & Quantization
INT8 Affine & Symmetric Quantization
Map FP32 tensors to INT8 and dequantize back using scale S and zero-point Z.
⏱️ Time:
O(N)💾 Space:
显存压缩至原先的 1/4🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
def quantize_int8_symmetric(x: np.ndarray) -> tuple:
"""对称量化 (零点 Z=0)"""
max_val =
scale = max_val / 127.0 if max_val > 0 else 1.0
q = np.clip(np.round(x / scale), -127, 127).astype(np.int8)
return q, float(scale)
def dequantize_int8_symmetric(q: np.ndarray, scale: float) -> np.ndarray:
"""对称反量化"""
return q.astype(np.float32) * scale
def quantize_int8_affine(x: np.ndarray) -> tuple:
"""非对称仿射量化 (适用于激活值全为正数如 ReLU/GELU 场景)"""
x_min, x_max = np.min(x), np.max(x)
if x_min == x_max:
return np.zeros_like(x, dtype=np.int8), 1.0, 0
scale = (x_max - x_min) / 255.0
zero_point = int(np.round(-x_min / scale) - 128)
q =
return q, float(scale), int(zero_point)
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: How does SmoothQuant handle activation outliers in W8A8 LLM quantization?
💡
Core Mental Anchor / Mnemonic
Fit dynamic range via scale S, anchor real 0.0 via zero-point Z (or Z=0 for symmetric).
📐 Mathematical Derivation & Core Formula
### Linear Affine & Symmetric Quantization Mapping
Quantizing a real tensor into integers (e.g., INT8 ):
1. Scale ($S$): ;
2. Zero-Point ($Z$): , ensuring exact
3. Symmetric quantization: Sets so , avoiding zero-point offset terms during integer matmuls.
Quantizing a real tensor into integers (e.g., INT8 ):
1. Scale ($S$): ;
2. Zero-Point ($Z$): , ensuring exact
0.0 representation for padding;3. Symmetric quantization: Sets so , avoiding zero-point offset terms during integer matmuls.
🔄 Tensor Dimensions & Shape Flow
x (FP32) -> min/max -> compute S, Z -> round(x/S) + Z -> clip[-128, 127] -> q (INT8)
🛡️ Numerical Stability & Pitfalls Checklist
- Guard
scale = 1.0whenmax == min(constant tensors) to prevent zero division. - Cast back to FP32 before applying
(q - Z) * Sduring dequantization to prevent int8 wraparound.
💻 Reference Code Implementation
import numpy as np
def quantize_int8_symmetric(x: np.ndarray) -> tuple:
"""对称量化 (零点 Z=0)"""
max_val = np.max(np.abs(x))
scale = max_val / 127.0 if max_val > 0 else 1.0
q = np.clip(np.round(x / scale), -127, 127).astype(np.int8)
return q, float(scale)
def dequantize_int8_symmetric(q: np.ndarray, scale: float) -> np.ndarray:
"""对称反量化"""
return q.astype(np.float32) * scale
def quantize_int8_affine(x: np.ndarray) -> tuple:
"""非对称仿射量化 (适用于激活值全为正数如 ReLU/GELU 场景)"""
x_min, x_max = np.min(x), np.max(x)
if x_min == x_max:
return np.zeros_like(x, dtype=np.int8), 1.0, 0
scale = (x_max - x_min) / 255.0
zero_point = int(np.round(-x_min / scale) - 128)
q = np.clip(np.round(x / scale) + zero_point, -128, 127).astype(np.int8)
return q, float(scale), int(zero_point)
🧪 Runnable Assertions & Validation
Copy and run directly in Python / Jupyter:
import numpy as np
x = np.linspace(-10.0, 10.0, 100)
q, scale = quantize_int8_symmetric(x)
x_recon = dequantize_int8_symmetric(q, scale)
# 量化误差最大不应超过半个量化台阶 scale/2
err = np.max(np.abs(x - x_recon))
assert err <= (scale / 2.0 + 1e-4)
print("✓ INT8 仿射与对称量化自测通过")🎯 Core Follow-up Inquiries & Reference Answers
Q1:How does SmoothQuant handle activation outliers in W8A8 LLM quantization?
SmoothQuant migrates quantization difficulty from spiky activation channels into smoother weights via per-channel scaling : , making both activations and weights easier to quantize cleanly to INT8.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.