6.3
6.3 Model Compression & Quantization
Hopper/Blackwell native FP8/NVFP4 compute, SmoothQuant/QuaRot outlier mitigation, AWQ activation-aware quantization, KV Cache compression, and quality gates.
Native FP8 / NVFP4 Compute, SmoothQuant & GPTQ
Ada/Hopper native FP8 and Blackwell NVFP4 fine-grained block quantization, SmoothQuant outlier channel scaling, GPTQ inverse Hessian error compensation, and GGUF cross-platform quantization.
🏢 Companies
NVIDIA、MIT HAN Lab、Neural Magic、llama.cpp、Hugging Face
🛠️ Tech Stack
FP8 (E4M3/E5M2)NVFP4SmoothQuantGPTQGGUFINT4Tensor Cores
💼 Roles & Salary
Quantization Engineer、Model Optimization Engineer、Inference Engineer
💰 $210K - $440K / year (Model Quantization) | ¥600K - ¥1.4M / year
📚 Prerequisites: FP8/FP4 Binary Specs & Dynamic Scaling • Quantization Scale & Zero-Point Derivations • GPTQ Inverse Hessian Taylor Expansion Math • GGUF Quantization Packaging & Dequantization
AWQ Salient Activation, KV Cache Quantization & Quality Gates
AWQ protecting salient top-1% activation channels, FP8/INT8 KV Cache quantization for 2x concurrency, calibration dataset standards, automated multi-benchmark regression gates, and fallback rollback strategies.
🏢 Companies
NVIDIA、MIT HAN Lab (AWQ)、AMD、Intel (OpenVINO)
🛠️ Tech Stack
AWQKV Cache QuantizationCalibration DatasetPerplexity RegressionQuality GatesMMLUAccuracy Fallback
💼 Roles & Salary
Quantization Engineer、Evaluation Engineer、Inference Engineer
💰 $205K - $430K / year (KV Cache Quant & Quality Gates) | ¥580K - ¥1.35M / year
📚 Prerequisites: Activation Outlier Distribution Analysis • Per-Channel vs Per-Group Quantization Granularity • KV Cache Quantization Impact on Attention • Automated Perplexity & Benchmark CI Pipelines