⚡ LLMsID: ptq-qat

Quantization PTQ/QAT

Q: Differences between PTQ and QAT, and when to use each?

【Quantization PTQ/QAT 核心考点 1】Differences between PTQ and QAT, and when to use each?。技术解析：Definition: Quantization maps FP16 weights W to low-bit integers x_q = \text{round}(x/s) + z with scale s = (x_{\max}-x_{\min})/(2^b-1) and zero-point z aligned to the b-bit range. PTQ (Post-Training Quantization) calibrates an already-trained model with a small calibration set, ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/ptq-qat

Q: Role of the calibration set in PTQ; how its size/distribution affect accuracy?

【Quantization PTQ/QAT 核心考点 2】Role of the calibration set in PTQ; how its size/distribution affect accuracy?。技术解析：Definition: Quantization maps FP16 weights W to low-bit integers x_q = \text{round}(x/s) + z with scale s = (x_{\max}-x_{\min})/(2^b-1) and zero-point z aligned to the b-bit range. PTQ (Post-Training Quantization) calibrates an already-trained model with a small calibration set, ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/ptq-qat

Q: What is the straight-through estimator (STE) in QAT and why is it needed?

【Quantization PTQ/QAT 核心考点 3】What is the straight-through estimator (STE) in QAT and why is it needed?。技术解析：Definition: Quantization maps FP16 weights W to low-bit integers x_q = \text{round}(x/s) + z with scale s = (x_{\max}-x_{\min})/(2^b-1) and zero-point z aligned to the b-bit range. PTQ (Post-Training Quantization) calibrates an already-trained model with a small calibration set, ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/ptq-qat

Q: Why does PTQ dominate for LLMs, and when is QAT required?

【Quantization PTQ/QAT 核心考点 4】Why does PTQ dominate for LLMs, and when is QAT required?。技术解析：Definition: Quantization maps FP16 weights W to low-bit integers x_q = \text{round}(x/s) + z with scale s = (x_{\max}-x_{\min})/(2^b-1) and zero-point z aligned to the b-bit range. PTQ (Post-Training Quantization) calibrates an already-trained model with a small calibration set, ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/ptq-qat

Q: How is quantization error measured, and what are common error sources?

【Quantization PTQ/QAT 核心考点 5】How is quantization error measured, and what are common error sources?。技术解析：Definition: Quantization maps FP16 weights W to low-bit integers x_q = \text{round}(x/s) + z with scale s = (x_{\max}-x_{\min})/(2^b-1) and zero-point z aligned to the b-bit range. PTQ (Post-Training Quantization) calibrates an already-trained model with a small calibration set, ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/ptq-qat

量化 PTQ/QAT

🎯Core Definition

Quantization maps FP16 weights

W

to low-bit integers

x_q = \text{round}(x/s) + z

with scale

s = (x_{\max}-x_{\min})/(2^b-1)

and zero-point

z

aligned to the b-bit range. PTQ (Post-Training Quantization) calibrates an already-trained model with a small calibration set, no retraining needed; QAT (Quantization-Aware Training) simulates quantization error during training via STE (straight-through estimator: quantized forward, approximate straight-through backward), making the model adapt to quantization.

💡Use Cases

PTQ dominates LLM inference deployment (GPTQ, AWQ, etc.) since large models cannot afford retraining — calibration takes minutes; QAT fits extreme low-bit (INT4/INT3 or lower) and accuracy-critical vertical scenarios, or joint quantization during fine-tuning.

⚡Key Problems Solved

naive uniform quantization collapses due to activation outliers and distribution mismatch; PTQ uses calibration and error compensation to contain the loss, while QAT trains weights to match the quantizer — a speed-vs-accuracy trade-off: PTQ is fast but lower ceiling, QAT is slow but higher ceiling.

🎯Core Interview Exam Points

Fundamentals · Production Scenarios · Deep Dive

Core Def

Explain the core definition, underlying mechanism/math, and key principles of Quantization PTQ/QAT.

Use Cases

In production engineering, what are the primary use cases for Quantization PTQ/QAT and what core pain points does it solve?

🔥 High-Frequency Deep-Dive Questions

Differences between PTQ and QAT, and when to use each?

Role of the calibration set in PTQ; how its size/distribution affect accuracy?

What is the straight-through estimator (STE) in QAT and why is it needed?

Why does PTQ dominate for LLMs, and when is QAT required?

How is quantization error measured, and what are common error sources?