⚡ LLMsID: low-bit

Low-Bit INT8/INT4

Q: What are the memory/bandwidth savings of INT8/INT4 vs FP16?

【Low-Bit INT8/INT4 核心考点 1】What are the memory/bandwidth savings of INT8/INT4 vs FP16?。技术解析：Definition: Low-bit quantization drops weights/activations to 8-bit or 4-bit integers. Storage goes from 2 bytes (FP16) to 1 byte (INT8) to 0.5 byte (INT4) per weight — INT4 weights cut weight memory by 75% (FP16 70B needs 140GB, INT4 only ~35GB). Each tensor is aligned with a sc... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/low-bit

Q: Why does INT4 save 75% memory — walk through the math?

【Low-Bit INT8/INT4 核心考点 2】Why does INT4 save 75% memory — walk through the math?。技术解析：Definition: Low-bit quantization drops weights/activations to 8-bit or 4-bit integers. Storage goes from 2 bytes (FP16) to 1 byte (INT8) to 0.5 byte (INT4) per weight — INT4 weights cut weight memory by 75% (FP16 70B needs 140GB, INT4 only ~35GB). Each tensor is aligned with a sc... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/low-bit

Q: What is group-wise quantization and why does it help at low bits?

【Low-Bit INT8/INT4 核心考点 3】What is group-wise quantization and why does it help at low bits?。技术解析：Definition: Low-bit quantization drops weights/activations to 8-bit or 4-bit integers. Storage goes from 2 bytes (FP16) to 1 byte (INT8) to 0.5 byte (INT4) per weight — INT4 weights cut weight memory by 75% (FP16 70B needs 140GB, INT4 only ~35GB). Each tensor is aligned with a sc... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/low-bit

Q: How do you choose the calibration set, and why is it accuracy-critical?

【Low-Bit INT8/INT4 核心考点 4】How do you choose the calibration set, and why is it accuracy-critical?。技术解析：Definition: Low-bit quantization drops weights/activations to 8-bit or 4-bit integers. Storage goes from 2 bytes (FP16) to 1 byte (INT8) to 0.5 byte (INT4) per weight — INT4 weights cut weight memory by 75% (FP16 70B needs 140GB, INT4 only ~35GB). Each tensor is aligned with a sc... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/low-bit

Q: What are common accuracy-loss sources at low bits and how to mitigate them?

【Low-Bit INT8/INT4 核心考点 5】What are common accuracy-loss sources at low bits and how to mitigate them?。技术解析：Definition: Low-bit quantization drops weights/activations to 8-bit or 4-bit integers. Storage goes from 2 bytes (FP16) to 1 byte (INT8) to 0.5 byte (INT4) per weight — INT4 weights cut weight memory by 75% (FP16 70B needs 140GB, INT4 only ~35GB). Each tensor is aligned with a sc... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/LLM/card/low-bit

低比特 INT8/INT4

🎯Core Definition

Low-bit quantization drops weights/activations to 8-bit or 4-bit integers. Storage goes from 2 bytes (FP16) to 1 byte (INT8) to 0.5 byte (INT4) per weight — INT4 weights cut weight memory by 75% (FP16 70B needs 140GB, INT4 only ~35GB). Each tensor is aligned with a scale

s

and zero-point

z

x_q = \text{round}(x/s) + z

; 4-bit typically uses group-wise quantization (e.g., one shared scale per 128 weights) to avoid small-scale channels being swallowed by a wide range.

💡Use Cases

memory-constrained deployment (7B-70B on consumer GPUs and edge devices), inference bandwidth optimization, KV cache quantization, and offline one-shot conversion pipelines.

⚡Key Problems Solved

bigger models mean costlier memory and bandwidth; INT4 removes 75% of weight memory with no added latency, and calibration-set choice plus group-wise quantization keep accuracy loss manageable — the calibration set is the key to precision, since distribution mismatch is the main killer at low bits.

🎯Core Interview Exam Points

Fundamentals · Production Scenarios · Deep Dive

Core Def

Explain the core definition, underlying mechanism/math, and key principles of Low-Bit INT8/INT4.

Use Cases

In production engineering, what are the primary use cases for Low-Bit INT8/INT4 and what core pain points does it solve?

🔥 High-Frequency Deep-Dive Questions

What are the memory/bandwidth savings of INT8/INT4 vs FP16?

Why does INT4 save 75% memory — walk through the math?

What is group-wise quantization and why does it help at low bits?

How do you choose the calibration set, and why is it accuracy-critical?

What are common accuracy-loss sources at low bits and how to mitigate them?