Scalar Quantization (SQ8/SQ4)

Scalar Quantization (SQ) is a vector compression technique that independently maps each component of a high-dimensional vector to discrete fixed-point integers; SQ8 linearly projects 32-bit floating point (FP32) values across dimension ranges $[min_d, max_d]$ into 8-bit unsigned integers (uint8) via $q = \text{round}\left(255 \cdot \frac{x - \min}{\max - \min}\right)$, while SQ4 compresses down to 4-bit nibbles, achieving 4x to 8x memory reduction.

Building massive vector indices (tens of millions to billions of vectors) under memory constraints in vector databases, serving as raw vector caching or coarse pruning for HNSW.

100M 1536-dimensional FP32 vectors require 614GB RAM for raw storage, exceeding single-node capacity; SQ8 compresses this to ~153GB and leverages AVX-512 / VNNI integer SIMD instructions for blazing fast distance calculation while maintaining 98%+ Recall@10.

Derive the SQ8 linear quantization and de-quantization formulas and source of precision loss?

What is the Recall difference between Global SQ and Per-Dimension SQ under heavy-tailed distribution?

How do modern CPU AVX-512 VNNI instructions accelerate int8 dot product operations?

Compare SQ8 vs PQ in terms of memory compression ratio, index construction time, and reconstruction error?

Why does SQ8 recall rate improve after applying outlier clipping / percentile thresholding?