Quantization is just a histogram

Today it clicked: quantization is a histogram.

A histogram does two things. It cuts the number line into bins, and it forgets where inside a bin each value fell — all it remembers is which bin.

Quantization does exactly those two things to a model’s weights. Cut their range into $2^b$ bins. For each weight, store only its bin number — that takes $b$ bits instead of 32. When the weight is needed again, hand back the one value that stands for the whole bin.

The only difference is what gets kept. A histogram keeps how many values landed in each bin. A quantizer keeps which bin each value landed in. It is a histogram read backwards.

So a 4-bit model is a 16-bar histogram of its own weights, and an 8-bit photo is a 256-bar histogram of light.

4-bit

Where the bins go

Equal widthEqual massLeast error

Add an outlier

Bins

16

Rounding error

13.0% of σ

Bits used

3.21 of 4

Each weight is replaced by its bin’s value:

Once you see the histogram, the jargon turns into questions you would ask about any histogram.

How many bins?

That is the bit-width. Every extra bit doubles the bins and halves their width, so the error halves too. Drag the slider and watch it: 26% → 13% → 6.5%.

Where do the bins go?

• Equal width is uniform quantization: simple, and it keeps the arithmetic in plain integers. But weights are bell-shaped, so the tail bins sit nearly empty while the middle is crowded. At 4 bits, only 3.2 of the 4 bits carry information.
• Equal mass puts the same number of weights in every bin, so every code is used equally often: exactly 4 of 4 bits. This is the idea behind NF4, the 4-bit format from QLoRA. The price is fat tail bins, and a slightly larger error.
• Least error lets the data decide: each bin’s value is the average of what fell inside it, and each boundary sits halfway between neighbours. That is k-means on a line, and it wins on error. Its bins follow neither the flat line nor the bell curve but something in between: the curve’s cube root, a result from 1951.

What about outliers?

Tick Add an outlier. One stray weight stretches the range, and equal-width bins waste themselves on empty space: the error jumps from 13% to 22%.

This is why calibration exists, and it is histograms all the way down: PyTorch’s observer for it is literally called HistogramObserver. It builds a histogram of the values flowing through a layer, then decides where to clip.

What do we lose?

Exactly what a histogram loses: where each value sat inside its bin. Quantization error is the within-bin spread. Nothing more mysterious than that.

Real quantizers ask one more question — which weights matter — and methods like GPTQ and AWQ push the rounding error onto the ones that matter least. The bars get priorities. They are still bars.

So the next time someone says “we quantized it to 4 bits”, picture sixteen bars. All the art is in where you draw them.