quantization
- Quantization is just a histogram
A 4-bit model is a 16-bar histogram of its own weights. Bit-width, calibration, and NF4 all fall out of that one picture.
A 4-bit model is a 16-bar histogram of its own weights. Bit-width, calibration, and NF4 all fall out of that one picture.