Expand description
NF4 (NormalFloat4) quantization descriptor (scheme tag 0x04).
A non-linear 4-bit quantization scheme introduced by the QLoRA paper. Each
storage code in [0, 15] decodes to one of 16 fixed real-valued levels. Each
block along axis carries a single absmax scale stored in a separate buffer.
See docs/spec/quantization/nf4.md for the normative definition.
Structs§
- Nf4
- Quantization parameters for NF4 (NormalFloat4) block quantization.
Constants§
- NF4_LUT
- The 16 NF4 quantization levels, indexed by the unsigned 4-bit storage code.
- NF4_
MIN_ BLOCK_ SIZE - Minimum block size for NF4 (must be a power of two ≥ 8).