Skip to main content

Module quantization

Module quantization 

Source
Expand description

Quantization descriptor types for the Hurray tensor format.

Every quantized tensor carries a quantization descriptor immediately following the tensor descriptor header. The descriptor starts with a 4-byte header (scheme tag, scheme version, flags) followed by a scheme-specific payload.

§Supported schemes

Scheme tagTierType
0x011PerTensorAffine
0x021PerChannelAffine
0x031PerBlockAffine
0x041Nf4
0x051Mxfp

§Encoding overview

The canonical encoding path is:

  1. Call QuantizationDescriptor::encode_into to write into a caller-owned buffer (zero-alloc, required by Layer 5 streaming writers).
  2. Or call QuantizationDescriptor::encode_to_vec for a convenience Vec<u8>.

The canonical decoding path is:

  • Call QuantizationDescriptor::decode which returns the descriptor and the number of bytes consumed, allowing streaming readers to advance their cursor without re-scanning.

See docs/spec/quantization.md for the normative definition.

Re-exports§

pub use mxfp::Mxfp;
pub use mxfp::MXFP_CANONICAL_BLOCK_SIZE;
pub use mxfp::MXFP_MAX_BLOCK_SIZE;
pub use mxfp::MXFP_MIN_BLOCK_SIZE;
pub use nf4::Nf4;
pub use nf4::NF4_LUT;
pub use nf4::NF4_MIN_BLOCK_SIZE;
pub use per_block_affine::PerBlockAffine;
pub use per_block_affine::PER_BLOCK_AFFINE_MIN_BLOCK_SIZE;
pub use per_channel_affine::PerChannelAffine;
pub use per_tensor_affine::PerTensorAffine;

Modules§

mxfp
MXFP (OCP Microscaling) quantization descriptor (scheme tag 0x05).
nf4
NF4 (NormalFloat4) quantization descriptor (scheme tag 0x04).
per_block_affine
Per-block affine quantization descriptor (scheme tag 0x03).
per_channel_affine
Per-channel affine quantization descriptor (scheme tag 0x02).
per_tensor_affine
Per-tensor affine quantization descriptor (scheme tag 0x01).

Enums§

QuantizationDescriptor
A typed quantization descriptor carrying the parameters for one of the supported quantization schemes.
QuantizationSchemeTag
The wire scheme tag that identifies which quantization scheme a descriptor uses.

Functions§

validate_axis
Validates that axis is a valid axis index for a tensor of the given rank.
validate_buffer_placement
Validates that all quantization-parameter buffer indices in desc are valid within the buffers table and do not alias the tensor data buffer.