Struct Nf4

Source

pub struct Nf4 { /* private fields */ }

Expand description

Quantization parameters for NF4 (NormalFloat4) block quantization.

Each block along axis contains block_size elements, all sharing a single absmax scale stored in scale_buffer_index. The dequantization formula for element q (a 4-bit code in [0, 15]) with block index b is:

x_real = scale[b] * NF4_LUT[q]

The block index computation follows the same rule as Per-Block Affine (see docs/spec/quantization/per-block-affine.md § Block Layout).

§Wire format

Total descriptor length: 16 bytes (including the 4-byte header).

Offset	Field	Type
4	`axis`	`uint32` LE
8	`block_size`	`uint32` LE
12	`scale_buffer_index`	`uint32` LE

No flags are defined; the flags field MUST be 0x0000.

§Design notes

PartialEq, Eq, and Hash are all derived because this struct contains no floating-point fields.

Copy because the struct is ≤ 24 bytes with no Drop glue.

§Examples

use hurray_core::Nf4;

let q = Nf4::new(0, 64, 1).unwrap();
assert_eq!(q.axis(), 0);
assert_eq!(q.block_size(), 64);
assert_eq!(q.scale_buffer_index(), 1);

Implementations§

Source §

impl Nf4

Source

pub fn new(axis: u32, block_size: u32, scale_buffer_index: u32) -> Result<Self>

Creates a new Nf4 descriptor.

§Errors

Error::InvalidBlockSize — block_size is not a power of two or is less than NF4_MIN_BLOCK_SIZE (8).

§Examples

use hurray_core::{Nf4, Error};

assert!(Nf4::new(0, 64, 1).is_ok());
assert!(Nf4::new(0, 128, 1).is_ok());

// block_size < 8 is rejected.
assert!(Nf4::new(0, 4, 1).is_err());
// Non-power-of-two is rejected.
assert!(Nf4::new(0, 48, 1).is_err());

Source

pub fn axis(&self) -> u32

Returns the quantization axis index.

§Examples

use hurray_core::Nf4;

let q = Nf4::new(2, 64, 1).unwrap();
assert_eq!(q.axis(), 2);

Source

pub fn block_size(&self) -> u32

Returns the number of logical elements per block along axis.

§Examples

use hurray_core::Nf4;

let q = Nf4::new(0, 128, 1).unwrap();
assert_eq!(q.block_size(), 128);

Source

pub fn scale_buffer_index(&self) -> u32

Returns the buffer table index of the per-block absmax scale array.

§Examples

use hurray_core::Nf4;

let q = Nf4::new(0, 64, 3).unwrap();
assert_eq!(q.scale_buffer_index(), 3);

Source

pub fn num_blocks_per_axis(&self, shape_axis: u64) -> u64

Computes the number of blocks along axis for a given shape_axis size.

Uses ceil(shape_axis / block_size).

Returns 0 when shape_axis == 0 per the ADR-007 empty-axis carve-out.

§Examples

use hurray_core::Nf4;

let q = Nf4::new(0, 64, 1).unwrap();
assert_eq!(q.num_blocks_per_axis(128), 2);
assert_eq!(q.num_blocks_per_axis(65), 2); // partial trailing block
assert_eq!(q.num_blocks_per_axis(0), 0);  // ADR-007 empty-axis carve-out

Source

pub fn validate_against_shape_axis(&self, shape_axis: u64) -> Result<()>

Validates this descriptor against the resolved shape_axis size.

Rejects if shape_axis is the DYNAMIC sentinel (u64::MAX), or if shape_axis > 0 and block_size > shape_axis.

§Errors

Error::QuantizationShapeMismatch — shape_axis is the DYNAMIC sentinel, or shape_axis > 0 and block_size > shape_axis.

§Examples

use hurray_core::{Nf4, DYNAMIC};

let q = Nf4::new(0, 64, 1).unwrap();
assert!(q.validate_against_shape_axis(128).is_ok());
assert!(q.validate_against_shape_axis(64).is_ok());
assert!(q.validate_against_shape_axis(0).is_ok()); // ADR-007: waived
assert!(q.validate_against_shape_axis(32).is_err()); // block_size > shape_axis
assert!(q.validate_against_shape_axis(DYNAMIC).is_err()); // dynamic dimension

Source

pub fn valid_storage_types() -> &'static [ElementType]

Returns the set of storage ElementTypes that are valid for this scheme.

Per docs/spec/quantization/nf4.md § Valid Storage Types: only uint4 is valid for NF4.

WHY &'static [ElementType]: no allocation per call (design decision #5).

§Examples

use hurray_core::{ElementType, Nf4};

assert_eq!(Nf4::valid_storage_types(), &[ElementType::Uint4]);
assert!(!Nf4::valid_storage_types().contains(&ElementType::Int4));