hurray

Python bindings for the Hurray tensor interchange format.

hurray is a codec and a zero-copy bridge: it produces and consumes Hurray tensors and hands their buffers to the array ecosystem without copying. It does no arithmetic — the math belongs to whichever framework the buffer is handed to.

import hurray, numpy as np

# Wraps the array's buffer — no copy in, no copy out.
t = hurray.asarray(np.arange(12, dtype=np.float32).reshape(3, 4))
hurray.save("weights.hrry", {"w": t})

w = hurray.load("weights.hrry")["w"]
assert w.shape == (3, 4) and w.dtype == hurray.float32
np.asarray(w)[0]        # array([0., 1., 2., 3.], dtype=float32)

The API, by what it is for

Group Names
The tensor Tensor, Composite, Descriptor
Element types Dtype, the type constants (float32, int4, bfloat16, …), the dtype submodule
Devices Device, the device constants (cpu, cuda, …), the device submodule
Construction zeros, ones, full, empty and their _like forms, arange, linspace, eye
Interop asarray, from_dlpack, from_numpy, from_torch, from_scipy, from_hurray, sparse_coo
Layouts Layout and one subclass per layout (RowMajorLayout, CsrLayout, BlockPagedLayout, …)
Quantization PerTensorAffine, PerChannelAffine, PerBlockAffine, NF4, MXFP, decode_quantization
Buffers BufferHandle, aligned_allocator, MIN_BUFFER_ALIGNMENT, PAGE_ALIGNMENT
Files save, load
Streaming StreamWriter, StreamReader
Display set_print_options, get_print_options, print_options
Errors InvalidDescriptorError, BufferError, CopyRequiredError, UnsupportedError, FileError, StreamError, InternalError

Interchange protocols

A Tensor is a producer for __dlpack__, the NumPy array protocols, and Hurray's own __hurray__ protocol, and a consumer of all three. __hurray__ is the only one that carries the full descriptor — layout, quantization, multiple buffers — across a process boundary; DLPack and NumPy carry a single strided buffer, which is all they model.

1from .hurray import *
2
3__doc__ = hurray.__doc__
4if hasattr(hurray, "__all__"):
5    __all__ = hurray.__all__
__version__ = '0.1.0'
class InvalidDescriptorError(builtins.ValueError):

Raised when a Hurray tensor descriptor fails validation. Subclass of :exc:ValueError.

class BufferError(builtins.ValueError):

Raised for buffer size or alignment errors. Subclass of :exc:ValueError.

.. note:: This is hurray.BufferError, distinct from the built-in builtins.BufferError used by the DLPack protocol for non-representable element types.

class CopyRequiredError(builtins.ValueError):

Raised when copy=False is passed to __array__ but a copy is required (e.g. a dtype cast is needed). Subclass of :exc:ValueError.

This follows the NumPy 2.0 __array__ convention (NEP 47): the caller requested zero-copy but the conversion cannot be done without copying.

class UnsupportedError(builtins.NotImplementedError):

Raised when an element type, memory layout, or device combination is not supported by the current Hurray version. Subclass of :exc:NotImplementedError.

class InternalError(builtins.RuntimeError):

Raised when an unexpected Rust panic is caught by the binding layer. Subclass of :exc:RuntimeError.

If you see this exception, please file a bug report.

class FileError(builtins.OSError):

Raised by hurray.load() and hurray.save() for file-level errors: file not found, permission denied, corrupt HRRYFILE container, unexpected EOF. Subclass of :exc:OSError.

class StreamError(builtins.OSError):

Raised by the streaming reader/writer for framing errors: frame corruption mid-stream, unexpected stream termination. Subclass of :exc:OSError.

class Dtype:

A Hurray element type descriptor.

Dtype objects are singletons: hurray.float32 is hurray.dtype.float32. They are immutable (frozen) and hashable — safe to use as dict keys.

Tier classification

Tier Meaning Accessible at
1 Array API-compatible hurray.<name> and hurray.dtype.<name>
2 Extended / sub-byte hurray.dtype.<name> only

Examples (Python)

import hurray

assert hurray.float32.name == "float32"
assert hurray.float32.is_array_api
assert hurray.float32 is hurray.dtype.float32   # singleton identity

assert hurray.dtype.int4.bit_width == 4
assert not hurray.dtype.int4.is_array_api
assert not hasattr(hurray, "int4")              # Tier 2: no top-level alias

# Usable as a dict key.
dtypes = {hurray.float32: "fp32", hurray.dtype.int4: "i4"}
assert dtypes[hurray.float32] == "fp32"
def from_name(cls, /, name):

Parse a Dtype from its canonical name string.

Raises hurray.InvalidDescriptorError for unknown names.

Examples

import hurray

assert hurray.Dtype.from_name("float32") is hurray.float32
assert hurray.Dtype.from_name("int4") is hurray.dtype.int4

try:
    hurray.Dtype.from_name("not_a_type")
except hurray.InvalidDescriptorError:
    pass
def from_tag(cls, /, tag):

Parse a Dtype from its wire tag — the inverse of Dtype.tag.

This is what a decoder does with the byte it read. Reserved tags (assigned to no type in this version of the format) and the permanently invalid sentinels 0x00 and 0xFF are both refused, and the error says which: a reserved tag may mean the producer is newer than this reader, while an invalid one is a corrupt descriptor.

Examples

name

The canonical lowercase spec name for this type (e.g. "float32", "int4").

Examples

assert hurray.float32.name == "float32"
assert hurray.dtype.int4.name == "int4"
is_float

True if this is a floating-point or complex type.

Examples

assert hurray.float32.is_float
assert not hurray.int32.is_float
is_signed

True if this type is signed (has a sign bit or two's-complement sign).

Unsigned integers, bool, and float8_e8m0 return False.

Examples

assert hurray.int32.is_signed
assert not hurray.uint32.is_signed
tag

This type's wire tag: the byte that identifies it in an encoded descriptor.

The tags are normative (element-types.md § Type Tags), so this is what a producer writes and a consumer reads — and what hurray-inspect prints.

Examples

import hurray

assert hurray.float32.tag == 0x03
assert hurray.Dtype.from_tag(hurray.float32.tag) is hurray.float32
is_sub_byte

True if elements occupy fewer than 8 bits (e.g. int4, uint4, bool).

Examples

assert hurray.dtype.int4.is_sub_byte
assert not hurray.float32.is_sub_byte
is_array_api

True iff this is a Tier 1 (Array API-compatible) type.

Equivalent to self.tier == 1.

Examples

assert hurray.float32.is_array_api
assert not hurray.dtype.int4.is_array_api
is_integer

True if this is a signed or unsigned integer type (not float, not bool).

Examples

assert hurray.int32.is_integer
assert not hurray.float32.is_integer
bit_width

Bit width of one scalar element (e.g. 32 for float32, 4 for int4).

Examples

assert hurray.float32.bit_width == 32
assert hurray.dtype.int4.bit_width == 4
tier

The tier of this type: 1 for Array API-compatible types, 2 for extended types.

Examples

assert hurray.float32.tier == 1
assert hurray.dtype.int4.tier == 2
element_alignment

The natural alignment of a single element, in bytes.

This is the element's own alignment, not the buffer's: a float32 buffer starts on a 64-byte boundary (hurray.MIN_BUFFER_ALIGNMENT) but its elements are 4-aligned within it. Sub-byte types report 1, since a packed element has no address of its own.

Examples

import hurray

assert hurray.float32.element_alignment == 4
assert hurray.float64.element_alignment == 8
assert hurray.dtype.int4.element_alignment == 1     # packed, two per byte
def buffer_size_bytes(dtype, count):

The number of bytes a buffer needs to hold count elements of dtype.

Not count * dtype.bit_width // 8: sub-byte types pack, and the packing rules differ per width (memory-layout.md § Sub-byte packing). int4 fits two elements per byte and rounds up; bool fits eight; the 6-bit float types pack four elements into three bytes. Getting this wrong produces a buffer that is one byte short of the last element, which the descriptor validator catches and the caller then has to debug.

Examples

import hurray

assert hurray.buffer_size_bytes(hurray.float32, 100) == 400
assert hurray.buffer_size_bytes(hurray.dtype.int4, 7) == 4        # ceil(7 / 2)
assert hurray.buffer_size_bytes(hurray.bool, 9) == 2              # ceil(9 / 8)
assert hurray.buffer_size_bytes(hurray.dtype.float6_e2m3, 100) == 75  # ceil(100/4)*3
assert hurray.buffer_size_bytes(hurray.float32, 0) == 0

Errors

import hurray

try:
    hurray.buffer_size_bytes(hurray.Dtype.from_tag(0xF2), 10)
except hurray.InvalidDescriptorError:
    pass

assert hurray.ExtensionType(bit_width=24).buffer_size_bytes(10) == 30
float16 = float16
bfloat16 = bfloat16
float32 = float32
float64 = float64
int8 = int8
uint8 = uint8
int16 = int16
uint16 = uint16
int32 = int32
uint32 = uint32
int64 = int64
uint64 = uint64
bool = bool
class Device:

A Hurray device descriptor: a (kind, device_id, memory_class) triple.

Device objects are immutable (frozen) and hashable — safe to use as dict keys.

Constructor

hurray.Device(kind, device_id=0, memory_class="standard")
Parameter Type Default Accepted values
kind str — "cpu", "cuda", "rocm", "metal", "vulkan", "webgpu", "hexagon", "level_zero", "opencl"
device_id int 0 Non-negative integer
memory_class str "standard" "standard", "host_pinned", "unified", "peer"

Examples (Python)

import hurray

cpu = hurray.Device("cpu")
assert cpu.kind == "cpu"
assert cpu.device_id == 0
assert cpu.memory_class == "standard"

gpu = hurray.Device("cuda", 1)
assert gpu.kind == "cuda"
assert gpu.device_id == 1

# Well-known constants
assert hurray.Device("cpu") == hurray.device.cpu

# Hashable
d = {hurray.device.cpu: "cpu0"}
assert d[hurray.Device("cpu")] == "cpu0"
memory_class

The memory access class string (e.g. "standard", "unified").

Examples

assert hurray.Device("cpu").memory_class == "standard"
memory_class_tag

The memory class's wire byte, for the same reason as Device.tag.

Examples

import hurray

assert hurray.Device("cuda", 0, "unified").memory_class_tag == 0x02
assert hurray.Device("cuda", 0, 0xF1).memory_class_tag == 0xF1
is_private

Whether this device's tag is in the private range 0xF0–0xFE.

A private tag means an agreement between one producer and one consumer, so a reader that does not hold that agreement should refuse the tensor rather than guess at it.

Examples

import hurray

assert hurray.Device(0xF2).is_private
assert not hurray.Device("cuda").is_private
tag

The device tag's wire byte.

The only thing that distinguishes one private device from another: kind reports "private" for every tag in 0xF0–0xFE, because the spec gives them no names.

Examples

import hurray

assert hurray.Device("cuda").tag == 0x01
assert hurray.Device(0xF2).tag == 0xF2
kind

The device kind string (e.g. "cpu", "cuda").

Examples

assert hurray.Device("cuda", 1).kind == "cuda"
device_id

The device ordinal (0-based index within devices of the same kind).

Examples

assert hurray.Device("cuda", 2).device_id == 2
class Tensor:

A Hurray tensor: element type, shape, device, and a data buffer.

Construction

hurray.Tensor(buffer, dtype, shape, device=None)
Parameter Type Default
buffer bytes or bytearray —
dtype hurray.Dtype —
shape list[int] —
device hurray.Device hurray.device.cpu

Zero-copy interop

Use hurray.from_numpy or hurray.from_torch to create tensors that share the source buffer without copying. The Tensor holds a strong Python reference to the source object so its buffer remains valid.

Examples (Python)

import struct, hurray

buf = struct.pack("6f", 1.0, 2.0, 3.0, 4.0, 5.0, 6.0)
t = hurray.Tensor(buf, hurray.float32, [2, 3])

assert t.shape == (2, 3)
assert t.ndim == 2
assert t.size == 6
assert t.dtype == hurray.float32
assert t.device.kind == "cpu"
def buffer(self, /, index):

A 1-D uint8 view over the buffer at descriptor index index.

The generic way to reach any buffer, including the ones with no named accessor: CSF has 2 * rank + 1 buffers and block-paged has three. Ask the tensor's layout what each index holds.

uint8 is the only honest element type here — the buffers of one tensor differ, with values taking the tensor's dtype, index buffers uint64, and MXFP scales e8m0 — and it cannot misreport any of them. The view covers exactly the buffer's declared byte size.

Errors

  • IndexError — no buffer at that index.

Examples

import struct, hurray

csr = hurray.Tensor(
    struct.pack("2f", 5.0, 7.0), hurray.float32, [2, 2],
    aux_buffers=[struct.pack("2Q", 0, 1), struct.pack("3Q", 0, 1, 2)],
    layout=hurray.CsrLayout(nnz=2),
)
assert csr.buffer(2).shape == (24,)      # row_ptr: 3 uint64 entries
assert csr.buffer(2)hurray.dtype == hurray.uint8
def to_scipy(self, /):

Convert a CSR or CSC tensor to the matching scipy.sparse matrix (zero-copy where possible).

copy=False is passed to the SciPy constructor. SciPy may copy internally if it does not accept the uint64 index dtype — this is outside Hurray's control and is documented here so callers can plan accordingly.

Raises

  • ImportError — SciPy is not installed.
  • hurray.UnsupportedError — values dtype is Tier 2 / quantized (SciPy has no equivalent) (D17).
  • hurray.UnsupportedError — any layout other than CSR or CSC. For COO, use .values / .indices directly and construct scipy.sparse.coo_matrix.

Examples

import scipy.sparse, hurray

# (Assuming `sparse` is a hurray.Tensor with layout == "csr")
m = sparse.to_scipy()
assert isinstance(m, scipy.sparse.csr_matrix)
def to_torch(self, /):

Return a torch.Tensor sharing this tensor's buffer (zero-copy via DLPack).

torch is imported at call time — import hurray does not require PyTorch to be installed (D7).

Errors

  • ImportError — PyTorch is not installed.
  • builtins.BufferError — element type not representable in DLPack.
  • hurray.UnsupportedError — device/layout not supported via DLPack.

Examples

import hurray
t = hurray.Tensor(bytes(16), hurray.float32, [4])
torch_t = t.to_torch()
buffer_handles

This tensor's buffer table: what each buffer declares about itself.

One hurray.BufferHandle per buffer, in descriptor order, so len(t.buffer_handles) == t.buffer_count. A handle is five scalars — it holds no reference to the tensor and none to the bytes, so keeping one costs nothing and pins nothing.

Use buffer(i) to read a buffer's bytes and buffer_handles[i] to ask about them. The split is deliberate: on a non-CPU tensor the second must work where the first cannot (ADR-037 § 2).

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
(handle,) = t.buffer_handles
assert handle.byte_size == 16
assert handle.device is t.device
size

Total number of logical elements, or None if any dimension is dynamic.

Examples

assert hurray.Tensor(buf, hurray.float32, [2, 3]).size == 6
values

A hurray.Tensor view over the stored values, for layouts that store them separately from their index structure.

The view borrows this tensor's buffer — no copy — and keeps it alive.

Errors

  • AttributeError — the layout has no separate values buffer.

Examples

import numpy as np, hurray

t = hurray.sparse_coo(
    np.array([5.0, 7.0], dtype=np.float32),
    np.array([[0, 0], [1, 1]], dtype=np.uint64),
    [2, 2],
)
assert t.values.shape == (2,)
buffer_count

Number of buffers this tensor holds, including the data buffer.

1 for an ordinary dense tensor; more when the descriptor references quantization scales, sparse index arrays, or a page table.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
assert t.buffer_count == 1
quantization

The quantization scheme, or None if the tensor is not quantized.

Returns the same class you would pass to the constructor — PerTensorAffine, PerChannelAffine, PerBlockAffine, NF4, or MXFP — so a descriptor read off disk or off the wire can be inspected, and passed straight back to build another tensor.

Errors

Examples

import struct, hurray

t = hurray.Tensor(
    bytes(8), hurray.int8, [2, 4],
    aux_buffers=[struct.pack("2f", 0.02, 0.017)],
    quantization=hurray.PerChannelAffine.symmetric(0, 1),
)

q = t.quantization
assert q.axis == 0
assert q.scale_buffer_index == 1

assert hurray.Tensor(bytes(16), hurray.float32, [4]).quantization is None
shard

The shard descriptor, or None if this tensor is not a shard.

Examples

import hurray

t = hurray.Tensor(
    bytes(24), hurray.float32, [2, 3],
    shard=hurray.Shard([4, 3], [2, 0]),
)
assert t.shard.parent_shape == (4, 3)
assert hurray.Tensor(bytes(16), hurray.float32, [4]).shard is None
dtype

The element type of this tensor.

Examples

assert t.dtype == hurray.float32
shape

The shape of this tensor as a tuple of int (or None for dynamic dims).

Dynamic dimensions (DYNAMIC = u64::MAX in the wire format) are mapped to None. All other dimensions are returned as Python int.

Examples

t = hurray.Tensor(buf, hurray.float32, [2, 3])
assert t.shape == (2, 3)
nnz

Number of stored non-zero elements, for layouts that track it.

Errors

  • AttributeError — the layout has no notion of nnz (D10 discipline: inapplicable attributes behave as absent, so hasattr reports the truth).

Examples

import numpy as np, hurray

t = hurray.sparse_coo(
    np.array([5.0, 7.0], dtype=np.float32),
    np.array([[0, 0], [1, 1]], dtype=np.uint64),
    [2, 2],
)
assert t.nnz == 2
assert not hasattr(hurray.Tensor(bytes(16), hurray.float32, [4]), "nnz")
statistics

The statistics section, or None if the tensor carries none.

Examples

import hurray

t = hurray.Tensor(
    bytes(24), hurray.float32, [2, 3],
    statistics=hurray.Statistics(nnz=6),
)
assert t.statistics.nnz == 6
assert hurray.Tensor(bytes(16), hurray.float32, [4]).statistics is None
extension_type

The extension type section, or None for a standard dtype.

Examples

import hurray

t = hurray.Tensor(
    bytes(12), hurray.Dtype.from_tag(0xF2), [4],
    extension_type=hurray.ExtensionType(bit_width=24, is_signed=True),
)
assert t.extension_type.bit_width == 24
assert hurray.Tensor(bytes(16), hurray.float32, [4]).extension_type is None
ndim

Number of dimensions (rank) of this tensor.

Examples

assert hurray.Tensor(buf, hurray.float32, [2, 3]).ndim == 2
descriptor

This tensor's descriptor: everything it declares, apart from its bytes.

The half of the tensor that travels first and on its own — encode it with descriptor.encode() to put a Hurray tensor inside a container of your own.

Examples

import hurray

tensor = hurray.Tensor(bytes(48), hurray.float32, [3, 4])
wire = tensor.descriptor.encode()

assert hurray.Descriptor.decode(wire) == tensor.descriptor
device

The device this tensor resides on.

Examples

assert t.device.kind == "cpu"
indices

A hurray.Tensor view over the COO index buffer, shape [nnz, rank], uint64.

Errors

  • AttributeError — this is not a COO tensor.

Examples

assert t.indices.shape == (t.nnz, t.ndim)   # COO only
row_indices

A hurray.Tensor view over the CSC row-index buffer, shape [nnz], uint64.

Errors

  • AttributeError — this is not a CSC tensor.

Examples

assert t.row_indices.shape == (t.nnz,)      # CSC only
layout

The memory layout of this tensor, as a hurray.Layout object.

The object carries that layout's parameters — nnz, strides, page_size, and so on — which a string could not. Its name is layout.name: one of "row_major", "col_major", "strided", "tiled", "morton", "hilbert", "coo", "csr", "csc", "csf", "block_paged", "composite", or "extension" for a private or unrecognised layout tag.

Layout is a property of a tensor, not a different kind of object (ADR-031): a COO tensor and a row-major tensor are both hurray.Tensor.

Read-only, and a fresh object each access: assigning a layout would silently reinterpret the existing buffers, and the object holds no reference back to this tensor, so t.layout is t.layout is False while == holds.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
assert t.layout.name == "row_major"
assert isinstance(t.layout, hurray.RowMajorLayout)
assert t.layout == hurray.RowMajorLayout()
col_ptr

A hurray.Tensor view over the CSC column-pointer buffer, shape [ncols + 1], uint64.

Errors

  • AttributeError — this is not a CSC tensor.

Examples

assert t.col_ptr.shape == (t.shape[1] + 1,)  # CSC only
T

Transpose view — not yet implemented.

Raises NotImplementedError; lands in a future pass.

col_indices

A hurray.Tensor view over the CSR column-index buffer, shape [nnz], uint64.

Errors

  • AttributeError — this is not a CSR tensor.

Examples

assert t.col_indices.shape == (t.nnz,)      # CSR only
row_ptr

A hurray.Tensor view over the CSR row-pointer buffer, shape [nrows + 1], uint64.

Errors

  • AttributeError — this is not a CSR tensor.

Examples

assert t.row_ptr.shape == (t.shape[0] + 1,)  # CSR only
def from_numpy(array, *, copy=None):

Wrap a NumPy ndarray as a hurray.Tensor, sharing its buffer when the format permits it.

When the array's base address meets the format's 64-byte alignment floor, the returned Tensor borrows the buffer and holds a strong Python reference to the source ndarray so it stays valid for the Tensor's entire lifetime. When it does not, the bytes are copied into an aligned allocation — see copy below.

Requirements

  • The array MUST be C-contiguous (row-major). Pass numpy.ascontiguousarray(arr) first if the array is Fortran-order or has non-contiguous strides (D5).
  • The array dtype MUST map to a Hurray Tier 1 element type.
  • The array MUST reside on CPU (device 0).

The copy argument (ADR-037 § 6)

buffer-protocol.md § Alignment requires every non-empty buffer to start on a 64-byte boundary, and NumPy does not promise one. A large array served by a fresh mmap never has one — glibc puts a 16-byte chunk header before the pointer — and any other array's address is a matter of heap history. Either way the caller cannot arrange for it, which is why this decision is made per call rather than assumed.

  • copy=None (default) — copy only when the source is under-aligned.
  • copy=False — never copy; raise hurray.BufferError naming the alignment the source actually has, so the cost is visible rather than a silent memcpy.
  • copy=True — always copy.

Errors

Examples

import numpy as np, hurray

arr = np.array([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]], dtype=np.float32)
t = hurray.from_numpy(arr)
assert t.shape == (2, 3)
assert t.dtype == hurray.float32
assert t.device.kind == "cpu"
assert t.buffer_handles[0].alignment >= hurray.MIN_BUFFER_ALIGNMENT
def from_torch(tensor, *, copy=None):

Wrap a torch.Tensor as a hurray.Tensor, sharing its buffer when the format permits it (via DLPack).

torch is resolved at call time — import hurray does not require PyTorch (D7).

copy means what it means in from_numpy, and applies to the array DLPack hands back: PyTorch's CPU allocator is 64-byte aligned for most tensors, so copy=False usually succeeds here where it usually fails for arrays NumPy allocated.

Errors

  • ImportError — PyTorch is not installed.
  • builtins.BufferError — element type not representable in DLPack.
  • hurray.BufferError — copy=False was requested for an under-aligned tensor.

Examples

import torch, hurray

t_torch = torch.zeros(2, 3, dtype=torch.float32)
t = hurray.from_torch(t_torch)
assert t.shape == (2, 3)
assert t.dtype == hurray.float32
def layout_tag_kind(tag):

Which category of the layout tag space tag falls in.

Returns "named", "reserved", "private", or "invalid".

The four call for different reactions, which is the whole reason to ask. A named tag is one this build understands. A reserved tag is one a future version of the format may assign — most likely the producer is newer than this reader, so relaying the tensor on is reasonable while interpreting its bytes is not. A private tag (0xF0–0xFE) belongs to an out-of-band agreement between a particular producer and consumer. An invalid tag can never appear in a conformant descriptor, so it means corruption or a framing error.

One function rather than the four predicates hurray-core exposes: a caller wants to branch on the answer, not ask four yes/no questions in sequence.

Examples

import hurray

assert hurray.layout_tag_kind(0x07) == "named"      # CSR
assert hurray.layout_tag_kind(0x10) == "reserved"
assert hurray.layout_tag_kind(0xF3) == "private"
assert hurray.layout_tag_kind(0x00) == "invalid"

# What a relay does with a tag it does not recognise:
tag = 0x10
if hurray.layout_tag_kind(tag) == "reserved":
    pass    # pass the tensor on; do not touch its buffer
class Layout:

The memory layout of a tensor: a tag plus that layout's parameters.

Layout is the base of every layout class and is not constructible from Python — there is no layout that is only "a layout". It is returned directly only as the fallback for a layout tag this build of hurray does not yet bind, in which case tag and name are still available.

Examples (Python)

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
assert isinstance(t.layout, hurray.Layout)
assert isinstance(t.layout, hurray.RowMajorLayout)
assert t.layout.name == "row_major"
assert t.layout.tag == 0x01
def validate_against_shape(self, /, shape):

Check this layout against a tensor shape, raising if they cannot go together.

Each layout constrains the shapes it can describe: CSR and CSC are rank-2, a tiled layout's tile shape must match the tensor's rank, a block-paged cache needs its paged axis to exist. hurray.Tensor runs this for you at construction — call it directly when you are choosing a layout for a shape you have not built a tensor for yet, or checking a layout you decoded against a shape you intend to use it with.

A dynamic dimension (None) satisfies any extent constraint: there is nothing to check until it is resolved.

Errors

Examples

import hurray

csr = hurray.CsrLayout(nnz=5)
csr.validate_against_shape([4, 5])          # rank-2: fine

try:
    csr.validate_against_shape([2, 3, 4])   # CSR is rank-2 only
except hurray.InvalidDescriptorError:
    pass
buffer_count

The number of buffers this layout requires, or None when that is not statically knowable.

0 for a composite head, which owns no buffers; None for a private or unknown layout, whose buffer requirements only its definer knows.

Examples

import hurray

assert hurray.RowMajorLayout().buffer_count == 1
assert hurray.CsrLayout(nnz=4).buffer_count == 3
assert hurray.UnknownLayout(0x0C, b"").buffer_count is None
name

The layout's name: "row_major", "csr", "block_paged", and so on.

Private and unrecognised tags both report "extension"; use isinstance to tell PrivateExtensionLayout from UnknownLayout, which is the distinction the wire format actually makes.

Examples

import hurray

assert hurray.CooLayout(nnz=2).name == "coo"
is_dense

True if this layout stores elements in a directly addressable order — the layouts DLPack, NumPy and PyTorch can consume.

Examples

import hurray

assert hurray.RowMajorLayout().is_dense
assert not hurray.CsrLayout(nnz=4).is_dense
is_virtual

True if this layout owns no data buffer of its own — only a composite head.

Examples

import hurray

assert not hurray.RowMajorLayout().is_virtual
tag

The layout tag byte, as defined by docs/spec/memory-layout.md.

Examples

import hurray

assert hurray.CsrLayout(nnz=4).tag == 0x07
class RowMajorLayout(hurray.Layout):

Row-major (C-order) layout. Tag 0x01.

The default: element [i, j] is at offset i * ncols + j.

Examples (Python)

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
assert t.layout == hurray.RowMajorLayout()
class ColMajorLayout(hurray.Layout):

Column-major (Fortran-order) layout. Tag 0x02.

Examples (Python)

import hurray

assert hurray.ColMajorLayout().name == "col_major"
class StridedLayout(hurray.Layout):

Strided layout with explicit per-dimension strides. Tag 0x03.

Strides are in logical elements, not bytes, and may be negative (a reversed axis) or zero (a broadcast axis). A reader arriving from NumPy, whose strides are in bytes, will otherwise read them wrongly.

Examples (Python)

import hurray

l = hurray.StridedLayout([4, 1])
assert l.strides == (4, 1)
assert l.name == "strided"
strides

The per-dimension strides, in logical elements.

Examples

import hurray

assert hurray.StridedLayout([4, 1]).strides == (4, 1)
class TiledLayout(hurray.Layout):

Tiled (blocked) layout. Tag 0x04.

Elements are grouped into fixed-size tiles; outer_layout orders the tiles and inner_layout orders the elements within a tile. A tiled layout may nest another tiled layout as its inner layout, up to the depth bound core enforces.

Examples (Python)

import hurray

l = hurray.TiledLayout([4, 4], inner_layout="col_major")
assert l.tile_shape == (4, 4)
assert l.outer_layout == "row_major"
assert l.inner_layout == "col_major"
inner_layout

How elements are ordered within a tile, as a lowercase name.

Examples

import hurray

assert hurray.TiledLayout([8, 8], inner_layout="col_major").inner_layout == "col_major"
tile_shape

The tile extent along each dimension.

Examples

import hurray

assert hurray.TiledLayout([8, 8]).tile_shape == (8, 8)
outer_layout

How tiles are ordered relative to each other, as a lowercase name.

Examples

import hurray

assert hurray.TiledLayout([8, 8]).outer_layout == "row_major"
outer_strides

Tile-level strides in logical elements, or None unless the outer layout is "strided".

Examples

import hurray

assert hurray.TiledLayout([8, 8]).outer_strides is None
inner_strides

Within-tile strides in logical elements, or None unless the inner layout is "strided".

Examples

import hurray

assert hurray.TiledLayout([8, 8]).inner_strides is None
inner_tiled

The nested tiling, or None unless the inner layout is "tiled".

Examples

import hurray

outer = hurray.TiledLayout(
    [64, 64], inner_layout="tiled", inner_tiled=hurray.TiledLayout([8, 8])
)
assert outer.inner_tiled.tile_shape == (8, 8)
class MortonLayout(hurray.Layout):

Morton (Z-order curve) layout. Tag 0x05.

morton_bits[k] is the number of index bits interleaved for dimension k, so dimension k may be at most 2 ** morton_bits[k] long.

Examples (Python)

import hurray

l = hurray.MortonLayout([3, 3])
assert l.morton_bits == (3, 3)
morton_bits

The number of interleaved index bits per dimension.

Examples

import hurray

assert hurray.MortonLayout([3, 3]).morton_bits == (3, 3)
class HilbertLayout(hurray.Layout):

Hilbert curve layout. Tag 0x40.

Every dimension must be exactly 2 ** hilbert_order long, and hilbert_rank must equal the tensor's rank.

Examples (Python)

import hurray

l = hurray.HilbertLayout(3, 2)
assert l.hilbert_order == 3
assert l.hilbert_rank == 2
hilbert_order

The curve order: each dimension is 2 ** hilbert_order long.

Examples

import hurray

assert hurray.HilbertLayout(3, 2).hilbert_order == 3
hilbert_rank

The curve rank, which must equal the tensor's rank.

Examples

import hurray

assert hurray.HilbertLayout(3, 2).hilbert_rank == 2
class CooLayout(hurray.Layout):

Coordinate-list sparse layout. Tag 0x06. Two buffers: values, then indices.

Examples (Python)

import hurray

l = hurray.CooLayout(nnz=2, is_sorted=True)
assert l.nnz == 2
assert l.is_sorted
assert l.buffer_count == 2
is_sorted

Whether the coordinates are stored in lexicographic order.

Examples

import hurray

assert hurray.CooLayout(nnz=7, is_sorted=True).is_sorted
nnz

The number of stored non-zero elements.

Examples

import hurray

assert hurray.CooLayout(nnz=7).nnz == 7
class CsrLayout(hurray.Layout):

Compressed-sparse-row layout. Tag 0x07. Rank 2 only.

Three buffers: values, column indices, row pointers.

Examples (Python)

import hurray

l = hurray.CsrLayout(nnz=4)
assert l.nnz == 4
assert l.buffer_count == 3
nnz

The number of stored non-zero elements.

Examples

import hurray

assert hurray.CsrLayout(nnz=4).nnz == 4
class CscLayout(hurray.Layout):

Compressed-sparse-column layout. Tag 0x08. Rank 2 only.

Three buffers: values, row indices, column pointers.

Examples (Python)

import hurray

l = hurray.CscLayout(nnz=4)
assert l.nnz == 4
assert l.name == "csc"
nnz

The number of stored non-zero elements.

Examples

import hurray

assert hurray.CscLayout(nnz=4).nnz == 4
class CsfLayout(hurray.Layout):

Compressed-sparse-fiber layout. Tag 0x09. Rank 3 and above.

The rank-N generalisation of CSR/CSC: a tree of rank levels, each contributing a pos and a crd buffer, plus one values buffer — 2 * rank + 1 in total. Those buffers have no named accessors on hurray.Tensor; reach them with t.buffer(index), in the order this layout describes.

Examples (Python)

import hurray

l = hurray.CsfLayout(nnz=5, mode_order=[0, 1, 2])
assert l.mode_order == (0, 1, 2)
assert l.buffer_count == 7
nnz

The number of stored non-zero elements.

Examples

import hurray

assert hurray.CsfLayout(nnz=5, mode_order=[0, 1, 2]).nnz == 5
mode_order

The dimension nesting order.

Examples

import hurray

assert hurray.CsfLayout(nnz=5, mode_order=[2, 0, 1]).mode_order == (2, 0, 1)
class BlockPagedLayout(hurray.Layout):

Block-paged indirect layout for a PagedAttention KV cache. Tag 0x0A.

Three buffers: the page pool, the block table, and the sequence lengths. They have no named accessors on hurray.Tensor; reach them with t.buffer(index).

Examples (Python)

import hurray

l = hurray.BlockPagedLayout(page_size=16, num_pages=64, paged_axis=0, num_seqs=2)
assert l.page_size == 16
assert l.kv_role == "key"
assert l.buffer_count == 3
def validate_index_buffers(self, /, seq_ptr, block_table):

Check this cache's two index buffers against the four storage invariants.

A block-paged tensor's block_table and seq_ptr are the only thing standing between a consumer and an out-of-bounds read, and nothing about their contents is checked when the descriptor is built — the descriptor describes them, it does not contain them. So a producer assembling a KV cache checks them here, before shipping.

The invariants, from block-paged.md § Storage:

  1. seq_ptr[0] == 0
  2. seq_ptr is non-decreasing
  3. seq_ptr[num_seqs] == len(block_table)
  4. every block_table[k] < num_pages

num_pages and num_seqs come from the layout, so only the two buffers are passed. An empty batch (num_seqs == 0, seq_ptr == [0]) is valid.

Aliasing is not an error: two sequences naming the same page is how a shared prefix is represented, and it is the point of the layout.

Errors

Examples

import hurray

layout = hurray.BlockPagedLayout(page_size=4, num_pages=5, paged_axis=0, num_seqs=2)

# Sequence 1's page 0 aliases sequence 0's page 0 — a shared prefix.
layout.validate_index_buffers(seq_ptr=[0, 2, 3], block_table=[0, 1, 0])

try:
    layout.validate_index_buffers(seq_ptr=[0, 2, 3], block_table=[0, 5, 0])
except hurray.InvalidDescriptorError:
    pass    # page 5 == num_pages, one past the end of the pool
def validate_quantization_compatibility(self, /, quantization):

Check a quantization scheme against this cache's paging.

A paged KV cache is usually fp8 or int8, and the two interact: per-block-affine requires axis == 0 and block_size == page_size, so that scales stay per-page-slot and a shared page carries its own. Per-channel must not be applied to the paged axis at all.

Takes the quantization object rather than a scheme tag: the caller already has one, and a raw 0x03 is a byte to look up rather than a thing to pass.

Errors

Examples

import hurray

layout = hurray.BlockPagedLayout(page_size=4, num_pages=5, paged_axis=0, num_seqs=2)

layout.validate_quantization_compatibility(
    hurray.PerBlockAffine.symmetric(axis=0, block_size=4, scale_buffer_index=3)
)

try:
    layout.validate_quantization_compatibility(
        hurray.PerBlockAffine.symmetric(axis=0, block_size=8, scale_buffer_index=3)
    )
except hurray.InvalidDescriptorError:
    pass    # block_size 8 != page_size 4
block_table_index_type

The element type of the block table: "uint32" or "uint64".

Examples

import hurray

assert hurray.BlockPagedLayout(16, 64, 0, 2).block_table_index_type == "uint32"
num_seqs

Number of sequences sharing the page pool.

Examples

import hurray

assert hurray.BlockPagedLayout(16, 64, 0, 2).num_seqs == 2
kv_role

Which half of the KV cache this tensor holds: "key", "value", or "fused".

Examples

import hurray

assert hurray.BlockPagedLayout(16, 64, 0, 2, kv_role="value").kv_role == "value"
page_size

Number of tokens per page.

Examples

import hurray

assert hurray.BlockPagedLayout(16, 64, 0, 2).page_size == 16
num_pages

Number of pages in the shared pool.

Examples

import hurray

assert hurray.BlockPagedLayout(16, 64, 0, 2).num_pages == 64
paged_axis

The axis divided into pages.

Examples

import hurray

assert hurray.BlockPagedLayout(16, 64, 0, 2).paged_axis == 0
layer_index

The transformer layer this cache belongs to, or None if unstated.

Examples

import hurray

assert hurray.BlockPagedLayout(16, 64, 0, 2).layer_index is None
class CompositeLayout(hurray.Layout):

Composite (virtual) head. Tag 0x0B. Owns no buffers.

A composite head presents one logical view over an ordered set of member tensors bound by stream adjacency. It is readable here so that a head decoded from a stream reports its own layout truthfully; building a hurray.Tensor with it raises hurray.UnsupportedError, because the Python Tensor cannot yet represent a tensor that owns no buffers.

Examples (Python)

import hurray

l = hurray.CompositeLayout("overlay", member_count=3, combine_op="add")
assert l.composition_rule == "overlay"
assert l.combine_op == "add"
assert l.buffer_count == 0
assert l.is_virtual
member_count

The number of member tensors this head composes.

Examples

import hurray

assert hurray.CompositeLayout("group", 4).member_count == 4
composition_rule

The composition rule: "partition", "overlay", or "group".

Examples

import hurray

assert hurray.CompositeLayout("partition", 2).composition_rule == "partition"
combine_op

The combine operation for an overlay, or None for any other rule.

Kept separate from composition_rule rather than flattened into one string: for a partition or a group the operation is not merely unset, it does not apply.

Examples

import hurray

assert hurray.CompositeLayout("overlay", 2, combine_op="replace").combine_op == "replace"
assert hurray.CompositeLayout("partition", 2).combine_op is None
class PrivateExtensionLayout(hurray.Layout):

An implementation-private layout. Tags 0xF0–0xFE.

The buffer count is unknown, so a tensor built with this layout skips the buffer-size check every other layout gets: nothing in the descriptor says how large its buffers should be. That hole is intrinsic to a private tag.

Examples (Python)

import hurray

l = hurray.PrivateExtensionLayout(0xF0, extension_layout_id=7, extension_data=b"\x01")
assert l.tag == 0xF0
assert l.extension_layout_id == 7
assert l.buffer_count is None
extension_layout_id

The private extension identifier.

Examples

import hurray

assert hurray.PrivateExtensionLayout(0xF0, 7, b"").extension_layout_id == 7
extension_data

The opaque payload, as bytes.

Examples

import hurray

assert hurray.PrivateExtensionLayout(0xF0, 7, b"\x01\x02").extension_data == b"\x01\x02"
class UnknownLayout(hurray.Layout):

A layout tag this implementation does not recognise, accepted in permissive mode.

Constructible so that a permissive relay can rebuild a descriptor it decoded and write it back out unchanged. Like a private layout, its buffer count is unknown, so a tensor built with it skips the buffer-size check.

Examples (Python)

import hurray

l = hurray.UnknownLayout(0x0C, b"\x00\x01")
assert l.tag == 0x0C
assert l.raw_bytes == b"\x00\x01"
assert l.name == "extension"
raw_bytes

The unrecognised tag byte.

Examples

import hurray

assert hurray.UnknownLayout(0x0C).tag == 0x0C
def sparse_coo(values, indices, shape, *, copy=None):

Construct a COO Tensor from packed component arrays, zero-copy.

values is a 1-D array of nnz elements. indices is a 2-D uint64 array of shape [nnz, rank] giving each non-zero's coordinates in row-major (C-contiguous) order — Hurray's packed COO layout. shape is the dense tensor shape; its length is the rank and must equal indices.shape[1].

Both arrays are shared without copying; the returned tensor holds strong references to keep them alive. scipy.sparse.coo_matrix stores row/col as two arrays, so it is not accepted directly — repack first: indices = numpy.stack([m.row, m.col], axis=1).astype(numpy.uint64).

Examples

import numpy as np, hurray

values = np.array([5.0, 7.0], dtype=np.float32)
indices = np.array([[0, 0], [1, 1]], dtype=np.uint64)  # [nnz, rank]
t = hurray.sparse_coo(values, indices, [2, 2])
assert t.layout == "coo"
assert t.nnz == 2
def from_scipy(matrix, *, copy=None):

Wrap a scipy.sparse matrix as a hurray.Tensor without copying.

Supported formats

SciPy type Hurray format
csr_matrix / csr_array CSR
csc_matrix / csc_array CSC

COO format is not supported via zero-copy from_scipy because SciPy stores COO row/col as two separate arrays while Hurray requires a single packed [nnz, rank] uint64 index buffer (D19). Repack first:

indices = np.stack([m.row, m.col], axis=1).astype(np.uint64)

Then construct the hurray.Tensor directly from the component buffers.

Index dtype requirement (D16)

Hurray's CSR/CSC spec requires uint64 index arrays. SciPy typically uses int32 or int64. If the index arrays are not already uint64, this function raises hurray.UnsupportedError with a cast instruction.

Errors

The copy argument (ADR-037 § 6)

As in from_numpy, and applied to each of .data, .indices and .indptr independently: copy=None (default) copies only the components whose address misses the format's 64-byte floor, copy=False refuses instead of copying, copy=True copies all three.

Examples

import numpy as np, scipy.sparse, hurray

m = scipy.sparse.csr_matrix(
    np.array([[1.0, 0.0], [0.0, 2.0]], dtype=np.float32)
)
# Cast index arrays to uint64 first (D16).
m.indices = m.indices.astype(np.uint64)
m.indptr  = m.indptr.astype(np.uint64)

sparse = hurray.from_scipy(m)
assert sparse.layout == "csr"
assert sparse.nnz == 2
class StreamReader:

Reads tensors from a Hurray stream, one at a time.

Iterating yields a hurray.Tensor per tensor on the wire and stops at a clean end of stream. Nothing buffers the whole input: a tensor is available as soon as its descriptor and buffers have arrived, which is the property the streaming format exists for.

Sources

A filesystem path, an object with fileno() (a socket, a pipe, an open file), or bytes. An object with no descriptor — io.BytesIO — should be passed as .getvalue().

Examples (Python)

import hurray

for tensor in hurray.StreamReader("tensors.hrry"):
    print(tensor.shape, tensor.dtype)
def close(self, /):

Release the transport now rather than at collection.

class StreamWriter:

Writes tensors to a Hurray stream, one at a time.

finish flushes the transport, and a caller who forgets it loses whatever was still buffered — so the writer is a context manager and closing is automatic on the happy path. finish() is also available explicitly, and is idempotent.

Destinations

A filesystem path, an object with fileno(), or nothing at all — in which case the stream is built in memory and returned by getvalue().

Examples (Python)

import hurray

with hurray.StreamWriter() as writer:
    writer.write(hurray.Tensor(bytes(16), hurray.float32, [4]))
# writer.getvalue() holds the encoded stream
def write(self, /, item):

Write one tensor, with every buffer its descriptor references.

Errors

Examples

import hurray

with hurray.StreamWriter() as writer:
    writer.write(hurray.Tensor(bytes(16), hurray.float32, [4]))
def write_composite(self, /, composite):

Emits a composite as head-then-members, the wire's own binding rule.

def finish(self, /):

Finish the stream, writing its terminator.

Idempotent: finishing an already-finished stream is a no-op, so the explicit call and the context manager compose.

Examples

import hurray

writer = hurray.StreamWriter()
writer.finish()
writer.finish()                       # no-op, not an error
def getvalue(self, /):

The encoded stream, for a writer with no destination.

Repeatable, like io.BytesIO.getvalue(): the bytes stay in the writer, so calling it twice returns the same stream both times.

Errors

  • hurray.StreamError — this writer has a destination, so there is nothing to return; the bytes went there.

Examples

import hurray

with hurray.StreamWriter() as writer:
    writer.write(hurray.Tensor(bytes(16), hurray.float32, [4]))

assert len(writer.getvalue()) > 0
assert writer.getvalue() == writer.getvalue()
def from_hurray(obj):

Accept any object whose __hurray__() returns a valid "hurray_tensor" capsule and return a new hurray.Tensor that shares the buffer without copying.

Protocol

  1. Calls obj.__hurray__() to obtain the capsule.
  2. Verifies the capsule name is "hurray_tensor" and the ABI version matches.
  3. Renames the capsule to "used_hurray_tensor" (prevents double-destroy).
  4. Extracts the data pointer and byte size from the HurrayBuffer handle.
  5. Calls hurray_buffer_destroy (frees the handle; no release callback was set).
  6. Decodes the TensorDescriptor from the capsule context.
  7. Creates a new Tensor with BufferStore::Borrowed whose base keeps the source Tensor alive (D-NB3).

Errors

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
t2 = hurray.from_hurray(t)
assert t2.shape == t.shape
assert t2.dtype == t.dtype
def aligned_allocator():

Allocate NumPy arrays 64-byte aligned for the duration of a with block.

Arrays allocated inside the block satisfy the format's alignment floor, so hurray.from_numpy(arr, copy=False) shares their buffer instead of copying it. Arrays allocated outside are untouched.

The handler is stored per array, so an array allocated inside the block is freed through the matching deallocator long after the block exits. It is also thread- and context-local, so installing it cannot leak into unrelated code — with one consequence worth knowing: a thread started inside the block does not inherit it, and arrays that thread allocates get NumPy's default allocator and will be copied on ingest like any other.

Blocks nest: each restores the handler that was in place when it was entered.

Errors

Examples

import numpy as np, hurray

with hurray.aligned_allocator():
    weights = np.zeros((512, 512), dtype=np.float32)

tensor = hurray.from_numpy(weights, copy=False)
assert tensor.buffer_handles[0].alignment >= hurray.MIN_BUFFER_ALIGNMENT
class AlignedAllocatorCtx:

The context manager returned by aligned_allocator.

Examples

import numpy as np, hurray

with hurray.aligned_allocator():
    arr = np.zeros(1 << 20, dtype=np.float32)

assert arr.__array_interface__["data"][0] % hurray.MIN_BUFFER_ALIGNMENT == 0
t = hurray.from_numpy(arr, copy=False)      # no copy, and none was needed
class BufferHandle:

One row of a tensor's buffer table: what a buffer declares about itself.

Not constructible from Python — there is no field a caller could supply that the buffers do not already settle (ADR-037 § 4).

Examples (Python)

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
handle = t.buffer_handles[0]

assert handle.byte_size == 16
assert handle.alignment >= hurray.MIN_BUFFER_ALIGNMENT
assert handle.sync_mode == "producer_synced"
assert handle.device is t.device        # colocation: one device per descriptor
alignment

The alignment of the buffer's base address, in bytes.

Measured, never assumed: a buffer borrowed from NumPy declares what its address actually satisfies, and an owned buffer is allocated over-aligned so that the declaration is true. Always a power of two, and at least hurray.MIN_BUFFER_ALIGNMENT for a non-empty buffer.

Examples

import hurray

t = hurray.Tensor(bytes(4096), hurray.float32, [1024])
assert t.buffer_handles[0].alignment >= 64
byte_size

The buffer's declared size in bytes.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
assert t.buffer_handles[0].byte_size == 16
sync_mode

When the buffer may be read: "producer_synced", "event", or "consumer_stream".

Anything this binding constructs is "producer_synced" — the interpreter cannot enqueue device work through this API, so it cannot promise anything else. A tensor decoded from a stream or a file reports what the producer declared, and a buffer that is not "producer_synced" will refuse the paths that hand out its bytes until the binding can honour the wait.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
assert t.buffer_handles[0].sync_mode == "producer_synced"
is_empty

Whether this buffer declares zero bytes.

Examples

import hurray

assert not hurray.Tensor(bytes(16), hurray.float32, [4]).buffer_handles[0].is_empty
device

The device this buffer lives on.

Returns the tensor's own Device object: buffer-protocol.md § Device Colocation requires every buffer of one descriptor to share a device and memory class, so there is exactly one to report and no difference to branch on.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
assert t.buffer_handles[0]hurray.device is t.device
MIN_BUFFER_ALIGNMENT = 64
PAGE_ALIGNMENT = 4096
class Composite:

A composite tensor: a head presenting one logical view over ordered members.

Examples (Python)

import hurray, struct

tile = hurray.Tensor(bytes(128), hurray.float32, [8, 4], shard=hurray.Shard([8, 8], [0, 0]))
other = hurray.Tensor(bytes(128), hurray.float32, [8, 4], shard=hurray.Shard([8, 8], [0, 4]))

composite = hurray.Composite(
    "partition", shape=[8, 8], dtype=hurray.float32, members=[tile, other]
)
assert composite.member_count == 2
assert composite.layout.composition_rule == "partition"
member_roles

Each member's role, for an overlay: "base" then "correction", in wire order.

None for every member of a partition or a group, where the rule assigns no roles.

The role belongs to membership, not to the tensor — a tensor has no role, a tensor inside an overlay does — so it is read from the composite rather than from members[i]. That also makes an authored overlay and a decoded one answer identically, which a per-tensor accessor could not: the caller's own tensor object never carries a role, since the composite supplies it.

Examples

import hurray

group = hurray.Composite(
    "group", shape=[4], dtype=hurray.float32,
    members=[hurray.Tensor(bytes(16), hurray.float32, [4])],
)
assert group.member_roles == (None,)
shape

The logical shape the head presents.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
c = hurray.Composite("group", shape=[4], dtype=hurray.float32, members=[t])
assert c.shape == (4,)
member_count

How many members this head declares.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
c = hurray.Composite("group", shape=[4], dtype=hurray.float32, members=[t])
assert c.member_count == 1
dtype

The element type the head presents.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
c = hurray.Composite("group", shape=[4], dtype=hurray.float32, members=[t])
assert c.dtype == hurray.float32
members

The members, in wire order.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
c = hurray.Composite("group", shape=[4], dtype=hurray.float32, members=[t])
assert len(c.members) == 1
layout

The head's layout, as a hurray.CompositeLayout.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
c = hurray.Composite("group", shape=[4], dtype=hurray.float32, members=[t])
assert c.layout.composition_rule == "group"
assert c.layout.combine_op is None
ndim

Number of dimensions of the head's logical view.

Examples

import hurray

t = hurray.Tensor(bytes(16), hurray.float32, [4])
c = hurray.Composite("group", shape=[4], dtype=hurray.float32, members=[t])
assert c.ndim == 1
descriptor

The head descriptor: what this composite presents, owning no buffers.

The one descriptor that is only a descriptor — a composite head declares a shape, a dtype and a composition rule, and its data lives in its members.

Examples

import hurray

group = hurray.Composite(
    "group", shape=[4], dtype=hurray.float32,
    members=[hurray.Tensor(bytes(16), hurray.float32, [4])],
)
assert group.descriptor.buffer_count == 0
assert hurray.Descriptor.decode(group.descriptor.encode()) == group.descriptor
def zeros(shape, *, dtype=None, device=None):

Return a new tensor of the given shape, filled with zeros.

Default dtype is float64. Strict mode only.

Examples

import hurray

t = hurray.zeros([3, 4])
assert t.shape == (3, 4)
assert t.dtype == hurray.float64
def ones(shape, *, dtype=None, device=None):

Return a new tensor of the given shape, filled with ones.

Default dtype is float64. Strict mode only.

Examples

import hurray

t = hurray.ones([2, 3], dtype=hurray.float32)
assert t.shape == (2, 3)
assert t.dtype == hurray.float32
def full(shape, fill_value, *, dtype=None, device=None):

Return a new tensor of the given shape, filled with fill_value.

If dtype is None, it is inferred from fill_value (bool→Bool, complex→Complex128, float→Float64, int→Int64). Strict mode only.

Examples

import hurray

t = hurray.full([2, 2], 3.14, dtype=hurray.float32)
assert t.shape == (2, 2)
assert t.dtype == hurray.float32
def empty(shape, *, dtype=None, device=None):

Return a new uninitialized tensor of the given shape and type.

The buffer is zero-initialized (the contents of an "empty" tensor are unspecified). Default dtype is float64. Tier 1 only.

Examples

import hurray

t = hurray.empty([4, 4], dtype=hurray.float64)
assert t.shape == (4, 4)
def zeros_like(x, *, dtype=None, device=None):

Return a zeros tensor with the same shape and dtype as x.

dtype and device override the source tensor's values when given.

Examples

import hurray

src = hurray.ones([3, 3], dtype=hurray.float32)
z = hurray.zeros_like(src)
assert z.shape == (3, 3)
assert z.dtype == hurray.float32
def ones_like(x, *, dtype=None, device=None):

Return an all-ones tensor with the same shape and dtype as x.

dtype and device override the source tensor's values when given.

Examples

import hurray

src = hurray.zeros([2, 2], dtype=hurray.int32)
o = hurray.ones_like(src)
assert o.shape == (2, 2)
assert o.dtype == hurray.int32
def full_like(x, fill_value, *, dtype=None, device=None):

Return a tensor filled with fill_value with the same shape and dtype as x.

dtype and device override the source tensor's values when given.

Examples

import hurray

src = hurray.zeros([4], dtype=hurray.float32)
f = hurray.full_like(src, 7.0)
assert f.shape == (4,)
assert f.dtype == hurray.float32
def empty_like(x, *, dtype=None, device=None):

Return an uninitialized tensor with the same shape and dtype as x.

dtype and device override the source tensor's values when given.

Examples

import hurray

src = hurray.ones([3], dtype=hurray.float64)
e = hurray.empty_like(src)
assert e.shape == (3,)
assert e.dtype == hurray.float64
def arange(start, stop=None, step=None, *, dtype=None, device=None):

Return evenly spaced values over a given interval.

arange(stop) → values [0, 1, …, stop). arange(start, stop, step) → values [start, start+step, …).

Dtype is inferred as int64 if all of start, stop, step are Python integers; float64 otherwise. Pass an explicit dtype to override.

Examples

import hurray

t = hurray.arange(5)
assert t.shape == (5,)
assert t.dtype == hurray.int64

t2 = hurray.arange(0.0, 1.0, 0.25)
assert t2.shape == (4,)
assert t2.dtype == hurray.float64
def linspace(start, stop, num, *, dtype=None, device=None, endpoint=True):

Return num evenly spaced values in the closed interval [start, stop].

Default dtype is float64. Pass endpoint=False to exclude stop. Strict mode only.

Examples

import hurray

t = hurray.linspace(0.0, 1.0, 5)
assert t.shape == (5,)
assert t.dtype == hurray.float64
def eye(n_rows, n_cols=None, *, k=0, dtype=None, device=None):

Return a 2-D tensor with ones on the k-th diagonal and zeros elsewhere.

Default dtype is float64. Strict mode only.

Examples

import hurray

t = hurray.eye(3)
assert t.shape == (3, 3)
assert t.dtype == hurray.float64
def asarray(obj, *, dtype=None, device=None, copy=None):

Convert an object to a hurray.Tensor.

Input types handled:

  • hurray.Tensor — returned as-is (or copied if copy=True).
  • NumPy arrays — wrapped zero-copy via from_numpy.
  • Objects with __dlpack__ — wrapped zero-copy via DLPack.
  • Python lists, tuples, scalars — converted via numpy.asarray first.

Errors

Examples

import numpy as np, hurray

t = hurray.asarray([1.0, 2.0, 3.0])
assert t.shape == (3,)
assert t.dtype == hurray.float64
def from_dlpack(x, *, device=None, copy=None):

Construct a tensor from a DLPack capsule or an object with __dlpack__.

The from_dlpack entry point. x is any object implementing __dlpack__; the exchange is left to the DLPack consumer so that it negotiates device and stream with the producer. The result is wrapped as a hurray.Tensor, sharing the producer's buffer when its address meets the format's 64-byte alignment floor and copying into an aligned allocation when it does not — copy chooses between those, exactly as in hurray.from_numpy. device is accepted for signature compatibility; only CPU is supported in this version.

Errors

Examples

import numpy as np, hurray

arr = np.array([1.0, 2.0, 3.0], dtype=np.float64)
t = hurray.from_dlpack(arr)
assert t.shape == (3,)
assert t.dtype == hurray.float64
class Descriptor:

What a tensor declares about itself, without its data.

Obtained from Tensor.descriptor, Composite.descriptor, or Descriptor.decode — never constructed directly.

Examples (Python)

import hurray

tensor = hurray.Tensor(bytes(48), hurray.float32, [3, 4])
descriptor = tensor.descriptor

assert descriptor.dtype is hurray.float32
assert descriptor.shape == (3, 4)
assert descriptor.layout == hurray.RowMajorLayout()

wire = descriptor.encode()
assert hurray.Descriptor.decode(wire) == descriptor
def encode(self, /):

Encode to the binary descriptor, exactly as it appears on the wire.

Self-delimiting: bytes 6–9 hold the total length, so a reader consumes it without any external framing. Put the result wherever a container of your own has room, and hand the buffers over beside it.

Errors

Examples

import hurray

wire = hurray.Tensor(bytes(48), hurray.float32, [3, 4]).descriptor.encode()
assert isinstance(wire, bytes)
def decode(cls, /, data):

Decode a binary descriptor.

Trailing bytes are permitted and ignored — a descriptor is self-delimiting, so what follows it in a stream is the buffers it describes, not part of it.

Errors

Examples

import hurray

original = hurray.Tensor(bytes(48), hurray.float32, [3, 4]).descriptor
assert hurray.Descriptor.decode(original.encode()) == original
quantization

The quantization scheme, or None.

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.quantization is None
buffer_handles

One handle per buffer, in descriptor order.

Examples

import hurray

handles = hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.buffer_handles
assert handles[0].byte_size == 16
device

The device every buffer lives on (buffer-protocol.md § Device Colocation).

A composite head declares no buffers and so no device; it reports CPU, which is what the wire carries for a descriptor with an empty buffer table.

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.device.kind == "cpu"
layout

The memory layout.

Examples

import hurray

descriptor = hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor
assert descriptor.layout == hurray.RowMajorLayout()
extension_type

The extension type section, or None.

Present exactly when dtype is a private extension type (tag 0xF0–0xFE), which is the only way to learn an extension type's width — the Dtype itself reports 0.

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.extension_type is None

private = hurray.Dtype.from_tag(0xF2)
tensor = hurray.Tensor(
    bytes(12), private, [4],
    extension_type=hurray.ExtensionType(bit_width=24, is_signed=True),
)
assert tensor.descriptor.extension_type.bit_width == 24
buffer_count

How many buffers this descriptor declares. 0 for a composite head.

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.buffer_count == 1
byte_offset

Byte offset from the start of buffer 0 to logical element [0, …, 0].

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.byte_offset == 0
shard

The shard annotation, or None.

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.shard is None
size

The total element count, or None when a dimension is dynamic.

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [2, 2]).descriptor.size == 4
assert hurray.Tensor(b"", hurray.float32, [None, 2]).descriptor.size is None
encoded_len

The number of bytes Descriptor.encode will produce.

The same number the wire's own descriptor_length field carries, which is what makes a descriptor self-delimiting.

Examples

import hurray

descriptor = hurray.Tensor(bytes(48), hurray.float32, [3, 4]).descriptor
assert descriptor.encoded_len == len(descriptor.encode())
shape

The shape, with None for a dynamic dimension.

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.shape == (4,)
statistics

The advisory statistics section, or None.

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.statistics is None
version

The format version this descriptor declares, as (major, minor).

Examples

import hurray

major, minor = hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.version
assert major == 1
ndim

The number of dimensions.

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [2, 2]).descriptor.ndim == 2
dtype

The element type of this tensor's data.

Examples

import hurray

assert hurray.Tensor(bytes(16), hurray.float32, [4]).descriptor.dtype is hurray.float32
def load(path, *, names=None):

Load tensors from a Hurray file.

Opens the HRRYFILE at path and returns a dict mapping tensor names to hurray.Tensor objects. If names is given, only those tensors are loaded; otherwise every tensor in the file is returned.

Multi-buffer tensors load as a hurray.Tensor carrying every buffer in descriptor order (ADR-030). A sparse tensor round-trips its values and index arrays and comes back with its layout intact — there is one tensor class, so nothing is reconstructed into a different type. A composite comes back as a hurray.Composite, and its members do not also appear under their own names.

The GIL is released during file I/O so other Python threads are not blocked.

Errors

Examples

import hurray

tensors = hurray.load("model.hrry")
embeddings = tensors["embeddings"]   # hurray.Tensor
print(embeddings.shape, embeddings.dtype)

# Load only specific tensors
subset = hurray.load("model.hrry", names=["embeddings", "bias"])
def load_kv(path):

Read a Hurray file's key-value metadata section.

The other half of save(path, tensors, kv=...): what that writes, this reads back. Returns an empty dict for a file with no KV section.

A separate call rather than an argument to load because it answers a different question and returns a different thing — a flag that changed load's return type would make every caller unpack a tuple to ask about tensors. Reading it costs a footer seek, not a scan of the file.

Value types

The seven wire types map back to the Python objects that produced them: bool, int (from both int64 and uint64), float, str, bytes, and list of any of those.

Errors

  • hurray.FileError — the file is missing, truncated, or its KV section is malformed.

Examples

import hurray

hurray.save(
    "model.hrry",
    {"w": hurray.Tensor(bytes(16), hurray.float32, [4])},
    kv={"model": "demo", "layers": 12, "quantized": False},
)

meta = hurray.load_kv("model.hrry")
assert meta == {"model": "demo", "layers": 12, "quantized": False}
def save(path, tensors, *, kv=None):

Save tensors to a Hurray file.

Writes all entries in tensors (a dict mapping str names to hurray.Tensor objects) to the HRRYFILE at path. The optional kv argument stores file-level metadata as key-value pairs.

The GIL is released during file I/O so other Python threads are not blocked.

KV value types

kv values may be bool, int, float, str, bytes, or a homogeneous list of one of those scalar types (not nested lists).

Errors

Examples

import hurray

t = hurray.zeros((4, 4), dtype=hurray.float32)
hurray.save("model.hrry", {"weights": t}, kv={"version": "1.0"})

# Round-trip
loaded = hurray.load("model.hrry")
assert loaded["weights"].shape == (4, 4)
def set_print_options(*, sparse_display=None):

Set one or more print options for the current asyncio task / thread context.

Options not provided (left as None) are unchanged; this keeps the signature extensible as new options are added in the future.

Arguments

  • sparse_display — controls how Tensor.__repr__ renders a sparse-layout tensor:
    • "metadata" (default) — SciPy-style compact summary: hurray.Tensor(layout='csr', shape=(3, 3), nnz=4, dtype=float32)
    • "content" — PyTorch-style, appending per-format buffer arrays after the dtype field: hurray.Tensor(layout='csr', …, values=[…], col_indices=[…], row_ptr=[…])

Raises

ValueError if sparse_display is not "metadata" or "content".

Examples

import hurray
hurray.set_print_options(sparse_display="content")
hurray.set_print_options(sparse_display="metadata")  # restore default
def get_print_options():

Return the current print options as a dict.

Keys reflect the full set of configurable options; all values are strings.

Examples

import hurray
opts = hurray.get_print_options()
assert opts["sparse_display"] == "metadata"
class PrintOptionsCtx:

Context manager that temporarily overrides print options for the current task.

On __enter__, sets the requested option(s) and stores the reset token. On __exit__, resets each ContextVar to its prior value via the stored token. This is exception-safe and supports nesting correctly.

Arguments

Same keyword arguments as set_print_options.

Examples

import hurray
assert hurray.get_print_options()["sparse_display"] == "metadata"
with hurray.print_options(sparse_display="content"):
    assert hurray.get_print_options()["sparse_display"] == "content"
assert hurray.get_print_options()["sparse_display"] == "metadata"
def decode_quantization(data):

Decode a standalone quantization_descriptor section.

The inverse of each scheme's encode(). Returns whichever of the five classes the section's scheme tag names, so a caller reads the bytes without knowing in advance which scheme produced them — which is the point of a tagged section.

Trailing bytes are permitted and ignored: the section carries its own length, which is what lets it sit inside a descriptor with other sections after it.

Errors

Examples

import hurray

scheme = hurray.PerTensorAffine(0.5, 0)
wire = scheme.encode()

assert hurray.decode_quantization(wire) == scheme
class PerTensorAffine:

Per-tensor affine quantization: one scale and zero point for the whole tensor.

The only scheme whose parameters are inline in the descriptor — it needs no scale buffer, which is why a per-tensor-affine tensor is still single-buffer.

value = scale * (quantized - zero_point)

Errors

Examples (Python)

import hurray

q = hurray.PerTensorAffine(0.02, 128)
assert q.scale == 0.02
assert q.zero_point == 128
def encode(self, /):

Encode this scheme to its wire bytes.

The quantization_descriptor section as it appears inside a tensor descriptor, usable on its own — 16 to 24 bytes, so encode allocating is not worth avoiding.

Examples

import hurray

scheme = hurray.PerTensorAffine(0.5, 0)
assert hurray.decode_quantization(scheme.encode()) == scheme
zero_point

The zero point.

Examples

assert hurray.PerTensorAffine(0.5, 7).zero_point == 7
scale

The scale factor.

Examples

assert hurray.PerTensorAffine(0.5, 0).scale == 0.5
class PerChannelAffine:

Per-channel affine quantization: one scale per slice along axis.

Scales live in their own buffer, so a per-channel-quantized tensor has at least two buffers.

Examples (Python)

import hurray

q = hurray.PerChannelAffine.symmetric(axis=0, scale_buffer_index=1)
assert q.axis == 0
assert q.scale_buffer_index == 1
assert q.zero_point_buffer_index is None    # symmetric
def symmetric(axis, scale_buffer_index):

Symmetric per-channel quantization: scales only, no zero points.

Errors

Examples

q = hurray.PerChannelAffine.symmetric(axis=0, scale_buffer_index=1)
def asymmetric(axis, scale_buffer_index, zero_point_buffer_index):

Asymmetric per-channel quantization: scales and per-channel zero points.

Errors

Examples

q = hurray.PerChannelAffine.asymmetric(
    axis=0, scale_buffer_index=1, zero_point_buffer_index=2
)
assert q.zero_point_buffer_index == 2
def encode(self, /):

Encode this scheme to its wire bytes.

The quantization_descriptor section as it appears inside a tensor descriptor, usable on its own — 16 to 24 bytes, so encode allocating is not worth avoiding.

Examples

import hurray

scheme = hurray.PerChannelAffine.symmetric(axis=0, scale_buffer_index=1)
assert hurray.decode_quantization(scheme.encode()) == scheme
scale_buffer_index

Buffer index holding the per-channel scales.

Examples

assert hurray.PerChannelAffine.symmetric(0, 1).scale_buffer_index == 1
zero_point_buffer_index

Buffer index holding the per-channel zero points, or None when symmetric.

Examples

assert hurray.PerChannelAffine.symmetric(0, 1).zero_point_buffer_index is None
axis

The quantized axis.

Examples

assert hurray.PerChannelAffine.symmetric(1, 1).axis == 1
class PerBlockAffine:

Per-block affine quantization: one scale per contiguous block of block_size elements along axis.

Examples (Python)

import hurray

q = hurray.PerBlockAffine.symmetric(1, 64, 1, hurray.float32)
assert q.block_size == 64
def symmetric(axis, block_size, scale_buffer_index, scale_type):

Symmetric per-block quantization: scales only.

Errors

Examples

def asymmetric( axis, block_size, scale_buffer_index, zero_point_buffer_index, scale_type):

Asymmetric per-block quantization: scales and per-block zero points.

Errors

Examples

def encode(self, /):

Encode this scheme to its wire bytes.

The quantization_descriptor section as it appears inside a tensor descriptor, usable on its own — 16 to 24 bytes, so encode allocating is not worth avoiding.

Examples

import hurray

scheme = hurray.PerBlockAffine.symmetric(axis=0, block_size=64, scale_buffer_index=1, scale_type=hurray.float32)
assert hurray.decode_quantization(scheme.encode()) == scheme
axis

The quantized axis.

Examples

assert hurray.PerBlockAffine.symmetric(1, 32, 1, hurray.float32).axis == 1
block_size

Elements per block along axis.

Examples

assert hurray.PerBlockAffine.symmetric(1, 32, 1, hurray.float32).block_size == 32
scale_type

Element type of the scale values: float16, bfloat16, or float32.

Examples

q = hurray.PerBlockAffine.symmetric(1, 32, 1, hurray.float32)
assert q.scale_type == hurray.float32
scale_buffer_index

Buffer index holding the per-block scales.

Examples

assert hurray.PerBlockAffine.symmetric(1, 32, 1, hurray.float32).scale_buffer_index == 1
zero_point_buffer_index

Buffer index holding the per-block zero points, or None when symmetric.

Examples

assert hurray.PerBlockAffine.symmetric(1, 32, 1, hurray.float32).zero_point_buffer_index is None
class NF4:

NF4 (NormalFloat4) block quantization: 4-bit codes indexing a fixed information-theoretically optimal lookup table, with one scale per block.

Examples (Python)

import hurray

q = hurray.NF4(axis=1, block_size=64, scale_buffer_index=1)
assert q.block_size == 64
def encode(self, /):

Encode this scheme to its wire bytes.

The quantization_descriptor section as it appears inside a tensor descriptor, usable on its own — 16 to 24 bytes, so encode allocating is not worth avoiding.

Examples

import hurray

scheme = hurray.NF4(axis=0, block_size=64, scale_buffer_index=1)
assert hurray.decode_quantization(scheme.encode()) == scheme
block_size

Elements per block along axis.

Examples

assert hurray.NF4(1, 64, 1).block_size == 64
scale_buffer_index

Buffer index holding the per-block scales.

Examples

assert hurray.NF4(1, 64, 1).scale_buffer_index == 1
axis

The quantized axis.

Examples

assert hurray.NF4(1, 64, 1).axis == 1
class MXFP:

MXFP (OCP Microscaling) block quantization: an 8-bit shared exponent per block alongside the element micro-floats.

Examples (Python)

import hurray

q = hurray.MXFP(axis=1, block_size=32, scale_buffer_index=1)
assert q.block_size == 32
def encode(self, /):

Encode this scheme to its wire bytes.

The quantization_descriptor section as it appears inside a tensor descriptor, usable on its own — 16 to 24 bytes, so encode allocating is not worth avoiding.

Examples

import hurray

scheme = hurray.MXFP(axis=0, block_size=32, scale_buffer_index=1)
assert hurray.decode_quantization(scheme.encode()) == scheme
axis

The quantized axis.

Examples

assert hurray.MXFP(1, 32, 1).axis == 1
block_size

Elements per block along axis.

Examples

assert hurray.MXFP(1, 32, 1).block_size == 32
scale_buffer_index

Buffer index holding the per-block shared exponents.

Examples

assert hurray.MXFP(1, 32, 1).scale_buffer_index == 1
class Statistics:

Precomputed statistics about a tensor's values.

Every field is optional. The descriptor's computed_mask — which records which statistics are meaningful — is derived from the arguments you pass, so a value can never be present with its validity bit unset. Omitted fields encode as zero with their bit clear.

Statistics are grouped as the wire format groups them: value_min, value_max and value_abs_max share one validity bit, as do value_mean and value_stddev, and nm_n/nm_m. Supplying part of a group and not the rest is an error rather than a silently half-filled section.

Errors

Examples (Python)

import hurray

s = hurray.Statistics(nnz=1024, value_min=-1.0, value_max=1.0, value_abs_max=1.0)
assert s.nnz == 1024
assert s.value_max == 1.0
assert s.value_mean is None      # not supplied, so not valid
nm_n

N in N:M structured sparsity, or None if not computed.

Examples

assert hurray.Statistics(nm_n=2, nm_m=4).nm_n == 2
value_mean

Arithmetic mean, or None if not computed.

Examples

s = hurray.Statistics(value_mean=0.5, value_stddev=0.1)
assert s.value_mean == 0.5
has_nan

Whether any NaN is present, or None if not computed.

Examples

assert hurray.Statistics(has_nan=False, has_inf=False).has_nan is False
computed_mask

The raw validity bitmask, as it appears on the wire.

Examples

assert hurray.Statistics().computed_mask == 0
assert hurray.Statistics(nnz=1).computed_mask == 1
value_max

Maximum element value, or None if not computed.

Examples

s = hurray.Statistics(value_min=-2.0, value_max=2.0, value_abs_max=2.0)
assert s.value_max == 2.0
nm_m

M in N:M structured sparsity, or None if not computed.

Examples

assert hurray.Statistics(nm_n=2, nm_m=4).nm_m == 4
value_abs_max

Maximum absolute element value, or None if not computed.

Examples

s = hurray.Statistics(value_min=-2.0, value_max=1.0, value_abs_max=2.0)
assert s.value_abs_max == 2.0
nnz

Non-zero element count, or None if not computed.

Examples

assert hurray.Statistics(nnz=10).nnz == 10
sparsity_ratio

Fraction of zero elements, or None if not computed.

Examples

assert hurray.Statistics().sparsity_ratio is None
has_inf

Whether any infinity is present, or None if not computed.

Examples

assert hurray.Statistics(has_nan=False, has_inf=True).has_inf is True
value_min

Minimum element value, or None if not computed.

Examples

s = hurray.Statistics(value_min=-2.0, value_max=2.0, value_abs_max=2.0)
assert s.value_min == -2.0
value_stddev

Population standard deviation, or None if not computed.

Examples

s = hurray.Statistics(value_mean=0.5, value_stddev=0.1)
assert s.value_stddev == 0.1
class Shard:

Describes this tensor's position within a larger logical tensor.

parent_shape is the shape of the whole tensor; shard_offset is where this piece starts along each dimension. Both must have the same length.

Errors

Examples (Python)

import hurray

# The second half of a [1024, 512] tensor along dimension 0.
s = hurray.Shard(parent_shape=[1024, 512], shard_offset=[512, 0])
assert s.parent_shape == (1024, 512)
assert s.shard_offset == (512, 0)
shard_offset

Starting index of this shard within the parent, per dimension.

Examples

assert hurray.Shard([8, 8], [4, 0]).shard_offset == (4, 0)
parent_shape

Shape of the logical parent tensor.

Examples

assert hurray.Shard([8, 8], [4, 0]).parent_shape == (8, 8)
class ExtensionType:

Describes a private extension element type — one whose tag is in 0xF0–0xFE.

The format reserves that tag range for types it does not standardize, and requires every descriptor using one to carry this section. It is what lets a consumer that has never heard of your type still size its buffers: bit_width and packing_factor are enough to compute bytes without understanding a single value.

A tensor whose dtype is an extension tag MUST carry one, and a tensor whose dtype is anything else MUST NOT — hurray.Tensor enforces both directions.

Sign fields

A float carries its sign in sign_bits, never in is_signed, which describes integer types only. An unsigned float — the shape of the built-in exponent-only float8_e8m0 — is therefore expressible: is_float=True, sign_bits=0.

packing_factor is not an argument. The spec leaves exactly one legal value for a given bit_width, so it is derived rather than restated.

Errors

Examples (Python)

import hurray

# A private 24-bit signed integer type.
ext = hurray.ExtensionType(bit_width=24, is_signed=True)
assert ext.packing_factor == 1
assert ext.buffer_size_bytes(10) == 30

# A private 4-bit type: two elements per byte, derived.
packed = hurray.ExtensionType(bit_width=4)
assert packed.packing_factor == 2
assert packed.buffer_size_bytes(7) == 4
def buffer_size_bytes(self, /, element_count):

Bytes needed to hold element_count elements of this type.

hurray.buffer_size_bytes cannot answer this: an extension Dtype reports a bit_width of 0, because the real width lives here. Sub-byte widths round up to the next whole byte.

Examples

import hurray

assert hurray.ExtensionType(bit_width=24).buffer_size_bytes(10) == 30
assert hurray.ExtensionType(bit_width=4).buffer_size_bytes(7) == 4   # ceil(7 / 2)
assert hurray.ExtensionType(bit_width=4).buffer_size_bytes(0) == 0
sign_bits

Number of sign bits — 1 for a signed float, 0 for an unsigned one.

Examples

# An exponent-only float, the float8_e8m0 shape: no sign, no mantissa.
scale = hurray.ExtensionType(
    bit_width=8, is_float=True, exponent_bits=8, exponent_bias=127,
)
assert scale.sign_bits == 0
exponent_bits

Number of exponent bits, for float types.

Examples

half = hurray.ExtensionType(
    bit_width=16, is_float=True, sign_bits=1, exponent_bits=5, mantissa_bits=10,
)
assert half.exponent_bits == 5
exponent_bias

Exponent bias, for float types.

Examples

half = hurray.ExtensionType(
    bit_width=16, is_float=True, sign_bits=1, exponent_bits=5, mantissa_bits=10,
    exponent_bias=15,
)
assert half.exponent_bias == 15
is_signed

Whether the type is a signed integer. Always False for a float — see sign_bits.

Examples

assert hurray.ExtensionType(bit_width=8, is_signed=True).is_signed is True
packing_factor

Elements packed per byte — 1 for whole-byte widths, 8 / bit_width below that.

Examples

assert hurray.ExtensionType(bit_width=8).packing_factor == 1
assert hurray.ExtensionType(bit_width=1).packing_factor == 8
mantissa_bits

Number of mantissa bits, for float types.

Examples

half = hurray.ExtensionType(
    bit_width=16, is_float=True, sign_bits=1, exponent_bits=5, mantissa_bits=10,
)
assert half.mantissa_bits == 10
has_nan

Whether NaN is representable.

Examples

assert hurray.ExtensionType(bit_width=8).has_nan is False
has_inf

Whether infinity is representable.

Examples

assert hurray.ExtensionType(bit_width=8).has_inf is False
is_float

Whether the type is floating-point.

Examples

assert hurray.ExtensionType(bit_width=8).is_float is False
bit_width

Bit width of one element.

Examples

assert hurray.ExtensionType(bit_width=24).bit_width == 24