Python Bindings Requirements — Hurray Implementation Requirements

Overview

The hurray-python package is the Python face of the Hurray interchange format: the library a Python program uses to produce Hurray tensors, consume them, and hand them off to the surrounding array ecosystem without copying. It is built on PyO3 and is a codec and zero-copy bridge — not a numerical or compute library, and not an implementation of the Python Array API Standard (see ADR-029). Its two goals are:

1. Produce / consume Hurray tensors — construct tensors and serialize them to the Hurray format, and parse Hurray data into usable Python objects, including the types the ecosystem cannot represent natively (Tier 2, quantized, sparse, composite).
2. Zero-copy interop — share buffers with NumPy, PyTorch, JAX, and CuPy without copying, via DLPack, the NumPy array protocols, and the native Hurray buffer protocol.

These interop protocols are standalone — DLPack in particular is an independent specification, not part of the Array API (a consumer's from_dlpack works on any object exposing __dlpack__, with no Array API namespace involved). hurray-python therefore does not claim, and MUST NOT advertise, Array API conformance; it is Array-API-interoperable (a Tier 1 tensor handed to NumPy/PyTorch becomes a real array those ecosystems can compute on), not Array-API-implementing.

Rationale

Note (non-normative): This section explains why hurray-python exists and the incentives that shape its surface. It is motivation, not a requirement.

hurray-python is the Python face of the Hurray format — the library a Python program uses to produce Hurray tensors (serialize its data into the format), consume them (load Hurray data into usable Python objects), and hand them off to the surrounding array ecosystem without copying. It is a codec and an interchange bridge, in the same spirit as the Python packages for other data formats: not a numerical or compute library, and not a place where array math lives. Its reason to exist is simply that Python is where the machine-learning ecosystem lives, and a format with no ergonomic Python entry point would never be adopted there.

Three incentives drive its design:

• Reach the ecosystem on day one. The array ecosystem already shares tensors through a widely implemented zero-copy handoff. By speaking that same handoff, hurray-python interoperates with NumPy, PyTorch, JAX, and CuPy immediately, for the common case (ordinary dense tensors of the standard element types), with no per-library adapters to write. This is why Hurray tensors are made to feel like ordinary arrays in that ecosystem rather than opaque blobs.

• Preserve full fidelity between Hurray-aware components. The common ecosystem handoff can only describe the ordinary dense case. Everything that makes Hurray worth having — compressed and quantized data, sparse and other specialized layouts, richer element types, and the metadata that travels with a tensor — falls outside what that handoff can carry. So hurray-python also offers a native Hurray interchange path that preserves the tensor in full, for sharing between components that both understand Hurray.

• Bridge the gap while adoption grows. In an ideal end state, producer and consumer libraries would understand Hurray natively, the same way they understand the common handoff today. Until then, hurray-python is the adapter that speaks both sides: it ingests tensors from the existing ecosystem and emits Hurray, and vice versa. This bridging role is deliberately temporary in ambition — it recedes naturally as more of the ecosystem adopts Hurray directly.

Finally, hurray-python offers a handful of direct convenience methods for the most common partners (NumPy, PyTorch) even though the generic zero-copy handoff already exists. This is a deliberate, adoption-minded choice: a new format is accepted or rejected on how much friction it adds, so a one-call, discoverable path matters. These methods also cover the cases the generic handoff cannot express, so that no data is silently left behind.

Tensor Surface

hurray-python MUST expose a hurray.Tensor class with an inspection and interop surface. This surface describes the tensor and hands it off zero-copy; it does not include array computation (elementwise math, reductions, linear algebra, indexing, or operators) — those belong to the framework the buffer is handed to. hurray.Tensor MUST NOT implement __array_namespace__, and the hurray module is not an Array API namespace (see ADR-029).

Inspection and interop methods

Dtype surface

hurray.Dtype MUST expose, in addition to name, bit_width, tier, and the is_float / is_integer / is_signed / is_sub_byte predicates:

Dtype.from_tag and Dtype.from_name MUST return the singleton for the type, not a new object: the class documents hurray.float32 is hurray.dtype.float32, and a lookup that returned an equal-but-not-identical object would break that quietly.

Because every private extension type shares the name "extension", repr MUST include the tag, which is the only thing distinguishing two of them.

Buffer sizing

The module MUST expose hurray.buffer_size_bytes(dtype, count) -> int, applying the packing rules of memory-layout.md § Sub-byte packing. count * dtype.bit_width // 8 is not a substitute: it is wrong for every sub-byte type.

Dynamic dimensions

A dynamic dimension MUST be spelled None in a shape passed to hurray.Tensor, matching what Tensor.shape returns for one, so that a shape read from a tensor can be passed back to build an equal descriptor (ADR-032 § 4). Implementations MUST NOT also expose a sentinel constant for it.

Functions that allocate a buffer or validate one against the shape — zeros, ones, empty, full, sparse_coo, Composite — MUST refuse a dynamic dimension with hurray.InvalidDescriptorError, naming the call and the offending index.

A tensor with a dynamic dimension MUST NOT render its buffer as data in repr or str: the wire sentinel reaches NumPy as -1, which means "infer this extent", so an unresolved dimension would print as an empty tensor.

Tier 1 dtype interop correspondence

Note (non-normative): Tier 1 element types use the standard numeric vocabulary, so a Tier 1 hurray.Tensor maps to a NumPy dtype without translation when handed to the ecosystem. This correspondence is an interop detail, not an Array API claim.

Tier 2 and quantized types

For Tier 2 element types (sub-byte integers, float8 variants) and quantized types, hurray-python MUST expose a hurray-namespaced dtype object (e.g., hurray.dtype.int4, hurray.dtype.float8_e4m3). These have no standard NumPy dtype and cross between Hurray-aware components via the native protocol or save/load.

Tensors with Tier 2 / quantized dtypes MAY still implement __dlpack__ where DLPack supports the element type. For element types outside the DLPack type enum, __dlpack__ MUST raise the Python built-in BufferError.

DLPack Interoperability

hurray-python MUST support DLPack for all element types that DLPack defines: float16, bfloat16, float32, float64, int8, uint8, int16, uint16, int32, uint32, int64, uint64, complex64, complex128, bool.

• __dlpack__() MUST return a PyCapsule named "dltensor_versioned" conforming to the DLPack specification (v1.0 or later).
• __dlpack__() MUST raise the Python built-in BufferError for any element type not in the DLPack type enum (e.g., int4, float8 variants, quantized types). BufferError is the conventional signal used across the ecosystem (NumPy, PyTorch) for a tensor that cannot be represented in DLPack.
• The DLPack capsule MUST reference the original buffer without copying. The buffer's reference count MUST be incremented when the capsule is created and decremented when the capsule is consumed or deleted.

Stream parameter semantics

The stream parameter of __dlpack__(stream=None) maps to the tensor's SyncMode:

Private device tags and memory classes

0xF0–0xFE is the private range in both spaces: an agreement between one producer and one consumer, which the spec gives no name.

• hurray.Device's kind and memory_class MUST each accept a name or a wire byte. The names cover what the spec assigns; the byte is how a private value is reached, since there is no name to pass.
• Device.tag and Device.memory_class_tag MUST expose the wire bytes, and Device.is_private MUST report whether the device tag is in the private range.
• kind and memory_class MUST report "private" for every value in the range — that is what the spec calls them — so repr MUST include the tag. Without it two different vendor devices print identically, which is a reader unable to tell what it is holding.
• A reserved or permanently-invalid byte MUST be refused rather than accepted as a private one: a later version of the format may assign it, and a descriptor built on a guess would then mean something else.

The same rule governs private element types, whose repr carries the tag for the same reason (§ Dtype surface).

Device ID

DLPack requires a device_id integer (e.g., GPU index) that is not stored in the Hurray wire format. The device_id MUST be passed as runtime metadata when constructing a hurray.Tensor and defaults to 0 (the first device of that type). __dlpack_device__() MUST return this runtime device_id.

• __dlpack_device__() MUST return the correct (DLDeviceType, device_id) pair according to the Device Tag Mapping (Hurray ↔ DLPack) table below.

Device and Memory Class Mapping (Hurray → DLPack)

The correct DLPack DLDeviceType for a Hurray buffer is determined by the combination of device_tag and memory_class. DLPack encodes what Hurray separates into two orthogonal fields as a single flat enum (e.g., kDLCUDAHost, kDLCUDAManaged). The binding layer is responsible for the translation; neither raw Hurray values NOR raw DLPack integers are stored in the other system's fields.

See ADR-020 for the rationale behind the two-field design.

Full mapping table

Combinations not listed above (e.g., WebGPU + HOST_PINNED, which is invalid per the per-device validity table in ADR-020) MUST NOT occur in a conforming descriptor; if encountered, the binding layer SHOULD raise hurray.UnsupportedError.

Notes on the mapping:

1. Hurray's device_tag values are intentionally distinct from DLPack's DLDeviceType integers (ADR-016). Translation is the binding layer's responsibility — Hurray tag values MUST NOT be passed directly to DLPack consumers, and DLPack integers MUST NOT be stored in a Hurray buffer handle.
2. When implementing __dlpack_device__, the binding layer MUST derive the DLDeviceType from the (device_tag, memory_class) pair using the table above. It MUST NOT return the raw Hurray device_tag value.
3. CPU HOST_PINNED and UNIFIED both map to kDLCPU because DLPack does not distinguish host-pinned from ordinary host memory from the CPU's perspective. The memory_class field carries this information for consumers that need it.
4. Metal STANDARD, HOST_PINNED, and UNIFIED all map to kDLMetal (8) because DLPack does not distinguish Metal storage modes. Consumers that need to distinguish them MUST read the Hurray memory_class field directly. Note: HOST_PINNED (StorageManaged) is deprecated on Apple Silicon; see ADR-020.
5. ROCm UNIFIED has no DLPack equivalent (kDLROCMManaged does not exist in DLPack v1.0). The binding MUST raise hurray.UnsupportedError.
6. PEER memory has no DLPack equivalent for any device type. The binding MUST raise hurray.UnsupportedError for any PEER buffer exposed via DLPack.
7. For implementation-private device tags (0xF0–0xFE), the binding layer MUST NOT fabricate a DLPack mapping. It MUST raise hurray.UnsupportedError unless the consumer has explicitly agreed on a private mapping out of band.
8. For combinations in notes 5, 6, and 7 where hurray.UnsupportedError is raised, Hurray-aware consumers SHOULD use the native interchange protocol (__hurray__) instead. See Native Interchange Protocol.

Native Interchange Protocol

hurray-python MUST expose a native interchange protocol for hurray-to-hurray zero-copy transfers covering (device_tag, memory_class) combinations that DLPack v1.0 cannot represent (see ADR-023).

• hurray.Tensor.__hurray__(stream=None) -> PyCapsule — MUST return a PyCapsule named "hurray_tensor" wrapping a HurrayBufferList pointer from hurray-ffi. Available for all dtypes (Tier 1, Tier 2, quantized).
• hurray.from_hurray(obj, /) -> hurray.Tensor — MUST accept any object whose __hurray__ returns a valid capsule and reconstruct a hurray.Tensor that owns the transferred buffers.

Multi-buffer tensors

A tensor whose descriptor references more than one buffer — per-channel / NF4 / MXFP quantization, sparse layouts, block-paged, composite — MUST carry every buffer in a single capsule (ADR-030).

• The capsule pointer MUST be a HurrayBufferList owning one HurrayBuffer per buffer. A single-buffer tensor is the N = 1 case, not a separate path.
• Element i of the list MUST be the buffer at index i of the descriptor's buffer table. Buffer indices appearing in quantization descriptors (scale_buffer_index, zero_point_buffer_index), layout descriptors, and composite members index the list directly.
• A producer MUST NOT emit a capsule whose list length differs from the descriptor's buffer count, and a consumer MUST reject such a capsule rather than construct a tensor whose buffer indices do not resolve.
• hurray.Tensor MUST expose the descriptor's optional sections for reading: quantization (returning the scheme class, or None), statistics, shard, and buffer_count. The quantization getter MUST return an object of the same class the constructor accepts, so an inspected scheme can be reused to build another tensor without conversion.
• A sparse-layout hurray.Tensor MUST carry its component buffers through this protocol, with its values and index buffers in descriptor order. A separate __hurray_sparse_buffer__ protocol MUST NOT be introduced: sparse is the multi-buffer case, not a distinct one.
• hurray.save() MUST write every buffer of a tensor, and hurray.load() MUST accept multi-buffer tensors, rejecting any whose buffer count disagrees with its descriptor.

Capsule lifetime

The PyCapsule lifetime rules MUST match DLPack discipline:

• Capsule name on creation: "hurray_tensor".
• Capsule name after consumption: "used_hurray_tensor". The consumer MUST rename the capsule before taking ownership, exactly as DLPack consumers rename "dltensor" to "used_dltensor".
• Capsule destructor: if the capsule is destroyed while still named "hurray_tensor", the destructor MUST call hurray_buffer_list_destroy on the wrapped pointer, which destroys every handle the list owns. If the capsule has been consumed (renamed), the consumer MUST call hurray_buffer_list_destroy exactly once. Handles obtained from hurray_buffer_list_get are borrowed and MUST NOT be destroyed individually.
• The source Tensor's Python reference count MUST be incremented when the capsule is created and decremented in both the destructor and consume paths. See Buffer Lifetime and Ownership.

stream parameter

__hurray__(stream=None) uses the same stream semantics as __dlpack__: see Stream parameter semantics.

The capsule context

The capsule context MUST be a HurrayTensorContext from hurray-ffi (ADR-034), carrying the encoded TensorDescriptor and the producing build's HURRAY_C_ABI_VERSION. It MUST NOT be a structure private to hurray-python: the protocol serves consumers in other languages, and a private structure leaves them the buffers without the descriptor that says what the buffers are.

The strong reference that keeps the source tensor alive MUST travel as the context's opaque owner, released through its owner_release callback, so that no Python type reaches the C ABI.

ABI versioning

hurray.from_hurray MUST read the version with hurray_tensor_context_abi_version before any other accessor and before dereferencing the buffer list; a mismatch MUST raise hurray.UnsupportedError. See C FFI § Tensor Context for why that ordering is normative.

Discovery

Consumers MUST discover support by probing hasattr(obj, '__hurray__'). There is no separate capability flag on the hurray namespace.

Descriptor Encoding

This section uses RFC 2119 key words: MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL.

The descriptor is the format's central artifact. hurray-python MUST be able to produce and consume it, or a Python program cannot carry a Hurray tensor in a container of its own nor read one that arrived out of band.

hurray.Descriptor

A Descriptor is what a tensor declares, without its data. It MUST expose dtype, shape, ndim, size, layout, buffer_handles, buffer_count, device, quantization, shard, statistics, byte_offset, version, and encoded_len, and MUST agree with the tensor it came from on every one of them.

It MUST NOT be constructible from Python: a constructor would duplicate hurray.Tensor's entire parameter list to build the half of it that carries no data. Descriptors come from Tensor.descriptor, from Composite.descriptor (the head — the one descriptor that is only a descriptor), or from Descriptor.decode.

decode MUST return a Descriptor and MUST NOT return a Tensor: a decoded descriptor has no buffers, and a Tensor holding none would be the same class of false statement ADR-037 removed from alignment.

Encoding

• Descriptor.encode() MUST produce the binary descriptor exactly as it appears on the wire, and Descriptor.decode(bytes) MUST recover an equal descriptor from it.
• decode MUST accept trailing bytes and ignore them. The descriptor carries its own length; in a stream what follows it is the data it describes.
• encoded_len MUST equal len(encode()).

Standalone quantization sections

Each quantization class MUST expose encode(), and the module MUST expose hurray.decode_quantization(bytes) returning whichever class the section's scheme tag names — a caller reads the bytes without knowing in advance which scheme wrote them.

The classes MUST compare by value, like every other descriptor value object in this binding. A class carrying a floating-point field MUST NOT define __hash__: NaN would break the hash/equality contract, and being unhashable is the correct outcome rather than an omission.

Buffer Lifetime and Ownership

Zero-copy interop requires that the source object's buffer remains valid for the entire lifetime of any object that holds a pointer to it. The binding layer MUST enforce the following rules:

• hurray.from_numpy(array) → hurray.Tensor: The Tensor MUST hold a strong Python reference to the source ndarray for its own lifetime. The ndarray MUST NOT be garbage-collected while the Tensor is alive.

• hurray.Tensor.__array__() → ndarray: The returned ndarray MUST reference the source Tensor as its base object (via NumPy's base attribute or an equivalent mechanism), so that the Tensor is kept alive for as long as the ndarray holds the buffer.

• hurray.Tensor.__dlpack__() → capsule: The DLPack capsule destructor MUST decrement the Tensor's Python reference count when the capsule is consumed or deleted. The reference count MUST be incremented when the capsule is created. This ensures the Tensor (and therefore its buffer) is not freed while a DLPack consumer holds the capsule.

The same rules apply to the component Tensor views a sparse-layout tensor hands out.

Buffer Handles

This section uses RFC 2119 key words: MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL.

The binding MUST expose one hurray.BufferHandle per entry of a tensor's buffer table (ADR-037).

The class

hurray.BufferHandle MUST be immutable and MUST expose, as read-only properties:

It MUST compare and hash by value, and MUST NOT be constructible from Python: every field is settled by the buffers a tensor already holds, so there is nothing a caller could supply.

A BufferHandle MUST NOT hold a reference to its tensor or to any buffer. In a zero-copy format a metadata accessor that extends buffer lifetime is a defect: collecting handles across a stream MUST pin nothing.

Tensor.buffer_handles

hurray.Tensor.buffer_handles MUST be a tuple with one handle per buffer, in descriptor order, so len(t.buffer_handles) == t.buffer_count. It MUST be available for every layout and every device, including tensors whose bytes cannot be handed out.

Tensor.buffer(i) is unchanged and remains the byte-view accessor. The two are separate deliberately: on a device tensor the metadata question has an answer where the byte question does not.

BufferHandle.device MUST be the same object as Tensor.device, such that t.buffer_handles[i].device is t.device holds. buffer-protocol.md § Device Colocation requires every buffer of one descriptor to share a device and memory class, so there is exactly one to report.

Alignment is measured

A descriptor emitted by the binding MUST declare, for each buffer, an alignment its base address actually satisfies:

• An empty buffer MUST declare 1.
• An owned buffer MUST be allocated to at least MIN_BUFFER_ALIGNMENT and declare what it allocated.
• A borrowed buffer MUST have its base address measured, and MUST declare the largest power of two that address satisfies, capped at PAGE_ALIGNMENT. A stronger true declaration is legal and useful to IPC and RDMA consumers.

The binding MUST NOT declare an alignment it has not established. A consumer issues aligned loads on the strength of this field.

copy on ingest

Because buffer-protocol.md § Alignment sets a 64-byte floor that NumPy does not promise, every ingest entry point — from_numpy, from_torch, from_scipy, sparse_coo, from_dlpack, asarray — MUST accept a keyword-only copy: bool | None = None:

For a multi-buffer source the decision MUST be made per buffer: SciPy's .data, .indices and .indptr are three allocations with three addresses.

The aligned allocator

hurray-python MUST expose hurray.aligned_allocator(), a context manager that installs a 64-byte-aligned NumPy data-memory handler (NEP 49) for the duration of a block, so that arrays allocated inside it need no copy on ingest.

• It MUST restore the previously installed handler on exit, including when the block raises, and MUST support nesting.
• It MUST NOT be offered as a module-level install. The handler is thread-local, so a process-wide "install once" would silently fail to cover arrays allocated on other threads.
• It MUST raise hurray.UnsupportedError on NumPy older than 1.22, which predates NEP 49.

Implementations MUST document that a thread started inside the block does not inherit the policy.

sync_mode

sync_mode MUST be read-only, and no constructor MAY accept it. "event" asserts that a device event exists for a consumer to wait on; no Python API supplies one, so a settable field could only author a contract nothing could honour. Everything the binding constructs is therefore producer_synced.

A tensor holding any buffer whose sync_mode is not producer_synced MUST refuse the paths that hand out its bytes — buffer(i), the component views (.values, .indices, .indptr), __array__, to_torch — with hurray.UnsupportedError naming the mode. __dlpack__ MUST raise the built-in BufferError instead, as Stream parameter semantics already requires.

__hurray__ and StreamWriter.write MUST continue to relay such tensors unchanged: relaying a declaration is not reading a byte.

Alignment and the round-trip obligation

alignment is exempt from the round-trip obligation that governs layout, quantization, statistics and shard (ADR-032 § 4). Alignment describes an address, and a rebuild that copies bytes has a different address; a tensor that arrived declaring 4096 MAY honestly declare 64 once rebuilt. Implementations MUST NOT add a settable alignment= to force equality.

Constants

The module MUST expose hurray.MIN_BUFFER_ALIGNMENT and hurray.PAGE_ALIGNMENT, mirroring hurray-core.

NumPy Interoperability

For CPU tensors with Tier 1 element types, hurray-python MUST support:

• hurray.Tensor.__array__() — MUST return a NumPy ndarray backed by the same buffer (zero-copy for C-contiguous / row-major tensors; a copy is acceptable for other layouts if NumPy cannot represent them natively). See Buffer Lifetime and Ownership.
• hurray.from_numpy(array) — MUST create a hurray.Tensor that shares the NumPy array's buffer without copying, for C-contiguous arrays. See Buffer Lifetime and Ownership.

For tensors with Tier 2 / quantized element types, hurray.Tensor.__array__() MUST raise hurray.UnsupportedError. NumPy has no dtype for int4, float8 variants, or quantized/scaled types; returning an ndarray is structurally impossible. Callers that need the raw bytes SHOULD use the native protocol or DLPack directly.

PyTorch Interoperability

For CPU and CUDA tensors, hurray-python MUST support zero-copy conversion via DLPack:

• hurray.from_torch(tensor) — MUST call tensor.__dlpack__() and wrap the result in a hurray.Tensor without copying.
• hurray.Tensor.to_torch() — MUST call self.__dlpack__() and construct a torch.Tensor via torch.utils.dlpack.from_dlpack without copying.

Layouts and Sparse Tensor Support

hurray-python MUST expose exactly one tensor class, hurray.Tensor, for every layout (ADR-031). There MUST NOT be a separate class per layout family: a sparse tensor is a hurray.Tensor whose layout happens to be COO, CSR, or CSC, matching the format, where sparse is a layout_tag inside the ordinary tensor descriptor.

• .layout — MUST return a hurray.Layout object, not a string (ADR-032). See § Layout Descriptor Classes below.
• .values — a hurray.Tensor view over the values buffer.
• .indices (COO) or .col_indices / .row_ptr (CSR) or .row_indices / .col_ptr (CSC) — hurray.Tensor views over the index buffers.
• .nnz — the stored non-zero count, for layouts that track one.
• .to_scipy() — MUST convert a CSR or CSC tensor to the corresponding scipy.sparse matrix type without copying where scipy's memory layout is compatible.
• hurray.from_scipy(matrix) — MUST wrap a scipy.sparse matrix as a hurray.Tensor without copying.

Accessors that do not apply to a tensor's layout MUST raise AttributeError, so that hasattr reports whether a tensor actually supports them (ADR-031 § 2, extending design decision D10). They MUST NOT raise hurray.UnsupportedError, which would make hasattr return True for every accessor on every tensor.

Protocols that require a densely addressable element buffer — __dlpack__, __array__, __array_interface__, to_torch — MUST reject non-dense layouts, naming the layout in the error message.

hurray.Tensor MUST additionally expose buffer(index), returning a 1-D uint8 view of exactly the declared byte size of the buffer at that tensor-descriptor buffer index. CSF has 2 * rank + 1 buffers and block-paged has three, none of which have named accessors; without a generic accessor their parameters would be reachable while their buffers were not. uint8 is the only honest element type for a generic view — the buffers within one tensor differ (values take the tensor's dtype, index buffers are uint64, MXFP scales are e8m0) — and it cannot misreport a dtype. The layout object tells the caller what each index means. An index with no buffer MUST raise IndexError.

Layout Descriptor Classes

hurray-python MUST expose a class hierarchy of layout descriptors (ADR-032). A layout carries parameters — nnz, strides, page_size — that a string cannot.

The hierarchy

hurray.Layout                        # base: holds the core layout descriptor
├── RowMajorLayout   ColMajorLayout
├── StridedLayout    TiledLayout     MortonLayout    HilbertLayout
├── CooLayout        CsrLayout       CscLayout       CsfLayout
├── BlockPagedLayout
├── CompositeLayout
├── PrivateExtensionLayout
└── UnknownLayout

The base MUST implement tag, name, buffer_count, is_dense and is_virtual once; subclasses are typed façades reading their own fields off the same descriptor. The base MUST NOT be constructible from Python.

buffer_count MUST be 0 for a composite head — a known zero — and None for a private or unknown layout, whose count is genuinely not knowable.

When the core layout enum gains a variant a build has not bound, .layout MUST return a bare Layout carrying tag and name. It MUST NOT be reported as UnknownLayout, which asserts that the tag was unrecognised rather than merely unbound.

Value semantics

Layout objects MUST be immutable, MUST compare by value, and MUST hash consistently with that equality. .layout is read-only, and MAY return a fresh object per access: t.layout is t.layout is not guaranteed, t.layout == t.layout is.

Layout classes MUST NOT define equality against strings. repr MUST take the form CsrLayout(nnz=4).

Frozen name and enumeration strings

Layout.name is one of: "row_major", "col_major", "strided", "tiled", "morton", "hilbert", "coo", "csr", "csc", "csf", "block_paged", "composite", or "extension" for a private or unrecognised tag. These MUST match the layout names in docs/spec/memory-layout.md. PrivateExtensionLayout and UnknownLayout therefore share a name; isinstance is what distinguishes them, and they MUST remain separate classes because "a private layout I can identify" and "a tag from a newer spec version I could not parse" are different facts a permissive relay depends on.

A single internal helper MUST produce the name string, so that .name and the layout named in error messages (__dlpack__, __array__, to_scipy) cannot drift.

Small closed enumerations are exposed as lowercase strings, not as Python enum classes:

CompositeLayout MUST expose the composition rule and the combine operation as two properties, with combine being None for non-overlay rules. They MUST NOT be flattened into one string, and the raw combine byte MUST NOT be exposed for rules where it means "not applicable".

Strides are in logical elements

StridedLayout.strides and the tiled layouts' outer_strides and inner_strides are in logical elements, signed, and may be negative or zero — not in bytes as NumPy's are.

Authoring

hurray.Tensor MUST accept a layout keyword holding a hurray.Layout instance, mirroring quantization=. Omitting it means row-major. A non-Layout value MUST raise TypeError.

A layout string MUST NOT be accepted. A string cannot carry nnz or strides, so layout="csr" is a request that cannot be honoured; accepting it would create a second, lossy authoring path.

The layout object is a declaration; the buffers are evidence. They MUST agree, and the constructor MUST check three tiers:

A layout MUST NOT be inferred from the buffers, and the buffers MUST NOT be reinterpreted to fit the layout. CooLayout(nnz=4) supplied with a two-element values buffer MUST raise: the descriptor MUST NOT be silently corrected to nnz=2, and MUST NOT be accepted as given, since it would encode and decode cleanly and hand the consumer an out-of-bounds read. Over-sized buffers MUST be permitted; alignment and padding slack are legitimate.

Consequently nnz MUST be a required argument on the sparse layout constructors. Inference belongs to the array-shaped constructors — hurray.sparse_coo, hurray.from_scipy — which are handed the arrays and can derive it.

For every constructible layout, rebuilding a tensor from another tensor's layout, quantization, statistics, shard and buffers MUST produce an equal descriptor.

Tag classification

The module MUST expose hurray.layout_tag_kind(tag) -> str, returning "named", "reserved", "private", or "invalid". The four MUST partition the whole byte space.

A single function rather than the four predicates hurray-core exposes: the caller's purpose is to branch on the answer, and the four categories are mutually exclusive. The distinction is actionable — a reserved tag suggests a producer newer than this reader, so relaying the tensor is reasonable while interpreting its buffer is not; a private tag refers to an out-of-band agreement; an invalid tag cannot appear in a conformant descriptor at all.

Checking a layout against a shape

Layout.validate_against_shape(shape) MUST apply the layout's own rank and extent constraints, raising hurray.InvalidDescriptorError when the pair cannot go together. It MUST accept a dynamic dimension without complaint: an unresolved extent cannot violate an extent constraint.

The result MUST agree with what hurray.Tensor enforces at construction — the two are the same rule reached by two doors.

Block-paged validation

BlockPagedLayout MUST expose two checks:

• validate_index_buffers(seq_ptr, block_table) — the four storage invariants of block-paged.md § Storage. num_pages and num_seqs come from the layout, so only the buffers are passed. Aliasing MUST be accepted: two sequences naming one page is how a shared prefix is represented.
• validate_quantization_compatibility(quantization) — that a scheme and the paging can go together. It MUST take the quantization descriptor object, not a scheme tag.

Both raise hurray.InvalidDescriptorError. They matter because a descriptor describes its index buffers rather than containing them: their contents are unchecked at construction, and they are what stands between a consumer and an out-of-bounds read.

Element addressing (element_offset) is deliberately not exposed. Resolving one logical coordinate at a time is indexing, which this package does not do.

Composite, private, and unknown

• CompositeLayout MUST be readable in full, so a composite head decoded from a stream reports its own layout truthfully. Constructing a hurray.Tensor with a composite layout MUST raise hurray.UnsupportedError: a composite head owns no buffers, which the Python Tensor cannot represent.
• UnknownLayout MUST be constructible, so a permissive relay can reconstruct a descriptor it decoded and write it back out. Its constructor MUST reject any tag for which a named variant exists — calling a known tag "unknown" would smuggle an unvalidated descriptor past every rank and buffer check.
• PrivateExtensionLayout exposes tag, extension_layout_id, and extension_data.

Known gap: because the buffer count of a private or unknown layout is unknowable, the buffer-count and buffer-size tiers cannot run for them. Nothing in such a descriptor states how many buffers it needs or how large they should be.

Composites

hurray-python MUST expose composite tensors through a hurray.Composite class (ADR-036). A composite MUST NOT be a hurray.Tensor: a composite contains tensors where a tensor has data, its head owns zero buffers, and len(members) has no meaning on a tensor.

composite = hurray.Composite(
    "partition", shape=[8, 8], dtype=hurray.float32, members=[tile0, tile1]
)

Construction

• composition_rule — "partition", "overlay", or "group".
• shape and dtype — the logical view the head presents. Both are required and MUST NOT be derived from the members: a partition's shape could be computed from its members' shards, and deriving it would silently reshape the head to match a caller's miscomputed offset.
• members — a sequence of hurray.Tensor or hurray.Composite, since the format nests. Anything else MUST raise TypeError naming the offending index.
• combine_op — required for "overlay", and MUST be rejected for the other rules, matching hurray.CompositeLayout, which is the same field.
• member_count is taken from members rather than stated: it counts what was passed, so it cannot disagree with it.

Validation MUST delegate to hurray-core's CompositeValidator — per-member boxes, partition coverage, overlay ordering, member count. The binding MUST NOT keep a second copy of those rules. Failures MUST surface as hurray.InvalidDescriptorError.

Surface

A Composite MUST compare by value — head and members alike — so that the round-trip obligation can be stated as equality. Its repr MUST be depth-aware, since composites nest.

Composite MUST NOT expose values, buffer, buffer_count, __array__, or __dlpack__. There is no data to expose; the members hold it.

I/O

• StreamWriter.write MUST accept a Composite and emit it as head-then-members.
• StreamReader MUST yield a Composite as one item. It MUST NOT surface a head and its members as separate items — that would lose the composition without raising.
• save MUST accept a Composite as a named entry; load MUST return one.
• Because the file container gives every tensor an index entry, a composite's members MUST be named "{head}.{index}", recursively. Those names are an artifact of the container, not of the composite, so they are generated rather than requested.
• load MUST return a composite under its head's name and MUST NOT also return its members as top-level entries. A member requested explicitly by name MUST still be returned.

Overlay member roles

An overlay's members carry a role — the first is the base and spans the whole index space, the rest are corrections. The format fixes this by position, so hurray.Composite MUST derive the roles rather than accept them: a parameter would ask the caller to restate what the rule already determines, and give them a way to state it wrongly.

This is not inference in ADR-032 § 4's sense. Nothing is read out of the buffers; the role follows from the composition rule and the member's position, both of which the caller stated.

A member that already carries a role — one decoded from the wire — MUST keep it, so a round trip cannot relabel it.

The roles MUST be readable as Composite.member_roles, a tuple of "base" / "correction" / None, and MUST NOT be exposed on the member tensor: a tensor has no role, a tensor within an overlay does. Reading from the composite also makes an authored overlay and a decoded one answer identically.

Because the role lives on the composite rather than on the caller's tensor, the descriptor a member presents to its parent differs from the one on its tensor. Implementations MUST use the former when validating, when writing, and when comparing composites for equality — a composite that does not equal its own round trip is the symptom of getting this wrong.

Not on the native protocol

Composite MUST NOT implement __hurray__. That capsule carries a buffer list and one descriptor; a composite is a tree, and flattening one would require wire structure the format does not define. hasattr(obj, "__hurray__") therefore keeps meaning what it means. Use the streaming or file path, which the format defines for exactly this.

Streaming

hurray-python MUST expose the streaming interchange format through two classes (ADR-035). The format's defining property is that neither side buffers the whole sequence, and the Python surface MUST preserve it: a reader MUST yield a tensor as soon as its descriptor and buffers have arrived, and a writer MUST emit each tensor as it is given one.

with hurray.StreamWriter(destination) as writer:
    writer.write(tensor)

for tensor in hurray.StreamReader(source):
    ...

Blocking

Both classes are blocking. Each owns a tokio runtime for its lifetime and MUST release the GIL around every call into it, so other Python threads run while a stream waits on its transport. An asyncio surface is deliberately absent; see ADR-035 § 1.

hurray.StreamReader

• MUST implement __iter__ returning itself and __next__ yielding a hurray.Tensor, raising StopIteration at a clean end of stream.
• MUST implement the context-manager protocol, and expose close(), so the transport can be released without waiting for collection. A closed reader MUST behave as exhausted rather than raise.
• MUST read with next_item semantics, not next_tensor: a composite head owns no buffers, so reading it as an ordinary tensor would yield an empty tensor and then surface the head's members as top-level tensors — a stream that decoded "successfully" having lost the composition.
• MUST raise hurray.UnsupportedError, naming the composite, when the stream contains one. It MUST NOT skip it.

hurray.StreamWriter

• MUST accept a hurray.Tensor in write, transmitting every buffer its descriptor references, in descriptor order.
• MUST implement the context-manager protocol, finishing the stream on exit. finish() MUST also be callable explicitly and MUST be idempotent, so the two paths compose.
• MUST raise hurray.StreamError when written to after finishing.
• With no destination, MUST build the stream in memory and return it from getvalue(). getvalue() on a writer that has a destination MUST raise hurray.StreamError.

Reader limits

hurray.StreamReader MUST accept keyword arguments bounding what a single frame may claim: max_descriptor_bytes, max_buffer_bytes, and max_composite_depth, each defaulting to hurray-io's own default. Exceeding one MUST raise hurray.StreamError.

These are not optional hygiene. A descriptor's length field is read before its contents, so a stream from an untrusted peer can ask a reader to allocate an arbitrary amount.

They MUST be keyword arguments rather than an options object: Python has keyword arguments, and an options class here would be a Rust shape in Python clothing.

Cross-machine transport

hurray.StreamReader and hurray.StreamWriter MUST accept cross_machine: bool = False, enforcing that every buffer's sync_mode is producer_synced. A device event or stream handle is meaningless on the far side of a network, so a descriptor that carries one across is unsatisfiable by construction; the check refuses it at the boundary rather than leaving a consumer to wait on an event that does not exist.

Transports

A file descriptor MUST be duplicated, so that finishing or closing the stream does not close the caller's. An object with no descriptor — io.BytesIO — is not accepted directly; the TypeError raised MUST say so, since the fix (getvalue()) is not guessable from a bare type error.

Exceptions

Note (non-normative): a stream has no end marker — frames are self-delimiting and the stream ends at EOF, which is the property that also forbids end-of-file indexes. A stream truncated mid-frame therefore raises hurray.StreamError, while one truncated exactly on a frame boundary is indistinguishable from a shorter stream and yields no error. That is a property of the format, not of the binding.

File metadata

hurray.load_kv(path) -> dict MUST return a file's key-value section, and MUST be the exact inverse of what save(path, tensors, kv=...) writes: a dict saved and reloaded MUST compare equal. A file with no KV section MUST read as an empty dict.

It MUST be a separate call rather than an argument to load, which answers a different question and returns a different thing.

bool MUST be tested before int in both directions, since Python's bool is a subclass of int. Python's int writes as int64, so a value written from Python never carries the uint64 tag; a uint64 written by another producer MUST read back as int.

A KvValue variant this build does not recognise MUST raise rather than being dropped: a metadata dict with a silent hole in it is worse than a refusal.

Writer output

StreamWriter.getvalue() MUST be repeatable, like io.BytesIO.getvalue(). A destructive implementation makes the second call return empty, which turns a later read into a clean end of stream — data loss reported as success.

Error Handling

All errors from the Rust core MUST be surfaced as Python exceptions:

hurray.FileError is raised by file-level operations (hurray.load(), hurray.save()): file not found, permission denied, corrupt HRRYFILE container, unexpected EOF.

hurray.StreamError is raised by the streaming reader/writer: frame corruption mid-stream, unexpected stream termination, framing errors on a pipe or socket.

Both FileError and StreamError are subclasses of OSError; callers that do not need to distinguish between the two MAY catch OSError directly.

FileError and StreamError are introduced in Layer 8b (file I/O bridge) and are not present in Layer 8a (core types + DLPack).

Panics from the Rust core MUST NOT propagate as Python crashes. The PyO3 binding layer MUST catch panics and convert them to hurray.InternalError (subclass of RuntimeError).

hurray.from_hurray (Layer 8c) MUST raise:

• hurray.BufferError if the capsule is null, already consumed (named "used_hurray_tensor"), or otherwise invalid.
• hurray.UnsupportedError if the HURRAY_C_ABI_VERSION embedded in the capsule does not match the consumer's linked version.

Conformance and Validation

hurray-python is validated against the shared golden test-vector corpus (conformance/vectors/), the same corpus the Rust implementation is checked against, plus the binding's own unit and integration tests.

• The Python binding MUST decode every descriptor and buffer in the golden corpus to the same logical values as the Rust reference, and MUST re-encode round-trippable vectors to byte-identical output. This is exercised in CI (see the python-conformance job).
• The array-api-tests suite is not used: it targets a whole conforming Array API namespace and presupposes the compute core, which hurray-python deliberately does not implement (see ADR-029). Cross-checking against the golden corpus maps directly onto the parts of the Hurray specification the binding actually covers.
• New behaviour MUST ship with tests that exercise the public interop surface (DLPack, NumPy/PyTorch bridges, the native protocol, save/load).

Benchmark Suite

hurray-python SHOULD maintain a benchmark suite that measures performance across the Hurray interchange and interop paths — the operations that define the binding's purpose: constructing tensors, moving buffers zero-copy, and serializing/parsing the Hurray format.

Benchmark categories

The suite SHOULD cover the following categories:

Tooling

• Benchmarks MUST be runnable via a standard Python benchmarking tool (e.g., pytest-benchmark or airspeed-velocity (asv)).
• Benchmarks SHOULD report: mean, standard deviation, and minimum latency; throughput in GiB/s for memory-bound operations.
• Regression tracking (detecting performance regressions across commits) SHOULD be automated in CI. Benchmarks that regress by more than 10% relative to the baseline SHOULD trigger a warning in the pull request.
• The benchmark suite MUST be runnable independently from the validation tests.

Compatibility Matrix

The hurray-python package MUST maintain a compatibility matrix document at hurray-python/COMPAT-MATRIX.md. This document lives alongside the code — not in the spec — because it changes with every release.

The compatibility matrix MUST record, for each hurray-python release series:

• The DLPack specification version(s) supported as producer and as consumer.
• The supported CPython version range.
• The Hurray format/descriptor version(s) the binding produces and consumes.

The matrix MUST be updated whenever a new hurray-python release changes any of these.

Packaging

• The package MUST be installable via pip install hurray.
• Wheels MUST be provided for CPython ≥ 3.10 on Linux (x86_64, aarch64), macOS (x86_64, arm64), and Windows (x86_64).
• The package MUST NOT require a Rust toolchain at install time (pre-built wheels are mandatory for distribution).
• Optional dependencies: numpy, torch, scipy (none required at import time; interop functions raise ImportError if the target library is not installed).