A zero-copy, streamable, language-agnostic tensor interchange format for AI/ML inference pipelines and scientific arrays. Zola 2026-07-26T00:00:00+00:00 https://www.pascalgillet.net/hurray/blog/atom.xml 2026-07-26T00:00:00+00:00 2026-07-26T00:00:00+00:00 Unknown https://www.pascalgillet.net/hurray/blog/introducing-hurray/ <p>Modern inference pipelines move tensors constantly: from disk into host memory, across process boundaries, between machines, and onto accelerators. The formats we reach for were each designed for one leg of that journey. <strong>DLPack</strong> nails in-process hand-off but has no notion of quantization or on-disk storage. <strong>SafeTensors</strong> and <strong>GGUF</strong> store weights well but say nothing about runtime interchange. <strong>Apache Arrow</strong> is superb for columnar analytics but isn't tensor-shaped.</p> <p><strong>Hurray</strong> is an attempt to cover the whole path with a single tensor descriptor.</p> <h2 id="two-formats-one-descriptor">Two formats, one descriptor</h2> <p>Hurray defines two binary formats that share the same descriptor encoding:</p> <ul> <li>a <strong>streaming format</strong> for runtime interchange — self-delimiting, no seek required, safe to start processing before the payload finishes arriving; and</li> <li>a <strong>file format</strong> for on-disk model storage — named tensors, a footer index for random access, and 4 KiB-aligned buffers for mmap-to-GPU zero-copy loading.</li> </ul> <p>Write a descriptor parser once and it works for both.</p> <h2 id="built-for-inference">Built for inference</h2> <p>Hurray treats layout diversity, quantization, and device memory as first-class concerns:</p> <ul> <li><strong>Twelve memory layouts</strong>, including tiled/blocked for GEMM, Morton and Hilbert curves for spatial locality, sparse COO/CSR/CSC/CSF, block-paged for PagedAttention KV caches, and composite tensors for partitioning and overlays.</li> <li><strong>Five quantization schemes</strong> — per-tensor, per-channel, and per-block affine, NF4 (QLoRA), and MXFP (OCP Microscaling / Blackwell) — with normative dequantization formulas.</li> <li><strong>Zero-copy</strong> buffer sharing through a stable, language-agnostic C ABI.</li> </ul> <h2 id="where-we-are">Where we are</h2> <p>The specification is at <code>0.1.0-draft</code> and the Rust reference implementation is underway. The format is designed to be <strong>evolvable</strong>: backward-compatible and forward-additive across the whole <code>1.x</code> line, with public tags never rebound once allocated.</p> <p>This site is where the specification, implementation requirements, cookbook, tutorials, and design decisions (ADRs) live — versioned alongside every release. Have a look at the <a href="/docs/stable/">docs</a>, and come say hello on <a href="https://github.com/pgillet/hurray">GitHub</a>.</p>