Sparse Tensors and SciPy Interop

Hurray exposes COO, CSR, and CSC sparse tensors as ordinary hurray.Tensor objects whose .layout is a CooLayout, CsrLayout, or CscLayout (ADR-031, ADR-032). There is no separate sparse class — sparse is a layout, not a different kind of object. For CSR and CSC, buffers are shared zero-copy with SciPy sparse matrices via hurray.from_scipy and Tensor.to_scipy().

Constructing a CSR tensor from SciPy

SciPy's csr_matrix stores three NumPy arrays: .data (values), .indices (column indices), and .indptr (row pointers). hurray.from_scipy wraps all three — the resulting Tensor holds a strong reference to the original SciPy matrix so the buffers remain valid.

Each component is shared or copied on its own merits: three arrays are three allocations with three addresses, and only an address that meets the format's 64-byte alignment floor can be shared. Pass copy=False to be told which one fell short instead of paying for a silent copy — see Buffer Protocol.

Index dtype requirement: Hurray's wire format requires uint64 index arrays. SciPy defaults to int32. Cast before calling from_scipy:

import numpy as np
import scipy.sparse as sp
import hurray

dense = np.array(
    [[1.0, 0.0, 2.0],
     [0.0, 3.0, 0.0],
     [4.0, 0.0, 5.0]],
    dtype=np.float32,
)
m = sp.csr_matrix(dense)

# Cast index arrays to uint64 (required by Hurray's spec).
m.indices = m.indices.astype(np.uint64)
m.indptr  = m.indptr.astype(np.uint64)

sparse = hurray.from_scipy(m)
print(sparse)
# hurray.Tensor(layout='csr', shape=(3, 3), nnz=5, dtype=float32)

Accessing component views

Each component buffer is accessible as a zero-copy hurray.Tensor view. The view borrows the parent tensor's buffer — the parent is kept alive for as long as any view is alive.

Accessing a format-specific attribute on the wrong format raises AttributeError:

try:
    sparse.indices          # COO's attribute, on a CSR tensor
    raise AssertionError("a csr tensor has col_indices, not indices")
except AttributeError as exc:
    print(exc)              # 'Tensor' object has no attribute 'indices';
                            # this is a csr tensor

To read values into a NumPy array (zero-copy for Tier 1 types):

vals_np = np.array(sparse.values)   # zero-copy via DLPack
col_idx_np = np.array(sparse.col_indices)
row_ptr_np = np.array(sparse.row_ptr)

SciPy zero-copy export

Tensor.to_scipy() returns the matching scipy.sparse matrix type. copy=False is passed to the SciPy constructor; SciPy may copy internally if it cannot accept uint64 index arrays (version-dependent).

m2 = sparse.to_scipy()
assert isinstance(m2, sp.csr_matrix)
assert (m2.toarray() == dense).all()

CSC tensors return csc_matrix. COO tensors raise hurray.UnsupportedError (see below).

COO format caveats

hurray.from_scipy does not support COO format zero-copy. SciPy stores COO row/col coordinates as two separate arrays, while Hurray's spec requires a single packed [nnz, rank] uint64 buffer. Passing a coo_matrix raises hurray.UnsupportedError with instructions.

Workaround — convert to CSR first (zero-copy from Hurray's perspective):

m_coo = sp.coo_matrix(np.eye(5, dtype=np.float32))
m_csr = m_coo.tocsr()   # SciPy makes one copy here
m_csr.indices = m_csr.indices.astype(np.uint64)
m_csr.indptr  = m_csr.indptr.astype(np.uint64)

sparse = hurray.from_scipy(m_csr)

Preferred — hurray.sparse_coo from packed arrays:

hurray.sparse_coo(values, indices, shape) builds a COO Tensor directly, sharing each array whose alignment allows it and copying the rest (copy= works here too). indices is a 2-D uint64 array of shape [nnz, rank] (Hurray's packed layout); values is 1-D of length nnz.

import numpy as np, hurray

# Straight from packed arrays:
values = np.array([5.0, 7.0], dtype=np.float32)
indices = np.array([[0, 0], [1, 1]], dtype=np.uint64)   # [nnz, rank]
t = hurray.sparse_coo(values, indices, [2, 2])
assert t.layout == hurray.CooLayout(nnz=2) and t.nnz == 2

# Repacking a SciPy coo_matrix (one copy to interleave row/col, then zero-copy):
indices = np.stack([m_coo.row, m_coo.col], axis=1).astype(np.uint64)
t = hurray.sparse_coo(m_coo.data, indices, m_coo.shape)

Only the np.stack interleave copies; sparse_coo itself borrows both arrays and holds strong references so the buffers stay alive for the tensor's lifetime.

Tensor.to_scipy() on a COO tensor raises hurray.UnsupportedError. Access .values and .indices directly and construct scipy.sparse.coo_matrix manually if needed.

Strict mode and Tier 2 / quantized types

to_scipy() raises hurray.UnsupportedError for tensors with Tier 2 or quantized values dtypes (int4, float8 variants, etc.) because SciPy has no equivalent dtype. The index arrays are always uint64 and are unaffected.

SciPy as an optional dependency

import hurray does not require SciPy. from_scipy and to_scipy import scipy.sparse lazily at call time and raise ImportError if it is not installed:

try:
    sparse = hurray.from_scipy(m)
except ImportError:
    print("scipy not installed")

Runnable example

# From the repo root:
cd hurray-python
maturin develop          # build the extension
python examples/sparse_scipy.py