pub struct CompositeAssembler { /* private fields */ }Expand description
Rebuilds composite tensors from a stream of flat tensors.
Feed every tensor the stream yields, in wire order, to push. Plain
tensors come straight back out; a composite head opens a group that stays open until its
declared member_count members have been fed, at which point the whole composite is
returned as one item. Nesting is tracked with an explicit stack rather than recursion
(ADR-041 § 10), so arbitrarily deep composites cost stack space in the heap, not on the
call stack — and a half-assembled tree survives between calls.
Members are validated as they arrive, with the same CompositeValidator the writer
uses: member count, partition exact-cover, and overlay base-first ordering.
§Examples
use hurray_core::{
layout::{CompositeLayout, CompositionRule},
BufferHandle, DeviceTag, ElementType, LayoutDescriptor, Shape, SyncMode,
TensorDescriptor, MIN_BUFFER_ALIGNMENT,
};
use hurray_io::stream::{
CompositeAssembler, StreamItem, StreamTensor, DEFAULT_MAX_COMPOSITE_DEPTH,
};
// A group of one, and the tensor that fills it.
let head = TensorDescriptor::new(
1, 0, ElementType::Float32, Shape::new(vec![1u64])?, 0,
LayoutDescriptor::Composite(CompositeLayout::new(CompositionRule::Group, 1)?),
vec![], None, None, None, None,
)?;
let handle =
BufferHandle::new(4, MIN_BUFFER_ALIGNMENT, DeviceTag::Cpu, SyncMode::ProducerSynced)?;
let member = TensorDescriptor::new(
1, 0, ElementType::Float32, Shape::new(vec![1u64])?, 0,
LayoutDescriptor::RowMajor, vec![handle], None, None, None, None,
)?;
let mut assembler = CompositeAssembler::new(DEFAULT_MAX_COMPOSITE_DEPTH);
// The head opens the group: nothing comes out yet.
let head_tensor = StreamTensor { descriptor: head, buffers: vec![] };
assert!(assembler.push(head_tensor)?.is_none());
assert_eq!(assembler.depth(), 1);
// The member closes it, and the whole composite comes back as one item.
let member_tensor = StreamTensor {
descriptor: member,
buffers: vec![bytes::Bytes::from_static(&[0u8; 4])],
};
let item = assembler.push(member_tensor)?.expect("the group is complete");
let StreamItem::Composite(composite) = item else { panic!("expected a composite") };
assert_eq!(composite.members.len(), 1);
assert_eq!(assembler.depth(), 0);Implementations§
Source§impl CompositeAssembler
impl CompositeAssembler
Sourcepub fn new(max_depth: usize) -> Self
pub fn new(max_depth: usize) -> Self
Creates an assembler that refuses nesting deeper than max_depth.
DEFAULT_MAX_COMPOSITE_DEPTH is the
limit the readers apply.
§Examples
See the type-level example.
Sourcepub fn push(&mut self, tensor: StreamTensor) -> Result<Option<StreamItem>>
pub fn push(&mut self, tensor: StreamTensor) -> Result<Option<StreamItem>>
Feeds the next tensor from the stream.
Returns Some when a top-level item is complete — a plain tensor outside any
composite, or the outermost composite once its last member arrives — and None
while a composite is still being filled.
§Errors
Error::CompositeNestingTooDeep— a head arrived atmax_depthError::Core— the head is not a valid composite head, or the group failed validation (member count, partition coverage, overlay ordering)
§Examples
See the type-level example.
Sourcepub fn finish(&self) -> Result<()>
pub fn finish(&self) -> Result<()>
Reports end of input.
§Errors
Error::TornComposite — the stream ended with a composite still open. The
innermost one is reported: that is the level whose members actually went missing.
§Examples
use hurray_io::stream::{CompositeAssembler, DEFAULT_MAX_COMPOSITE_DEPTH};
// Nothing open, so end of input is clean.
let assembler = CompositeAssembler::new(DEFAULT_MAX_COMPOSITE_DEPTH);
assert!(assembler.finish().is_ok());