Struct EncodedRows

Source

pub struct EncodedRows {
    data: Bytes,
    offsets: Vec<usize>,
}

Expand description

A contiguous set of Avro encoded rows.

EncodedRows stores:

a single backing byte buffer (bytes::Bytes)
a Vec<usize> of row boundary offsets (length = rows + 1)

This lets callers get per-row payloads as zero-copy Bytes slices.

For compatibility with APIs that require owned Vec<u8>, use: let vecs: Vec<Vec<u8>> = rows.iter().map(|b| b.to_vec()).collect();

Fields§

§data: Bytes§offsets: Vec<usize>

Implementations§

Source §

impl EncodedRows

Source

pub fn new(data: Bytes, offsets: Vec<usize>) -> Self

Create a new EncodedRows from a backing buffer and row boundary offsets.

offsets must have length rows + 1, and be monotonically non-decreasing. The last offset should equal data.len().

Source

pub fn len(&self) -> usize

Returns the number of encoded rows stored in this container.

Source

pub fn is_empty(&self) -> bool

Returns true if this container holds no encoded rows.

Source

pub fn bytes(&self) -> &Bytes

Returns a reference to the single contiguous backing buffer.

This buffer contains the payloads of all rows concatenated together.

§Note

To access individual row payloads, prefer using Self::row or Self::iter rather than slicing this buffer manually.

Source

pub fn offsets(&self) -> &[usize]

Returns the row boundary offsets.

The returned slice always has the length self.len() + 1. The nth row payload corresponds to bytes[offsets[n] ... offsets[n+1]].

Source

pub fn row(&self, n: usize) -> Result<Bytes, AvroError>

Return the nth row as a zero-copy Bytes slice.

§Errors

Returns an error if n is out of bounds or if the internal offsets are invalid (e.g., offsets are not within the backing buffer).

§Examples

use std::sync::Arc;
use arrow_array::{ArrayRef, Int32Array, RecordBatch};
use arrow_schema::{DataType, Field, Schema};
use arrow_avro::writer::WriterBuilder;
use arrow_avro::writer::format::AvroSoeFormat;

let schema = Schema::new(vec![Field::new("x", DataType::Int32, false)]);
let batch = RecordBatch::try_new(
    Arc::new(schema.clone()),
    vec![Arc::new(Int32Array::from(vec![1, 2])) as ArrayRef],
)?;

let mut encoder = WriterBuilder::new(schema).build_encoder::<AvroSoeFormat>()?;
encoder.encode(&batch)?;
let rows = encoder.flush();

assert_eq!(rows.iter().count(), 2);

Source

pub fn iter(&self) -> impl ExactSizeIterator<Item = Bytes> + '_

Iterate over rows as zero-copy Bytes slices.

This iterator is infallible and is intended for the common case where EncodedRows is produced by Encoder::flush, which guarantees valid offsets.

§Examples

use std::sync::Arc;
use arrow_array::{ArrayRef, Int32Array, RecordBatch};
use arrow_schema::{DataType, Field, Schema};
use arrow_avro::writer::WriterBuilder;
use arrow_avro::writer::format::AvroSoeFormat;

let schema = Schema::new(vec![Field::new("x", DataType::Int32, false)]);
let batch = RecordBatch::try_new(
    Arc::new(schema.clone()),
    vec![Arc::new(Int32Array::from(vec![10, 20])) as ArrayRef],
)?;

let mut encoder = WriterBuilder::new(schema).build_encoder::<AvroSoeFormat>()?;
encoder.encode(&batch)?;
let rows = encoder.flush();

assert_eq!(rows.iter().count(), 2);