Crate arrow_buffer 

Low-level buffer abstractions for Apache Arrow Rust

Byte Storage abstractions

• MutableBuffer: Raw memory buffer that can be mutated and grown
• Buffer: Immutable buffer that is shared across threads

Typed Abstractions

There are also several wrappers over Buffer with methods for easier manipulation:

• BooleanBuffer: Bitmasks (buffer of packed bits)
• NullBuffer: Arrow null (validity) bitmaps (BooleanBuffer with extra utilities)
• ScalarBuffer<T>: Typed buffer for primitive types (e.g., i32, f64)
• OffsetBuffer<O>: Offsets used in variable-length types (e.g., strings, lists)
• RunEndBuffer<E>: Run-ends used in run-encoded data

Platform Support

Only little-endian platforms are officially supported and tested in CI. Big-endian platforms are not tested in CI and may not work correctly. Fixes for big-endian platforms are welcome and handled on a best-effort basis, but compatibility is not guaranteed.

Re-exports

pub use buffer::*;

pub use builder::*;

Modules

alloc

Defines the low-level Allocation API for shared memory regions

arith 🔒

bigint 🔒

bit_chunk_iterator

Types for iterating over bitmasks in 64-bit chunks

bit_iterator

Types for iterating over packed bitmasks

bit_mask

Utils for working with packed bit masks

bit_util

Utils for working with bits

buffer

Types of shared memory region

builder

Buffer builders

bytes 🔒

This module contains an implementation of a contiguous immutable memory region that knows how to de-allocate itself, Bytes. Note that this is a low-level functionality of this crate.

error 🔒

Errors returned by arrow-buffer.

interval 🔒

native 🔒

pool 🔒

This module contains traits for memory pool traits and an implementation for tracking memory usage.

util 🔒

Structs

IntervalDayTime

Value of an IntervalDayTime array

IntervalMonthDayNano

Value of an IntervalMonthDayNano array

OverflowError

An arithmetic overflow.

TrackingMemoryPool

A simple MemoryPool that reports the total memory usage

i256

A signed 256-bit integer

Traits

ArrowNativeType

Trait expressing a Rust type that has the same in-memory representation as Arrow.

MemoryPool

A pool of memory that can be reserved and released.

MemoryReservation

A memory reservation within a MemoryPool that is freed on drop

ToByteSlice

Allows conversion from supported Arrow types to a byte slice.