Skip to main content

Crate arrow_buffer

Crate arrow_buffer 

Source
Expand description

Low-level buffer abstractions for Apache Arrow Rust

§Byte Storage abstractions

  • MutableBuffer: Raw memory buffer that can be mutated and grown
  • Buffer: Immutable buffer that is shared across threads

§Typed Abstractions

There are also several wrappers over Buffer with methods for easier manipulation:

§Platform Support

Only little-endian platforms are officially supported and tested in CI. Big-endian platforms are not tested in CI and may not work correctly. Fixes for big-endian platforms are welcome and handled on a best-effort basis, but compatibility is not guaranteed.

Re-exports§

pub use error::OverflowError;
pub use error::SliceOutOfBoundsError;
pub use bigint::i256;
pub use buffer::*;
pub use builder::*;

Modules§

alloc
Defines the low-level Allocation API for shared memory regions
arith 🔒
bigint 🔒
bit_chunk_iterator
Types for iterating over bitmasks in 64-bit chunks
bit_iterator
Types for iterating over packed bitmasks
bit_mask
Utils for working with packed bit masks
bit_util
Utils for working with bits
buffer
Types of shared memory region
builder
Buffer builders
bytes 🔒
This module contains an implementation of a contiguous immutable memory region that knows how to de-allocate itself, Bytes. Note that this is a low-level functionality of this crate.
divisor 🔒
error 🔒
Errors returned by arrow-buffer.
interval 🔒
native 🔒
pool 🔒
This module contains traits for memory pool traits and an implementation for tracking memory usage.
private
util 🔒

Macros§

divisor_enum
n_bit_divisor
A divisor with an N-bit multiplier, which computes the quotient as MULUH(m, n') >> shift for n' from $body. Up and Down differ only in n', and the macro keeps them from duplicating the rest. |n| ... looks like a closure but is macro syntax for the expression, because a const fn can’t call a closure
native_float
native_integer
signed_divisor
unsigned_divisor

Structs§

AddIndicatorDivisorI32
A i32 divisor whose magnitude isn’t a power of two, see DivisorI32
AddIndicatorDivisorI64
A i64 divisor whose magnitude isn’t a power of two, see DivisorI64
AddIndicatorDivisorU32
A u32 divisor that isn’t a power of two, see DivisorU32
AddIndicatorDivisorU64
A u64 divisor that isn’t a power of two, see DivisorU64
DownDivisorU32
A u32 divisor whose rounded-up multiplier isn’t exact in N bits, see DivisorU32
IntervalDayTime
Value of an IntervalDayTime array
IntervalMonthDayNano
Value of an IntervalMonthDayNano array
PowerOfTwoDivisorI32
A i32 divisor whose magnitude is a power of two, see DivisorI32
PowerOfTwoDivisorI64
A i64 divisor whose magnitude is a power of two, see DivisorI64
PowerOfTwoDivisorU32
A u32 divisor that is a power of two, see DivisorU32
PowerOfTwoDivisorU64
A u64 divisor that is a power of two, see DivisorU64
TrackedReservation
This is a wrapper for the reservation so we can standardize on changing and avoid race conditions in memory accounting
Tracker
TrackingMemoryPool
A simple MemoryPool that reports the total memory usage
UpDivisorU32
A u32 divisor whose rounded-up multiplier is exact in N bits, see DivisorU32
UpDivisorU64
A u64 divisor whose rounded-up multiplier is exact in N bits, see DivisorU64

Enums§

DivisorI32
A i32 divisor, precomputed once to divide many values by it
DivisorI64
A i64 divisor, precomputed once to divide many values by it
DivisorU32
A u32 divisor, precomputed once to divide many values by it
DivisorU64
A u64 divisor, precomputed once to divide many values by it

Constants§

CHEAP_SATURATION
Whether a saturating increment on u32 lanes is cheap: one uqadd on aarch64, and pminud from SSE4.1 and a subtract on x86. Baseline x86-64 has no unsigned 32-bit min and emulates it with compares, so there the saturating increment of DivisorU32::Down costs more than the add and halve of DivisorU32::AddIndicator

Traits§

ArrowNativeType
Trait expressing a Rust type that has the same in-memory representation as Arrow.
MemoryPool
A pool of memory that can be reserved and released.
MemoryReservation
A memory reservation within a MemoryPool that is freed on drop
ToByteSlice
Allows conversion from supported Arrow types to a byte slice.

Functions§

lock_reservation
Lock a memory reservation, recovering from a poisoned lock.