Crate arrow_data 

Low-level array data abstractions for Apache Arrow Rust

For a higher-level, strongly-typed interface see arrow_array

Platform Support

Only little-endian platforms are officially supported and tested in CI. Big-endian platforms are not tested in CI and may not work correctly. Fixes for big-endian platforms are welcome and handled on a best-effort basis, but compatibility is not guaranteed.

Modules

bit_iterator

Types for iterating over packed bitmasks

bit_mask

Utils for working with packed bit masks

byte_view 🔒

data 🔒

Contains ArrayData, a generic representation of Arrow array data which encapsulates common attributes and operations for Arrow array.

decimal

Maximum and minimum values for Decimal256, Decimal128, Decimal64 and Decimal32.

equal 🔒

Module containing functionality to compute array equality. This module uses ArrayData and does not depend on dynamic casting of Array.

ffi

Contains declarations to bind to the C Data Interface.

transform

Low-level array data abstractions.

Structs

ArrayData

A generic representation of Arrow array data which encapsulates common attributes and operations for Arrow array.

ArrayDataBuilder

Builder for ArrayData type

ByteView

Helper to access views of GenericByteViewArray (StringViewArray and BinaryViewArray) where the length is greater than 12 bytes.

DataTypeLayout

Layout specification for a data type

Enums

BufferSpec

Layout specification for a single data type buffer

Constants

MAX_INLINE_VIEW_LEN

The maximum number of bytes that can be stored inline in a byte view.

Functions

checked_len_plus_offset

contains_nulls

count_nulls

get_fixed_size_binary_width

Get byte width of FixedSizeBinary size

layout

Return the expected DataTypeLayout Arrays of this data type are expected to have

new_buffers

Creates 2 [MutableBuffer]s with a given capacity (in slots).

utf8_span

Returns the first and last of offsets, and the bytes of values between them as a str. Returns None if the offsets are not valid indexes into values or those bytes are not valid UTF-8.

validate_binary_view

Validates the combination of views and buffers is a valid BinaryView

validate_string_view

Validates the combination of views and buffers is a valid StringView

validate_view_impl

Type Aliases

ArrayDataRef

A thread-safe, shared reference to the Arrow array data.