Module array

Statically typed implementations of Arrow Arrays

See [arrow_array] for examples and usage instructions

Modules

array

The concrete array definitions

builder

Defines push-based APIs for constructing arrays

cast

Defines helper functions for downcasting dyn Array to concrete types

ffi

Contains declarations to bind to the C Data Interface.

ffi_stream

Contains declarations to bind to the C Stream Interface.

iterator

Idiomatic iterators for Array

run_iterator

Idiomatic iterator for RunArray

temporal_conversions

Conversion methods for dates and times.

timezone

Timezone for timestamp arrays

types

Zero-sized types used to parameterize generic array implementations

Macros

create_array

Creates an array from a literal slice of values, suitable for rapid testing and development.

downcast_dictionary_array

Downcast an Array to a DictionaryArray based on its DataType, accepts a number of subsequent patterns to match the data type

downcast_integer

Given one or more expressions evaluating to an integer DataType invokes the provided macro m with the corresponding integer ArrowPrimitiveType, followed by any additional arguments

downcast_integer_array

Given one or more expressions evaluating to an integer PrimitiveArray invokes the provided macro with the corresponding array, along with match statements for any non integer array types

downcast_primitive

Given one or more expressions evaluating to primitive DataType invokes the provided macro m with the corresponding ArrowPrimitiveType, followed by any additional arguments

downcast_primitive_array

Downcast an Array to a PrimitiveArray based on its DataType accepts a number of subsequent patterns to match the data type

downcast_run_array

Downcast an Array to a RunArray based on its DataType, accepts a number of subsequent patterns to match the data type

downcast_run_end_index

Given one or more expressions evaluating to an integer DataType invokes the provided macro m with the corresponding integer RunEndIndexType, followed by any additional arguments

downcast_temporal

Given one or more expressions evaluating to primitive DataType invokes the provided macro m with the corresponding ArrowPrimitiveType, followed by any additional arguments

downcast_temporal_array

Downcast an Array to a temporal PrimitiveArray based on its DataType accepts a number of subsequent patterns to match the data type

record_batch

Creates a record batch from literal slice of values, suitable for rapid testing and development.

Structs

ArrayData

A generic representation of Arrow array data which encapsulates common attributes and operations for Arrow array.

ArrayDataBuilder

Builder for ArrayData type

ArrayIter

An iterator that returns Some(T) or None, that can be used on any ArrayAccessor

BooleanArray

An array of boolean values

BooleanBufferBuilder

Builder for BooleanBuffer

BooleanBuilder

Builder for BooleanArray

BufferBuilder

Builder for creating a Buffer object.

ByteView

Helper to access views of GenericByteViewArray (StringViewArray and BinaryViewArray) where the length is greater than 12 bytes.

DataTypeLayout

Layout specification for a data type

DictionaryArray

An array of dictionary encoded values

FixedSizeBinaryArray

An array of fixed size binary arrays

FixedSizeBinaryBuilder

Builder for FixedSizeBinaryArray

FixedSizeBinaryDictionaryBuilder

Builder for DictionaryArray of FixedSizeBinaryArray

FixedSizeListArray

An array of [fixed length lists], similar to JSON arrays (e.g. ["A", "B"]).

FixedSizeListBuilder

Builder for FixedSizeListArray

GenericByteArray

An array of variable length byte arrays

GenericByteBuilder

Builder for GenericByteArray

GenericByteDictionaryBuilder

Builder for DictionaryArray of GenericByteArray

GenericByteRunBuilder

Builder for RunArray of GenericByteArray

GenericByteViewArray

Variable-size Binary View Layout: An array of variable length bytes views.

GenericByteViewBuilder

A builder for GenericByteViewArray

GenericListArray

An array of variable length lists, similar to JSON arrays (e.g. ["A", "B", "C"]). This struct specifically represents the list layout. Refer to GenericListViewArray for the list-view layout.

GenericListBuilder

Builder for GenericListArray

GenericListViewArray

An array of variable length lists, specifically in the list-view layout.

GenericListViewBuilder

Builder for GenericListViewArray

MapArray

An array of key-value maps

MapBuilder

Builder for MapArray

MapFieldNames

The Field names for a MapArray

MutableArrayData

Efficiently create an ArrayData from one or more existing ArrayDatas by copying chunks.

NativeAdapter

An optional primitive value

NullArray

An array of null values

NullBufferBuilder

Builder for creating NullBuffer

NullBuilder

Builder for NullArray

OffsetBufferBuilder

Builder of OffsetBuffer

PrimitiveArray

An array of primitive values, of type ArrowPrimitiveType

PrimitiveBuilder

Builder for PrimitiveArray

PrimitiveDictionaryBuilder

Builder for DictionaryArray of PrimitiveArray

PrimitiveRunBuilder

Builder for RunArray of PrimitiveArray

RecordBatch

A two-dimensional batch of column-oriented data with a defined schema.

RecordBatchIterator

Generic implementation of RecordBatchReader that wraps an iterator.

RecordBatchOptions

Options that control the behaviour used when creating a RecordBatch.

RunArray

An array of run-end encoded values

Scalar

A wrapper around a single value Array that implements Datum and indicates compute kernels should treat this array as a scalar value (a single value).

StructArray

An array of structs

StructBuilder

Builder for StructArray

TypedDictionaryArray

A DictionaryArray typed on its child values array

TypedRunArray

A RunArray typed typed on its child values array

UnionArray

An array of values of varying types

UnionBuilder

Builder for UnionArray

Enums

BufferSpec

Layout specification for a single data type buffer

Capacities

Define capacities to pre-allocate for child data or data buffers.

Traits

AnyDictionaryArray

A DictionaryArray with the key type erased

Array

An array in the arrow columnar format

ArrayAccessor

A generic trait for accessing the values of an Array

ArrayBuilder

Trait for dealing with different array builders at runtime

ArrowNativeTypeOp

Trait for ArrowNativeType that adds checked and unchecked arithmetic operations, and totally ordered comparison operations

ArrowNumericType

A subtype of primitive type that represents numeric values.

ArrowPrimitiveType

Trait for primitive values.

AsArray

An extension trait for dyn Array that provides ergonomic downcasting

BinaryArrayType

A trait for Arrow String Arrays, currently three types are supported:

Datum

A possibly Scalar Array

OffsetSizeTrait

A type that can be used within a variable-size array to encode offset information

RecordBatchReader

Trait for types that can read RecordBatch’s.

RecordBatchWriter

Trait for types that can write RecordBatch’s.

StringArrayType

A trait for Arrow String Arrays, currently three types are supported:

Functions

as_boolean_array

Force downcast of an Array, such as an ArrayRef to BooleanArray, panicking on failure.

as_dictionary_array

Force downcast of an Array, such as an ArrayRef to DictionaryArray<T>, panic’ing on failure.

as_fixed_size_list_array

Force downcast of an Array, such as an ArrayRef to FixedSizeListArray, panicking on failure.

as_generic_binary_array

Force downcast of an Array, such as an ArrayRef to GenericBinaryArray<S>, panicking on failure.

as_generic_list_array

Force downcast of an Array, such as an ArrayRef to GenericListArray<T>, panicking on failure.

as_large_list_array

Force downcast of an Array, such as an ArrayRef to LargeListArray, panicking on failure.

as_largestring_array

Force downcast of an Array, such as an ArrayRef to LargeStringArray, panicking on failure.

as_list_array

Force downcast of an Array, such as an ArrayRef to ListArray, panicking on failure.

as_map_array

Force downcast of an Array, such as an ArrayRef to MapArray, panicking on failure.

as_null_array

Force downcast of an Array, such as an ArrayRef to NullArray, panicking on failure.

as_primitive_array

Force downcast of an Array, such as an ArrayRef, to PrimitiveArray<T>, panic’ing on failure.

as_run_array

Force downcast of an Array, such as an ArrayRef to RunArray<T>, panic’ing on failure.

as_string_array

Force downcast of an Array, such as an ArrayRef to StringArray, panicking on failure.

as_struct_array

Force downcast of an Array, such as an ArrayRef to StructArray, panicking on failure.

as_union_array

Force downcast of an Array, such as an ArrayRef to UnionArray, panicking on failure.

downcast_array

Downcasts a dyn Array to a concrete type

export_array_into_raw^⚠Deprecated

Exports an array to raw pointers of the C Data Interface provided by the consumer.

layout

Return the expected DataTypeLayout Arrays of this data type are expected to have

make_array

Constructs an array using the input data. Returns a reference-counted Array instance.

make_builder

Returns a builder with capacity for capacity elements of datatype DataType.

make_comparator

Returns a comparison function that compares two values at two different positions between the two arrays.

make_view

Create a view based on the given data, block id and offset.

new_empty_array

Creates a new empty array

new_null_array

Creates a new array of data_type of length length filled entirely of NULL values

Type Aliases

ArrayDataRef

A thread-safe, shared reference to the Arrow array data.

ArrayRef

A reference-counted reference to a generic Array

BinaryArray

A GenericBinaryArray of [u8] using i32 offsets

BinaryBuilder

Builder for BinaryArray

BinaryDictionaryBuilder

Builder for DictionaryArray of BinaryArray

BinaryRunBuilder

Builder for RunArray of BinaryArray

BinaryViewArray

A GenericByteViewArray of [u8]

BinaryViewBuilder

Array builder for BinaryViewArray

BooleanIter

an iterator that returns Some(T) or None, that can be used on any BooleanArray

Date32Array

A PrimitiveArray of days since UNIX epoch stored as i32

Date32BufferBuilder

Buffer builder for 32-bit date type.

Date32Builder

A 32-bit date array builder.

Date64Array

A PrimitiveArray of milliseconds since UNIX epoch stored as i64

Date64BufferBuilder

Buffer builder for 64-bit date type.

Date64Builder

A 64-bit date array builder.

Decimal128Array

A PrimitiveArray of 128-bit fixed point decimals

Decimal128BufferBuilder

Buffer builder for 128-bit decimal type.

Decimal128Builder

A decimal 128 array builder

Decimal256Array

A PrimitiveArray of 256-bit fixed point decimals

Decimal256BufferBuilder

Buffer builder for 256-bit decimal type.

Decimal256Builder

A decimal 256 array builder

DurationMicrosecondArray

A PrimitiveArray of elapsed durations in microseconds

DurationMicrosecondBufferBuilder

Buffer builder for elaspsed time of microseconds unit.

DurationMicrosecondBuilder

An elapsed time in microseconds array builder.

DurationMillisecondArray

A PrimitiveArray of elapsed durations in milliseconds

DurationMillisecondBufferBuilder

Buffer builder for elaspsed time of milliseconds unit.

DurationMillisecondBuilder

An elapsed time in milliseconds array builder.

DurationNanosecondArray

A PrimitiveArray of elapsed durations in nanoseconds

DurationNanosecondBufferBuilder

Buffer builder for elaspsed time of nanoseconds unit.

DurationNanosecondBuilder

An elapsed time in nanoseconds array builder.

DurationSecondArray

A PrimitiveArray of elapsed durations in seconds

DurationSecondBufferBuilder

Buffer builder for elaspsed time of second unit.

DurationSecondBuilder

An elapsed time in seconds array builder.

DynComparator

Compare the values at two arbitrary indices in two arrays.

FixedSizeBinaryIter

an iterator that returns Some(T) or None, that can be used on any FixedSizeBinaryArray

FixedSizeListIter

an iterator that returns Some(T) or None, that can be used on any FixedSizeListArray

Float16Array

A PrimitiveArray of f16

Float16BufferBuilder

Buffer builder for 16-bit floating point type.

Float16Builder

A 16-bit floating point array builder.

Float32Array

A PrimitiveArray of f32

Float32BufferBuilder

Buffer builder for 32-bit floating point type.

Float32Builder

A 32-bit floating point array builder.

Float64Array

A PrimitiveArray of f64

Float64BufferBuilder

Buffer builder for 64-bit floating point type.

Float64Builder

A 64-bit floating point array builder.

GenericBinaryArray

A GenericBinaryArray for storing [u8]

GenericBinaryBuilder

Array builder for GenericBinaryArray

GenericBinaryIter

an iterator that returns Some(T) or None, that can be used on any BinaryArray

GenericListArrayIter

an iterator that returns Some(T) or None, that can be used on any ListArray

GenericListViewArrayIter

an iterator that returns Some(T) or None, that can be used on any ListArray

GenericStringArray

A GenericByteArray for storing str

GenericStringBuilder

Array builder for GenericStringArray

GenericStringIter

an iterator that returns Some(T) or None, that can be used on any Utf8Array

Int8Array

A PrimitiveArray of i8

Int8BufferBuilder

Buffer builder for signed 8-bit integer type.

Int8Builder

A signed 8-bit integer array builder.

Int8DictionaryArray

A DictionaryArray indexed by i8

Int16Array

A PrimitiveArray of i16

Int16BufferBuilder

Buffer builder for signed 16-bit integer type.

Int16Builder

A signed 16-bit integer array builder.

Int16DictionaryArray

A DictionaryArray indexed by i16

Int16RunArray

A RunArray with i16 run ends

Int32Array

A PrimitiveArray of i32

Int32BufferBuilder

Buffer builder for signed 32-bit integer type.

Int32Builder

A signed 32-bit integer array builder.

Int32DictionaryArray

A DictionaryArray indexed by i32

Int32RunArray

A RunArray with i32 run ends

Int64Array

A PrimitiveArray of i64

Int64BufferBuilder

Buffer builder for signed 64-bit integer type.

Int64Builder

A signed 64-bit integer array builder.

Int64DictionaryArray

A DictionaryArray indexed by i64

Int64RunArray

A RunArray with i64 run ends

IntervalDayTimeArray

A PrimitiveArray of “calendar” intervals in days and milliseconds

IntervalDayTimeBufferBuilder

Buffer builder for “calendar” interval in days and milliseconds.

IntervalDayTimeBuilder

A “calendar” interval in days and milliseconds array builder.

IntervalMonthDayNanoArray

A PrimitiveArray of “calendar” intervals in months, days, and nanoseconds.

IntervalMonthDayNanoBufferBuilder

Buffer builder “calendar” interval in months, days, and nanoseconds.

IntervalMonthDayNanoBuilder

A “calendar” interval in months, days, and nanoseconds array builder.

IntervalYearMonthArray

A PrimitiveArray of “calendar” intervals in whole months

IntervalYearMonthBufferBuilder

Buffer builder for “calendar” interval in months.

IntervalYearMonthBuilder

A “calendar” interval in months array builder.

LargeBinaryArray

A GenericBinaryArray of [u8] using i64 offsets

LargeBinaryBuilder

Builder for LargeBinaryArray

LargeBinaryDictionaryBuilder

Builder for DictionaryArray of LargeBinaryArray

LargeBinaryRunBuilder

Builder for RunArray of LargeBinaryArray

LargeListArray

A GenericListArray of variable size lists, storing offsets as i64.

LargeListBuilder

Builder for LargeListArray

LargeListViewArray

A GenericListViewArray of variable size lists, storing offsets as i64.

LargeListViewBuilder

Builder for LargeListViewArray

LargeStringArray

A GenericStringArray of str using i64 offsets

LargeStringBuilder

Builder for LargeStringArray

LargeStringDictionaryBuilder

Builder for DictionaryArray of LargeStringArray

LargeStringRunBuilder

Builder for RunArray of LargeStringArray

ListArray

A GenericListArray of variable size lists, storing offsets as i32.

ListBuilder

Builder for ListArray

ListViewArray

A GenericListViewArray of variable size lists, storing offsets as i32.

ListViewBuilder

Builder for ListViewArray

MapArrayIter

an iterator that returns Some(T) or None, that can be used on any MapArray

PrimitiveIter

an iterator that returns Some(T) or None, that can be used on any PrimitiveArray

StringArray

A GenericStringArray of str using i32 offsets

StringBuilder

Builder for StringArray

StringDictionaryBuilder

Builder for DictionaryArray of StringArray

StringRunBuilder

Builder for RunArray of StringArray

StringViewArray

A GenericByteViewArray that stores utf8 data

StringViewBuilder

Array builder for StringViewArray

Time32MillisecondArray

A PrimitiveArray of milliseconds since midnight stored as i32

Time32MillisecondBufferBuilder

Buffer builder for 32-bit elaspsed time since midnight of millisecond unit.

Time32MillisecondBuilder

A 32-bit elaspsed time in milliseconds array builder.

Time32SecondArray

A PrimitiveArray of seconds since midnight stored as i32

Time32SecondBufferBuilder

Buffer builder for 32-bit elaspsed time since midnight of second unit.

Time32SecondBuilder

A 32-bit elaspsed time in seconds array builder.

Time64MicrosecondArray

A PrimitiveArray of microseconds since midnight stored as i64

Time64MicrosecondBufferBuilder

Buffer builder for 64-bit elaspsed time since midnight of microsecond unit.

Time64MicrosecondBuilder

A 64-bit elaspsed time in microseconds array builder.

Time64NanosecondArray

A PrimitiveArray of nanoseconds since midnight stored as i64

Time64NanosecondBufferBuilder

Buffer builder for 64-bit elaspsed time since midnight of nanosecond unit.

Time64NanosecondBuilder

A 64-bit elaspsed time in nanoseconds array builder.

TimestampMicrosecondArray

A PrimitiveArray of microseconds since UNIX epoch stored as i64

TimestampMicrosecondBufferBuilder

Buffer builder for timestamp type of microsecond unit.

TimestampMicrosecondBuilder

A timestamp microsecond array builder.

TimestampMillisecondArray

A PrimitiveArray of milliseconds since UNIX epoch stored as i64

TimestampMillisecondBufferBuilder

Buffer builder for timestamp type of millisecond unit.

TimestampMillisecondBuilder

A timestamp millisecond array builder.

TimestampNanosecondArray

A PrimitiveArray of nanoseconds since UNIX epoch stored as i64

TimestampNanosecondBufferBuilder

Buffer builder for timestamp type of nanosecond unit.

TimestampNanosecondBuilder

A timestamp nanosecond array builder.

TimestampSecondArray

A PrimitiveArray of seconds since UNIX epoch stored as i64

TimestampSecondBufferBuilder

Buffer builder for timestamp type of second unit.

TimestampSecondBuilder

A timestamp second array builder.

UInt8Array

A PrimitiveArray of u8

UInt8BufferBuilder

Buffer builder for usigned 8-bit integer type.

UInt8Builder

An usigned 8-bit integer array builder.

UInt8DictionaryArray

A DictionaryArray indexed by u8

UInt16Array

A PrimitiveArray of u16

UInt16BufferBuilder

Buffer builder for usigned 16-bit integer type.

UInt16Builder

An usigned 16-bit integer array builder.

UInt16DictionaryArray

A DictionaryArray indexed by u16

UInt32Array

A PrimitiveArray of u32

UInt32BufferBuilder

Buffer builder for usigned 32-bit integer type.

UInt32Builder

An usigned 32-bit integer array builder.

UInt32DictionaryArray

A DictionaryArray indexed by u32

UInt64Array

A PrimitiveArray of u64

UInt64BufferBuilder

Buffer builder for usigned 64-bit integer type.

UInt64Builder

An usigned 64-bit integer array builder.

UInt64DictionaryArray

A DictionaryArray indexed by u64