Expand description
Statically typed implementations of Arrow Arrays
See [arrow_array] for examples and usage instructions
Modules§
- The concrete array definitions
- Defines push-based APIs for constructing arrays
- Defines helper functions for downcasting
dyn Array
to concrete types - Contains declarations to bind to the C Data Interface.
- Contains declarations to bind to the C Stream Interface.
- Idiomatic iterators for
Array
- Idiomatic iterator for
RunArray
- Conversion methods for dates and times.
- Timezone for timestamp arrays
- Zero-sized types used to parameterize generic array implementations
Macros§
- Downcast an
Array
to aDictionaryArray
based on itsDataType
, accepts a number of subsequent patterns to match the data type - Given one or more expressions evaluating to an integer
DataType
invokes the provided macrom
with the corresponding integerArrowPrimitiveType
, followed by any additional arguments - Given one or more expressions evaluating to primitive
DataType
invokes the provided macrom
with the correspondingArrowPrimitiveType
, followed by any additional arguments - Downcast an
Array
to aPrimitiveArray
based on itsDataType
accepts a number of subsequent patterns to match the data type - Given one or more expressions evaluating to an integer
DataType
invokes the provided macrom
with the corresponding integerRunEndIndexType
, followed by any additional arguments - Given one or more expressions evaluating to primitive
DataType
invokes the provided macrom
with the correspondingArrowPrimitiveType
, followed by any additional arguments - Downcast an
Array
to a temporalPrimitiveArray
based on itsDataType
accepts a number of subsequent patterns to match the data type
Structs§
- A generic representation of Arrow array data which encapsulates common attributes and operations for Arrow array.
- Builder for
ArrayData
type - An iterator that returns Some(T) or None, that can be used on any
ArrayAccessor
- An array of boolean values
- Builder for
BooleanBuffer
- Builder for
BooleanArray
- Builder for creating a Buffer object.
- Helper to access views of
GenericByteViewArray
(StringViewArray
andBinaryViewArray
) where the length is greater than 12 bytes. - Layout specification for a data type
- An array of dictionary encoded values
- An array of fixed size binary arrays
- Builder for
FixedSizeBinaryArray
- An array of [fixed length lists], similar to JSON arrays (e.g.
["A", "B"]
). - Builder for
FixedSizeListArray
- An array of variable length byte arrays
- Builder for
GenericByteArray
- Builder for
DictionaryArray
ofGenericByteArray
- Builder for
RunArray
ofGenericByteArray
- Variable-size Binary View Layout: An array of variable length bytes view arrays.
- A builder for
GenericByteViewArray
- An array of variable length lists, similar to JSON arrays (e.g.
["A", "B", "C"]
). - Builder for
GenericListArray
- Different from
crate::GenericListArray
as it stores both an offset and length meaning that take / filter operations can be implemented without copying the underlying data. - An array of key-value maps
- Builder for
MapArray
- An optional primitive value
- An array of null values
- Builder for
NullArray
- An array of primitive values, of type
ArrowPrimitiveType
- Builder for
PrimitiveArray
- Builder for
DictionaryArray
ofPrimitiveArray
- Builder for
RunArray
ofPrimitiveArray
- A two-dimensional batch of column-oriented data with a defined schema.
- Generic implementation of RecordBatchReader that wraps an iterator.
- Options that control the behaviour used when creating a
RecordBatch
. - An array of run-end encoded values
- An array of structs
- Builder for
StructArray
- A
DictionaryArray
typed on its child values array - A
RunArray
typed typed on its child values array - An array of values of varying types
- Builder for
UnionArray
Enums§
- Layout specification for a single data type buffer
- Define capacities to pre-allocate for child data or data buffers.
Traits§
- A
DictionaryArray
with the key type erased - An array in the arrow columnar format
- A generic trait for accessing the values of an
Array
- Trait for dealing with different array builders at runtime
- Trait for
ArrowNativeType
that adds checked and unchecked arithmetic operations, and totally ordered comparison operations - A subtype of primitive type that represents numeric values.
- Trait for primitive values.
- An extension trait for
dyn Array
that provides ergonomic downcasting - A type that can be used within a variable-size array to encode offset information
- Trait for types that can read
RecordBatch
’s. - Trait for types that can write
RecordBatch
’s.
Functions§
- as_
decimal_ array Deprecated Force downcast of an Array, such as an ArrayRef to Decimal128Array, panic’ing on failure. - Downcasts a
dyn Array
to a concrete type - export_
array_ ⚠into_ raw Deprecated Exports an array to raw pointers of the C Data Interface provided by the consumer. - Return the expected
DataTypeLayout
Arrays of this data type are expected to have - Constructs an array using the input
data
. Returns a reference-countedArray
instance. - Returns a builder with capacity
capacity
that corresponds to the datatypeDataType
This function is useful to construct arrays from an arbitrary vectors with known/expected schema. - Returns a comparison function that compares two values at two different positions between the two arrays.
- Create a view based on the given data, block id and offset.
- Creates a new empty array
- Creates a new array of
data_type
of lengthlength
filled entirely ofNULL
values
Type Aliases§
- A thread-safe, shared reference to the Arrow array data.
- A reference-counted reference to a generic
Array
- Builder for
BinaryArray
- Builder for
DictionaryArray
ofBinaryArray
- Builder for
RunArray
ofBinaryArray
- A
GenericByteViewArray
of[u8]
- Array builder for
BinaryViewArray
- an iterator that returns Some(T) or None, that can be used on any BooleanArray
- A
PrimitiveArray
of days since UNIX epoch stored asi32
- Buffer builder for 32-bit date type.
- A 32-bit date array builder.
- A
PrimitiveArray
of milliseconds since UNIX epoch stored asi64
- Buffer builder for 64-bit date type.
- A 64-bit date array builder.
- A
PrimitiveArray
of 128-bit fixed point decimals - Buffer builder for 128-bit decimal type.
- A decimal 128 array builder
- A
PrimitiveArray
of 256-bit fixed point decimals - Buffer builder for 256-bit decimal type.
- A decimal 256 array builder
- A
PrimitiveArray
of elapsed durations in microseconds - Buffer builder for elaspsed time of microseconds unit.
- An elapsed time in microseconds array builder.
- A
PrimitiveArray
of elapsed durations in milliseconds - Buffer builder for elaspsed time of milliseconds unit.
- An elapsed time in milliseconds array builder.
- A
PrimitiveArray
of elapsed durations in nanoseconds - Buffer builder for elaspsed time of nanoseconds unit.
- An elapsed time in nanoseconds array builder.
- A
PrimitiveArray
of elapsed durations in seconds - Buffer builder for elaspsed time of second unit.
- An elapsed time in seconds array builder.
- Compare the values at two arbitrary indices in two arrays.
- an iterator that returns Some(T) or None, that can be used on any FixedSizeBinaryArray
- an iterator that returns Some(T) or None, that can be used on any FixedSizeListArray
- A
PrimitiveArray
off16
- Buffer builder for 16-bit floating point type.
- A 16-bit floating point array builder.
- A
PrimitiveArray
off32
- Buffer builder for 32-bit floating point type.
- A 32-bit floating point array builder.
- A
PrimitiveArray
off64
- Buffer builder for 64-bit floating point type.
- A 64-bit floating point array builder.
- A
GenericBinaryArray
for storing[u8]
- Array builder for
GenericBinaryArray
- an iterator that returns Some(T) or None, that can be used on any BinaryArray
- an iterator that returns Some(T) or None, that can be used on any ListArray
- an iterator that returns Some(T) or None, that can be used on any ListArray
- A
GenericByteArray
for storingstr
- Array builder for
GenericStringArray
- an iterator that returns Some(T) or None, that can be used on any Utf8Array
- A
PrimitiveArray
ofi8
- Buffer builder for signed 8-bit integer type.
- A signed 8-bit integer array builder.
- A
DictionaryArray
indexed byi8
- A
PrimitiveArray
ofi16
- Buffer builder for signed 16-bit integer type.
- A signed 16-bit integer array builder.
- A
DictionaryArray
indexed byi16
- A
RunArray
withi16
run ends - A
PrimitiveArray
ofi32
- Buffer builder for signed 32-bit integer type.
- A signed 32-bit integer array builder.
- A
DictionaryArray
indexed byi32
- A
RunArray
withi32
run ends - A
PrimitiveArray
ofi64
- Buffer builder for signed 64-bit integer type.
- A signed 64-bit integer array builder.
- A
DictionaryArray
indexed byi64
- A
RunArray
withi64
run ends - A
PrimitiveArray
of “calendar” intervals in days and milliseconds - Buffer builder for “calendar” interval in days and milliseconds.
- A “calendar” interval in days and milliseconds array builder.
- A
PrimitiveArray
of “calendar” intervals in months, days, and nanoseconds. - Buffer builder “calendar” interval in months, days, and nanoseconds.
- A “calendar” interval in months, days, and nanoseconds array builder.
- A
PrimitiveArray
of “calendar” intervals in whole months - Buffer builder for “calendar” interval in months.
- A “calendar” interval in months array builder.
- Builder for
LargeBinaryArray
- Builder for
DictionaryArray
ofLargeBinaryArray
- Builder for
RunArray
ofLargeBinaryArray
- A
GenericListArray
of variable size lists, storing offsets asi64
. - Builder for
LargeListArray
- A
GenericListViewArray
of variable size lists, storing offsets asi64
. - Builder for
LargeStringArray
- Builder for
DictionaryArray
ofLargeStringArray
- Builder for
RunArray
ofLargeStringArray
- A
GenericListArray
of variable size lists, storing offsets asi32
. - Builder for
ListArray
- A
GenericListViewArray
of variable size lists, storing offsets asi32
. - an iterator that returns Some(T) or None, that can be used on any MapArray
- an iterator that returns Some(T) or None, that can be used on any PrimitiveArray
- Builder for
StringArray
- Builder for
DictionaryArray
ofStringArray
- Builder for
RunArray
ofStringArray
- A
GenericByteViewArray
that stores utf8 data - Array builder for
StringViewArray
- A
PrimitiveArray
of milliseconds since midnight stored asi32
- Buffer builder for 32-bit elaspsed time since midnight of millisecond unit.
- A 32-bit elaspsed time in milliseconds array builder.
- A
PrimitiveArray
of seconds since midnight stored asi32
- Buffer builder for 32-bit elaspsed time since midnight of second unit.
- A 32-bit elaspsed time in seconds array builder.
- A
PrimitiveArray
of microseconds since midnight stored asi64
- Buffer builder for 64-bit elaspsed time since midnight of microsecond unit.
- A 64-bit elaspsed time in microseconds array builder.
- A
PrimitiveArray
of nanoseconds since midnight stored asi64
- Buffer builder for 64-bit elaspsed time since midnight of nanosecond unit.
- A 64-bit elaspsed time in nanoseconds array builder.
- A
PrimitiveArray
of microseconds since UNIX epoch stored asi64
- Buffer builder for timestamp type of microsecond unit.
- A timestamp microsecond array builder.
- A
PrimitiveArray
of milliseconds since UNIX epoch stored asi64
- Buffer builder for timestamp type of millisecond unit.
- A timestamp millisecond array builder.
- A
PrimitiveArray
of nanoseconds since UNIX epoch stored asi64
- Buffer builder for timestamp type of nanosecond unit.
- A timestamp nanosecond array builder.
- A
PrimitiveArray
of seconds since UNIX epoch stored asi64
- Buffer builder for timestamp type of second unit.
- A timestamp second array builder.
- A
PrimitiveArray
ofu8
- Buffer builder for usigned 8-bit integer type.
- An usigned 8-bit integer array builder.
- A
DictionaryArray
indexed byu8
- A
PrimitiveArray
ofu16
- Buffer builder for usigned 16-bit integer type.
- An usigned 16-bit integer array builder.
- A
DictionaryArray
indexed byu16
- A
PrimitiveArray
ofu32
- Buffer builder for usigned 32-bit integer type.
- An usigned 32-bit integer array builder.
- A
DictionaryArray
indexed byu32
- A
PrimitiveArray
ofu64
- Buffer builder for usigned 64-bit integer type.
- An usigned 64-bit integer array builder.
- A
DictionaryArray
indexed byu64