Module variant_array 

VariantArray implementation

Re-exports

use crate::VariantArrayBuilder; 🔒

use crate::type_conversion::generic_conversion_single_value; 🔒

use crate::type_conversion::generic_conversion_single_value_with_result; 🔒

use crate::type_conversion::primitive_conversion_single_value; 🔒

use arrow::array::Array; 🔒

use arrow::array::ArrayRef; 🔒

use arrow::array::AsArray; 🔒

use arrow::array::StructArray; 🔒

use arrow::array::StructArrayAssembler; 🔒

use arrow::array::downcast_dictionary_array; 🔒

use arrow::array::downcast_run_array; 🔒

use arrow::array::new_null_array; 🔒

use arrow::buffer::NullBuffer; 🔒

use arrow::compute::cast; 🔒

use arrow::datatypes::Date32Type; 🔒

use arrow::datatypes::Decimal32Type; 🔒

use arrow::datatypes::Decimal64Type; 🔒

use arrow::datatypes::Decimal128Type; 🔒

use arrow::datatypes::Float16Type; 🔒

use arrow::datatypes::Float32Type; 🔒

use arrow::datatypes::Float64Type; 🔒

use arrow::datatypes::Int8Type; 🔒

use arrow::datatypes::Int16Type; 🔒

use arrow::datatypes::Int32Type; 🔒

use arrow::datatypes::Int64Type; 🔒

use arrow::datatypes::Time64MicrosecondType; 🔒

use arrow::datatypes::TimestampMicrosecondType; 🔒

use arrow::datatypes::TimestampNanosecondType; 🔒

use arrow::error::Result; 🔒

use arrow_schema::extension::Uuid as UuidExtension; 🔒

use arrow_schema::ArrowError; 🔒

use arrow_schema::DataType; 🔒

use arrow_schema::Field; 🔒

use arrow_schema::FieldRef; 🔒

use arrow_schema::TimeUnit; 🔒

use chrono::DateTime; 🔒

use chrono::NaiveTime; 🔒

use parquet_variant::Uuid; 🔒

use parquet_variant::Variant; 🔒

use parquet_variant::VariantDecimal4; 🔒

use parquet_variant::VariantDecimal8; 🔒

use parquet_variant::VariantDecimal16; 🔒

use parquet_variant::VariantDecimalType as _; 🔒

use parquet_variant::VariantType; 🔒

use std::borrow::Cow; 🔒

use std::sync::Arc; 🔒

Structs

ShreddedVariantFieldArray

One shredded field of a partially or perfectly shredded variant. For example, suppose the shredding schema for variant v treats it as an object with a single field a, where a is itself a struct with the single field b of type INT. Then the physical layout of the column is:

ShreddingState

Represents the shredding state of a VariantArray

VariantArray

An array of Parquet [Variant] values

VariantArrayIter

An iterator over VariantArray

Functions

all_null_value_column 🔒

Returns an all-null binary value column of the given length.

binary_array_value 🔒

Returns the logical bytes at the given index from a binary-like array, resolving dictionary and run-end encodings. Returns None for nulls or if the logical values aren’t binary-like.

canonicalize_and_verify_data_type 🔒

Recursively visits a data type, ensuring that it only contains data types that can legally appear in a (possibly shredded) variant array. It also narrows decimal types to the smallest valid precision (e.g. Decimal128 -> Decimal32 when the precision fits).

canonicalize_and_verify_data_type_impl 🔒

canonicalize_and_verify_field 🔒

canonicalize_shredded_types 🔒

Canonicalize shredded typed_value fields (e.g. decimal narrowing) and verify that all data types in the struct are legal for a variant array.

typed_value_field 🔒

Build the typed_value [FieldRef] for a shredded column.

typed_value_to_variant 🔒

returns the non-null element at index as a Variant

validate_binary_array 🔒

Validates that an array has a binary-like data type.

validate_metadata_array 🔒

Validates that a metadata array has binary-like logical values.

variant_from_arrays_at 🔒

Returns a [Variant] from a metadata and value byte arrays, returns None if one of them is of invalid type.