Module variant_array 

VariantArray implementation

Structs

ShreddedVariantFieldArray

One shredded field of a partially or perfectly shredded variant. For example, suppose the shredding schema for variant v treats it as an object with a single field a, where a is itself a struct with the single field b of type INT. Then the physical layout of the column is:

ShreddingState

Represents the shredding state of a VariantArray

StructArrayBuilder 🔒

Builds struct arrays from component fields

VariantArray

An array of Parquet [Variant] values

VariantArrayIter

An iterator over VariantArray

VariantType

Arrow Variant [ExtensionType].

Functions

binary_array_value 🔒

Returns the raw bytes at the given index from a binary-like array, return None if the array isn’t binary-like.

canonicalize_and_verify_data_type 🔒

Recursively visits a data type, ensuring that it only contains data types that can legally appear in a (possibly shredded) variant array. It also narrows decimal types to the smallest valid precision (e.g. Decimal128 -> Decimal32 when the precision fits).

canonicalize_and_verify_field 🔒

canonicalize_shredded_types 🔒

Canonicalize shredded typed_value fields (e.g. decimal narrowing) and verify that all data types in the struct are legal for a variant array.

typed_value_field 🔒

Build the typed_value [FieldRef] for a shredded column.

typed_value_to_variant 🔒

returns the non-null element at index as a Variant

validate_binary_array 🔒

Validates that an array has a binary-like data type.

variant_from_arrays_at 🔒

Returns a [Variant] from a metadata and value byte arrays, returns None if one of them is of invalid type.