Module variant_array

Expand description

VariantArray implementation

Structs§

BorrowedShreddingState: Similar to ShreddingState except it holds borrowed references of the target arrays. Useful for avoiding clone operations when the caller does not need a self-standing shredding state.
ShreddedVariantFieldArray: One shredded field of a partially or perfectly shredded variant. For example, suppose the shredding schema for variant v treats it as an object with a single field a, where a is itself a struct with the single field b of type INT. Then the physical layout of the column is:
ShreddingState: Represents the shredding state of a VariantArray
StructArrayBuilder 🔒: Builds struct arrays from component fields
VariantArray: An array of Parquet [Variant] values
VariantArrayIter: An iterator over VariantArray
VariantType: Arrow Variant [ExtensionType].

binary_array_value 🔒: Returns the raw bytes at the given index from a binary-like array, return None if the array isn’t binary-like.
canonicalize_and_verify_data_type 🔒: Recursively visits a data type, ensuring that it only contains data types that can legally appear in a (possibly shredded) variant array. It also narrows decimal types to the smallest valid precision (e.g. Decimal128 -> Decimal32 when the precision fits).
canonicalize_and_verify_field 🔒
canonicalize_shredded_types 🔒: Canonicalize shredded typed_value fields (e.g. decimal narrowing) and verify that all data types in the struct are legal for a variant array.
typed_value_to_variant 🔒: returns the non-null element at index as a Variant
validate_binary_array 🔒: Validates that an array has a binary-like data type.
variant_from_arrays_at 🔒: Returns a [Variant] from a metadata and value byte arrays, returns None if one of them is of invalid type.