Module variant_array

Module variant_array 

Source
Expand description

VariantArray implementation

StructsΒ§

BorrowedShreddingState
Similar to ShreddingState except it holds borrowed references of the target arrays. Useful for avoiding clone operations when the caller does not need a self-standing shredding state.
ShreddedVariantFieldArray
One shredded field of a partially or prefectly shredded variant. For example, suppose the shredding schema for variant v treats it as an object with a single field a, where a is itself a struct with the single field b of type INT. Then the physical layout of the column is:
ShreddingState
Represents the shredding state of a VariantArray
StructArrayBuilder πŸ”’
Builds struct arrays from component fields
VariantArray
An array of Parquet [Variant] values
VariantType
Arrow Variant [ExtensionType].

FunctionsΒ§

canonicalize_and_verify_data_type πŸ”’
Recursively visits a data type, ensuring that it only contains data types that can legally appear in a (possibly shredded) variant array. It also replaces Binary fields with BinaryView, since that’s what comes back from the parquet reader and what the variant code expects to find.
canonicalize_and_verify_field πŸ”’
cast_to_binary_view_arrays πŸ”’
Workaround for lack of direct support for BinaryArray https://github.com/apache/arrow-rs/issues/8387
is_valid_variant_decimal πŸ”’
Validates whether a given arrow decimal is a valid variant decimal
typed_value_to_variant πŸ”’
returns the non-null element at index as a Variant