Struct VariantArray 

pub struct VariantArray {
    inner: StructArray,
    metadata: ArrayRef,
    shredding_state: ShreddingState,
}

An array of Parquet [Variant] values

A VariantArray wraps an Arrow [StructArray] that stores the underlying metadata and value fields, and adds convenience methods to access the [Variant]s.

See VariantArrayBuilder for constructing VariantArray row by row.

See the examples below from converting between VariantArray and StructArray.

Documentation

Variant is documented as a canonical Arrow extension type in the Parquet Variant section of the official list of extension types on the Apache Arrow website.

Example: Check if a [StructArray] has the VariantType extension

Arrow Arrays only provide [DataType], but the extension type information is stored on a [Field]. Thus, you must have access to the Schema or [Field] to check for the extension type.

let schema = get_schema();
assert_eq!(schema.fields().len(), 2);
// first field is not a Variant
assert!(!schema.field(0).has_valid_extension_type::<VariantType>());
// second field is a Variant
assert!(schema.field(1).has_valid_extension_type::<VariantType>());

Example: Constructing the correct [Field] for a VariantArray

You can construct the correct [Field] for a VariantArray using the VariantArray::field method.

let variant_array = get_variant_array();
// First field is an integer id, second field is a variant
let schema = Schema::new(vec![
  Field::new("id", DataType::Int32, false),
  // call VariantArray::field to get the correct Field
  variant_array.field("var"),
]);

You can also construct the [Field] using VariantType directly

// The DataType of a VariantArray varies depending on how it is shredded
let data_type = variant_array.data_type().clone();
// First field is an integer id, second field is a variant
let schema = Schema::new(vec![
  Field::new("id", DataType::Int32, false),
  Field::new("var", data_type, false)
    // Add extension metadata to the field using `VariantType`
    .with_extension_type(VariantType),
]);

Example: Converting a VariantArray to a [StructArray]

// Create Variant Array
let mut builder = VariantArrayBuilder::new(10);
builder.append_variant(Variant::from("such wow"));
let variant_array = builder.build();
// convert to StructArray
let struct_array: StructArray = variant_array.into();

Example: Converting a [StructArray] to a VariantArray

let struct_array: StructArray = get_struct_array();
// try and create a VariantArray from it
let variant_array = VariantArray::try_new(&struct_array).unwrap();
assert_eq!(variant_array.value(0), Variant::from("such wow"));

Fields

inner: StructArray

Reference to the underlying StructArray

metadata: ArrayRef

The metadata column of this variant (Binary, LargeBinary, or BinaryView)

shredding_state: ShreddingState

how is this variant array shredded?

Implementations

impl VariantArray

pub fn try_new(inner: &dyn Array) -> Result<Self>

Creates a new VariantArray from a [StructArray].

Arguments

• inner - The underlying [StructArray] that contains the variant data.

Returns

• A new instance of VariantArray.

Errors:

• If the StructArray does not contain the required fields

Requirements of the StructArray

1. A required field named metadata which is binary, large_binary, or binary_view

2. An optional field named value that is binary, large_binary, or binary_view

3. An optional field named typed_value which can be any primitive type or be a list, large_list, list_view or struct

NOTE: It is also permissible for the metadata field to be Dictionary-Encoded, preferably (but not required) with an index type of int8.

pub(crate) fn from_parts( metadata: ArrayRef, value: Option<ArrayRef>, typed_value: Option<ArrayRef>, nulls: Option<NullBuffer>, ) -> Self

pub fn inner(&self) -> &StructArray

Returns a reference to the underlying [StructArray].

pub fn into_inner(self) -> StructArray

Returns the inner [StructArray], consuming self

pub fn shredding_state(&self) -> &ShreddingState

Return the shredding state of this VariantArray

pub fn value(&self, index: usize) -> Variant<'_, '_>

Return the [Variant] instance stored at the given row

This is a convenience wrapper that calls VariantArray::try_value and unwraps the Result. Use try_value if you need to handle conversion errors gracefully.

Panics

• if the index is out of bounds
• if the array value is null
• if try_value returns an error.

pub fn try_value(&self, index: usize) -> Result<Variant<'_, '_>>

Return the [Variant] instance stored at the given row

Note: This method does not check for nulls and the value is arbitrary (but still well-defined) if is_null returns true for the index.

Panics

Panics if

• the index is out of bounds
• the array value is null

Errors

Errors if

• the data in typed_value cannot be interpreted as a valid Variant

If this is a shredded variant but has no value at the shredded location, it will return [Variant::Null].

Performance Note

This is certainly not the most efficient way to access values in a VariantArray, but it is useful for testing and debugging.

Note: Does not do deep validation of the [Variant], so it is up to the caller to ensure that the metadata and value were constructed correctly.

pub fn metadata_field(&self) -> &ArrayRef

Return a reference to the metadata field of the [StructArray]

pub fn value_field(&self) -> Option<&ArrayRef>

Return a reference to the value field of the StructArray

pub fn typed_value_field(&self) -> Option<&ArrayRef>

Return a reference to the typed_value field of the StructArray, if present

pub fn field(&self, name: impl Into<String>) -> Field

Return a field to represent this VariantArray in a Schema with a particular name

pub fn data_type(&self) -> &DataType

Returns a new DataType representing this VariantArray’s inner type

pub fn slice(&self, offset: usize, length: usize) -> Self

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn nulls(&self) -> Option<&NullBuffer>

pub fn is_null(&self, index: usize) -> bool

Is the element at index null?

pub fn is_valid(&self, index: usize) -> bool

Is the element at index valid (not null)?

pub fn iter(&self) -> VariantArrayIter<'_>

Returns an iterator over the values in this array

Trait Implementations

impl Clone for VariantArray

fn clone(&self) -> VariantArray

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for VariantArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<VariantArray> for ArrayRef

fn from(variant_array: VariantArray) -> Self

Converts to this type from the input type.

impl From<VariantArray> for StructArray

fn from(variant_array: VariantArray) -> Self

Converts to this type from the input type.

impl<'m, 'v> FromIterator<Option<Variant<'m, 'v>>> for VariantArray

fn from_iter<T: IntoIterator<Item = Option<Variant<'m, 'v>>>>(iter: T) -> Self

Creates a value from an iterator.

impl<'m, 'v> FromIterator<Variant<'m, 'v>> for VariantArray

fn from_iter<T: IntoIterator<Item = Variant<'m, 'v>>>(iter: T) -> Self

Creates a value from an iterator.

impl PartialEq for VariantArray

fn eq(&self, other: &Self) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.