Struct VariantArray 

pub struct VariantArray {
    inner: StructArray,
    metadata: BinaryViewArray,
    shredding_state: ShreddingState,
}

An array of Parquet [Variant] values

A VariantArray wraps an Arrow [StructArray] that stores the underlying metadata and value fields, and adds convenience methods to access the [Variant]s.

See VariantArrayBuilder for constructing VariantArray row by row.

See the examples below from converting between VariantArray and StructArray.

Documentation

At the time of this writing, Variant has been accepted as an official extension type but not been published to the official list of extension types on the Apache Arrow website. See the Extension Type for Parquet Variant arrow ticket for more details.

Example: Check if a [StructArray] has the VariantType extension

Arrow Arrays only provide [DataType], but the extension type information is stored on a [Field]. Thus, you must have access to the Schema or [Field] to check for the extension type.

let schema = get_schema();
assert_eq!(schema.fields().len(), 2);
// first field is not a Variant
assert!(schema.field(0).try_extension_type::<VariantType>().is_err());
// second field is a Variant
assert!(schema.field(1).try_extension_type::<VariantType>().is_ok());

Example: Constructing the correct [Field] for a VariantArray

You can construct the correct [Field] for a VariantArray using the VariantArray::field method.

let variant_array = get_variant_array();
// First field is an integer id, second field is a variant
let schema = Schema::new(vec![
  Field::new("id", DataType::Int32, false),
  // call VariantArray::field to get the correct Field
  variant_array.field("var"),
]);

You can also construct the [Field] using VariantType directly

// The DataType of a VariantArray varies depending on how it is shredded
let data_type = variant_array.data_type().clone();
// First field is an integer id, second field is a variant
let schema = Schema::new(vec![
  Field::new("id", DataType::Int32, false),
  Field::new("var", data_type, false)
    // Add extension metadata to the field using `VariantType`
    .with_extension_type(VariantType),
]);

Example: Converting a VariantArray to a [StructArray]

// Create Variant Array
let mut builder = VariantArrayBuilder::new(10);
builder.append_variant(Variant::from("such wow"));
let variant_array = builder.build();
// convert to StructArray
let struct_array: StructArray = variant_array.into();

Example: Converting a [StructArray] to a VariantArray

let struct_array: StructArray = get_struct_array();
// try and create a VariantArray from it
let variant_array = VariantArray::try_new(&struct_array).unwrap();
assert_eq!(variant_array.value(0), Variant::from("such wow"));

Fields

inner: StructArray

Reference to the underlying StructArray

metadata: BinaryViewArray

The metadata column of this variant

shredding_state: ShreddingState

how is this variant array shredded?

Implementations

impl VariantArray

pub fn try_new(inner: &dyn Array) -> Result<Self>

Creates a new VariantArray from a [StructArray].

Arguments

• inner - The underlying [StructArray] that contains the variant data.

Returns

• A new instance of VariantArray.

Errors:

• If the StructArray does not contain the required fields

Requirements of the StructArray

1. A required field named metadata which is binary, large_binary, or binary_view

2. An optional field named value that is binary, large_binary, or binary_view

3. An optional field named typed_value which can be any primitive type or be a list, large_list, list_view or struct

NOTE: It is also permissible for the metadata field to be Dictionary-Encoded, preferably (but not required) with an index type of int8.

Currently, only [BinaryViewArray] are supported.

pub(crate) fn from_parts( metadata: BinaryViewArray, value: Option<BinaryViewArray>, typed_value: Option<ArrayRef>, nulls: Option<NullBuffer>, ) -> Self

pub fn inner(&self) -> &StructArray

Returns a reference to the underlying [StructArray].

pub fn into_inner(self) -> StructArray

Returns the inner [StructArray], consuming self

pub fn shredding_state(&self) -> &ShreddingState

Return the shredding state of this VariantArray

pub fn value(&self, index: usize) -> Variant<'_, '_>

Return the [Variant] instance stored at the given row

This is a convenience wrapper that calls VariantArray::try_value and unwraps the Result. Use try_value if you need to handle conversion errors gracefully.

Panics

• if the index is out of bounds
• if the array value is null
• if try_value returns an error.

pub fn try_value(&self, index: usize) -> Result<Variant<'_, '_>>

Return the [Variant] instance stored at the given row

Note: This method does not check for nulls and the value is arbitrary (but still well-defined) if is_null returns true for the index.

Panics

Panics if

• the index is out of bounds
• the array value is null

Errors

Errors if

• the data in typed_value cannot be interpreted as a valid Variant

If this is a shredded variant but has no value at the shredded location, it will return [Variant::Null].

Performance Note

This is certainly not the most efficient way to access values in a VariantArray, but it is useful for testing and debugging.

Note: Does not do deep validation of the [Variant], so it is up to the caller to ensure that the metadata and value were constructed correctly.

pub fn metadata_field(&self) -> &BinaryViewArray

Return a reference to the metadata field of the [StructArray]

pub fn value_field(&self) -> Option<&BinaryViewArray>

Return a reference to the value field of the StructArray

pub fn typed_value_field(&self) -> Option<&ArrayRef>

Return a reference to the typed_value field of the StructArray, if present

pub fn field(&self, name: impl Into<String>) -> Field

Return a field to represent this VariantArray in a Schema with a particular name

pub fn data_type(&self) -> &DataType

Returns a new DataType representing this VariantArray’s inner type

pub fn slice(&self, offset: usize, length: usize) -> Self

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn nulls(&self) -> Option<&NullBuffer>

pub fn is_null(&self, index: usize) -> bool

Is the element at index null?

pub fn is_valid(&self, index: usize) -> bool

Is the element at index valid (not null)?

pub fn iter(&self) -> VariantArrayIter<'_>

Returns an iterator over the values in this array

Trait Implementations

impl Clone for VariantArray

fn clone(&self) -> VariantArray

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for VariantArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<VariantArray> for ArrayRef

fn from(variant_array: VariantArray) -> Self

Converts to this type from the input type.

impl From<VariantArray> for StructArray

fn from(variant_array: VariantArray) -> Self

Converts to this type from the input type.

impl<'m, 'v> FromIterator<Option<Variant<'m, 'v>>> for VariantArray

fn from_iter<T: IntoIterator<Item = Option<Variant<'m, 'v>>>>(iter: T) -> Self

Creates a value from an iterator.

impl<'m, 'v> FromIterator<Variant<'m, 'v>> for VariantArray

fn from_iter<T: IntoIterator<Item = Variant<'m, 'v>>>(iter: T) -> Self

Creates a value from an iterator.

impl PartialEq for VariantArray

fn eq(&self, other: &VariantArray) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for VariantArray