VariantArray

Struct VariantArray 

pub struct VariantArray {
    inner: StructArray,
    metadata: GenericByteViewArray<BinaryViewType>,
    shredding_state: ShreddingState,
}
Expand description

An array of Parquet Variant values

A VariantArray wraps an Arrow [StructArray] that stores the underlying metadata and value fields, and adds convenience methods to access the Variants.

See VariantArrayBuilder for constructing VariantArray row by row.

See the examples below from converting between VariantArray and StructArray.

§Documentation

At the time of this writing, Variant has been accepted as an official extension type but not been published to the official list of extension types on the Apache Arrow website. See the Extension Type for Parquet Variant arrow ticket for more details.

§Example: Check if a [StructArray] has the VariantType extension

Arrow Arrays only provide [DataType], but the extension type information is stored on a [Field]. Thus, you must have access to the Schema or [Field] to check for the extension type.

let schema = get_schema();
assert_eq!(schema.fields().len(), 2);
// first field is not a Variant
assert!(schema.field(0).try_extension_type::<VariantType>().is_err());
// second field is a Variant
assert!(schema.field(1).try_extension_type::<VariantType>().is_ok());

§Example: Constructing the correct [Field] for a VariantArray

You can construct the correct [Field] for a VariantArray using the VariantArray::field method.

let variant_array = get_variant_array();
// First field is an integer id, second field is a variant
let schema = Schema::new(vec![
  Field::new("id", DataType::Int32, false),
  // call VariantArray::field to get the correct Field
  variant_array.field("var"),
]);

You can also construct the [Field] using VariantType directly

// The DataType of a VariantArray varies depending on how it is shredded
let data_type = variant_array.data_type().clone();
// First field is an integer id, second field is a variant
let schema = Schema::new(vec![
  Field::new("id", DataType::Int32, false),
  Field::new("var", data_type, false)
    // Add extension metadata to the field using `VariantType`
    .with_extension_type(VariantType),
]);

§Example: Converting a VariantArray to a [StructArray]

// Create Variant Array
let mut builder = VariantArrayBuilder::new(10);
builder.append_variant(Variant::from("such wow"));
let variant_array = builder.build();
// convert to StructArray
let struct_array: StructArray = variant_array.into();

§Example: Converting a [StructArray] to a VariantArray

let struct_array: StructArray = get_struct_array();
// try and create a VariantArray from it
let variant_array = VariantArray::try_new(&struct_array).unwrap();
assert_eq!(variant_array.value(0), Variant::from("such wow"));

Fields§

§inner: StructArray§metadata: GenericByteViewArray<BinaryViewType>§shredding_state: ShreddingState

Implementations§

§

impl VariantArray

pub fn try_new(inner: &dyn Array) -> Result<VariantArray, ArrowError>

Creates a new VariantArray from a [StructArray].

§Arguments
  • inner - The underlying [StructArray] that contains the variant data.
§Returns
  • A new instance of VariantArray.
§Errors:
  • If the StructArray does not contain the required fields
§Requirements of the StructArray
  1. A required field named metadata which is binary, large_binary, or binary_view

  2. An optional field named value that is binary, large_binary, or binary_view

  3. An optional field named typed_value which can be any primitive type or be a list, large_list, list_view or struct

NOTE: It is also permissible for the metadata field to be Dictionary-Encoded, preferably (but not required) with an index type of int8.

Currently, only [BinaryViewArray] are supported.

pub fn inner(&self) -> &StructArray

Returns a reference to the underlying [StructArray].

pub fn into_inner(self) -> StructArray

Returns the inner [StructArray], consuming self

pub fn shredding_state(&self) -> &ShreddingState

Return the shredding state of this VariantArray

pub fn value(&self, index: usize) -> Variant<'_, '_>

Return the Variant instance stored at the given row

Note: This method does not check for nulls and the value is arbitrary (but still well-defined) if is_null returns true for the index.

§Panics
  • if the index is out of bounds
  • if the array value is null

If this is a shredded variant but has no value at the shredded location, it will return Variant::Null.

§Performance Note

This is certainly not the most efficient way to access values in a VariantArray, but it is useful for testing and debugging.

Note: Does not do deep validation of the Variant, so it is up to the caller to ensure that the metadata and value were constructed correctly.

pub fn metadata_field(&self) -> &GenericByteViewArray<BinaryViewType>

Return a reference to the metadata field of the [StructArray]

pub fn value_field(&self) -> Option<&GenericByteViewArray<BinaryViewType>>

Return a reference to the value field of the StructArray

pub fn typed_value_field(&self) -> Option<&Arc<dyn Array>>

Return a reference to the typed_value field of the StructArray, if present

pub fn field(&self, name: impl Into<String>) -> Field

Return a field to represent this VariantArray in a Schema with a particular name

pub fn data_type(&self) -> &DataType

Returns a new DataType representing this VariantArray’s inner type

pub fn slice(&self, offset: usize, length: usize) -> VariantArray

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn nulls(&self) -> Option<&NullBuffer>

pub fn is_null(&self, index: usize) -> bool

Is the element at index null?

pub fn is_valid(&self, index: usize) -> bool

Is the element at index valid (not null)?

Trait Implementations§

§

impl Clone for VariantArray

§

fn clone(&self) -> VariantArray

Returns a duplicate of the value. Read more
1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
§

impl Debug for VariantArray

§

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more
§

impl From<VariantArray> for Arc<dyn Array>

§

fn from(variant_array: VariantArray) -> Arc<dyn Array>

Converts to this type from the input type.

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<T> ErasedDestructor for T
where T: 'static,

§

impl<T> Ungil for T
where T: Send,