VariantArray

Struct VariantArray 

Source
pub struct VariantArray {
    inner: StructArray,
    metadata: BinaryViewArray,
    shredding_state: ShreddingState,
}
Expand description

An array of Parquet [Variant] values

A VariantArray wraps an Arrow [StructArray] that stores the underlying metadata and value fields, and adds convenience methods to access the [Variant]s.

See VariantArrayBuilder for constructing VariantArray row by row.

See the examples below from converting between VariantArray and StructArray.

§Documentation

At the time of this writing, Variant has been accepted as an official extension type but not been published to the official list of extension types on the Apache Arrow website. See the Extension Type for Parquet Variant arrow ticket for more details.

§Example: Check if a [StructArray] has the VariantType extension

Arrow Arrays only provide [DataType], but the extension type information is stored on a [Field]. Thus, you must have access to the Schema or [Field] to check for the extension type.

let schema = get_schema();
assert_eq!(schema.fields().len(), 2);
// first field is not a Variant
assert!(schema.field(0).try_extension_type::<VariantType>().is_err());
// second field is a Variant
assert!(schema.field(1).try_extension_type::<VariantType>().is_ok());

§Example: Constructing the correct [Field] for a VariantArray

You can construct the correct [Field] for a VariantArray using the VariantArray::field method.

let variant_array = get_variant_array();
// First field is an integer id, second field is a variant
let schema = Schema::new(vec![
  Field::new("id", DataType::Int32, false),
  // call VariantArray::field to get the correct Field
  variant_array.field("var"),
]);

You can also construct the [Field] using VariantType directly

// The DataType of a VariantArray varies depending on how it is shredded
let data_type = variant_array.data_type().clone();
// First field is an integer id, second field is a variant
let schema = Schema::new(vec![
  Field::new("id", DataType::Int32, false),
  Field::new("var", data_type, false)
    // Add extension metadata to the field using `VariantType`
    .with_extension_type(VariantType),
]);

§Example: Converting a VariantArray to a [StructArray]

// Create Variant Array
let mut builder = VariantArrayBuilder::new(10);
builder.append_variant(Variant::from("such wow"));
let variant_array = builder.build();
// convert to StructArray
let struct_array: StructArray = variant_array.into();

§Example: Converting a [StructArray] to a VariantArray

let struct_array: StructArray = get_struct_array();
// try and create a VariantArray from it
let variant_array = VariantArray::try_new(&struct_array).unwrap();
assert_eq!(variant_array.value(0), Variant::from("such wow"));

Fields§

§inner: StructArray

Reference to the underlying StructArray

§metadata: BinaryViewArray

The metadata column of this variant

§shredding_state: ShreddingState

how is this variant array shredded?

Implementations§

Source§

impl VariantArray

Source

pub fn try_new(inner: &dyn Array) -> Result<Self, ArrowError>

Creates a new VariantArray from a [StructArray].

§Arguments
  • inner - The underlying [StructArray] that contains the variant data.
§Returns
  • A new instance of VariantArray.
§Errors:
  • If the StructArray does not contain the required fields
§Requirements of the StructArray
  1. A required field named metadata which is binary, large_binary, or binary_view

  2. An optional field named value that is binary, large_binary, or binary_view

  3. An optional field named typed_value which can be any primitive type or be a list, large_list, list_view or struct

NOTE: It is also permissible for the metadata field to be Dictionary-Encoded, preferably (but not required) with an index type of int8.

Currently, only [BinaryViewArray] are supported.

Source

pub(crate) fn from_parts( metadata: BinaryViewArray, value: Option<BinaryViewArray>, typed_value: Option<ArrayRef>, nulls: Option<NullBuffer>, ) -> Self

Source

pub fn inner(&self) -> &StructArray

Returns a reference to the underlying [StructArray].

Source

pub fn into_inner(self) -> StructArray

Returns the inner [StructArray], consuming self

Source

pub fn shredding_state(&self) -> &ShreddingState

Return the shredding state of this VariantArray

Source

pub fn value(&self, index: usize) -> Variant<'_, '_>

Return the [Variant] instance stored at the given row

Note: This method does not check for nulls and the value is arbitrary (but still well-defined) if is_null returns true for the index.

§Panics
  • if the index is out of bounds
  • if the array value is null

If this is a shredded variant but has no value at the shredded location, it will return [Variant::Null].

§Performance Note

This is certainly not the most efficient way to access values in a VariantArray, but it is useful for testing and debugging.

Note: Does not do deep validation of the [Variant], so it is up to the caller to ensure that the metadata and value were constructed correctly.

Source

pub fn metadata_field(&self) -> &BinaryViewArray

Return a reference to the metadata field of the [StructArray]

Source

pub fn value_field(&self) -> Option<&BinaryViewArray>

Return a reference to the value field of the StructArray

Source

pub fn typed_value_field(&self) -> Option<&ArrayRef>

Return a reference to the typed_value field of the StructArray, if present

Source

pub fn field(&self, name: impl Into<String>) -> Field

Return a field to represent this VariantArray in a Schema with a particular name

Source

pub fn data_type(&self) -> &DataType

Returns a new DataType representing this VariantArray’s inner type

Source

pub fn slice(&self, offset: usize, length: usize) -> Self

Source

pub fn len(&self) -> usize

Source

pub fn is_empty(&self) -> bool

Source

pub fn nulls(&self) -> Option<&NullBuffer>

Source

pub fn is_null(&self, index: usize) -> bool

Is the element at index null?

Source

pub fn is_valid(&self, index: usize) -> bool

Is the element at index valid (not null)?

Trait Implementations§

Source§

impl Clone for VariantArray

Source§

fn clone(&self) -> VariantArray

Returns a duplicate of the value. Read more
1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for VariantArray

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl From<VariantArray> for ArrayRef

Source§

fn from(variant_array: VariantArray) -> Self

Converts to this type from the input type.
Source§

impl From<VariantArray> for StructArray

Source§

fn from(variant_array: VariantArray) -> Self

Converts to this type from the input type.

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<T> Ungil for T
where T: Send,