pub struct VariantArray {
inner: StructArray,
metadata: BinaryViewArray,
shredding_state: ShreddingState,
}
Expand description
An array of Parquet [Variant
] values
A VariantArray
wraps an Arrow [StructArray
] that stores the underlying
metadata
and value
fields, and adds convenience methods to access
the [Variant
]s.
See VariantArrayBuilder
for constructing VariantArray
row by row.
See the examples below from converting between VariantArray
and
StructArray
.
§Documentation
At the time of this writing, Variant has been accepted as an official extension type but not been published to the official list of extension types on the Apache Arrow website. See the Extension Type for Parquet Variant arrow ticket for more details.
§Example: Check if a [StructArray
] has the VariantType
extension
Arrow Arrays only provide [DataType
], but the extension type information
is stored on a [Field
]. Thus, you must have access to the Schema
or
[Field
] to check for the extension type.
let schema = get_schema();
assert_eq!(schema.fields().len(), 2);
// first field is not a Variant
assert!(schema.field(0).try_extension_type::<VariantType>().is_err());
// second field is a Variant
assert!(schema.field(1).try_extension_type::<VariantType>().is_ok());
§Example: Constructing the correct [Field
] for a VariantArray
You can construct the correct [Field
] for a VariantArray
using the
VariantArray::field
method.
let variant_array = get_variant_array();
// First field is an integer id, second field is a variant
let schema = Schema::new(vec![
Field::new("id", DataType::Int32, false),
// call VariantArray::field to get the correct Field
variant_array.field("var"),
]);
You can also construct the [Field
] using VariantType
directly
// The DataType of a VariantArray varies depending on how it is shredded
let data_type = variant_array.data_type().clone();
// First field is an integer id, second field is a variant
let schema = Schema::new(vec![
Field::new("id", DataType::Int32, false),
Field::new("var", data_type, false)
// Add extension metadata to the field using `VariantType`
.with_extension_type(VariantType),
]);
§Example: Converting a VariantArray
to a [StructArray
]
// Create Variant Array
let mut builder = VariantArrayBuilder::new(10);
builder.append_variant(Variant::from("such wow"));
let variant_array = builder.build();
// convert to StructArray
let struct_array: StructArray = variant_array.into();
§Example: Converting a [StructArray
] to a VariantArray
let struct_array: StructArray = get_struct_array();
// try and create a VariantArray from it
let variant_array = VariantArray::try_new(&struct_array).unwrap();
assert_eq!(variant_array.value(0), Variant::from("such wow"));
Fields§
§inner: StructArray
Reference to the underlying StructArray
metadata: BinaryViewArray
The metadata column of this variant
shredding_state: ShreddingState
how is this variant array shredded?
Implementations§
Source§impl VariantArray
impl VariantArray
Sourcepub fn try_new(inner: &dyn Array) -> Result<Self, ArrowError>
pub fn try_new(inner: &dyn Array) -> Result<Self, ArrowError>
Creates a new VariantArray
from a [StructArray
].
§Arguments
inner
- The underlying [StructArray
] that contains the variant data.
§Returns
- A new instance of
VariantArray
.
§Errors:
- If the
StructArray
does not contain the required fields
§Requirements of the StructArray
-
A required field named
metadata
which is binary, large_binary, or binary_view -
An optional field named
value
that is binary, large_binary, or binary_view -
An optional field named
typed_value
which can be any primitive type or be a list, large_list, list_view or struct
NOTE: It is also permissible for the metadata field to be Dictionary-Encoded, preferably (but not required) with an index type of int8.
Currently, only [BinaryViewArray
] are supported.
pub(crate) fn from_parts( metadata: BinaryViewArray, value: Option<BinaryViewArray>, typed_value: Option<ArrayRef>, nulls: Option<NullBuffer>, ) -> Self
Sourcepub fn into_inner(self) -> StructArray
pub fn into_inner(self) -> StructArray
Returns the inner [StructArray
], consuming self
Sourcepub fn shredding_state(&self) -> &ShreddingState
pub fn shredding_state(&self) -> &ShreddingState
Return the shredding state of this VariantArray
Sourcepub fn value(&self, index: usize) -> Variant<'_, '_>
pub fn value(&self, index: usize) -> Variant<'_, '_>
Return the [Variant
] instance stored at the given row
Note: This method does not check for nulls and the value is arbitrary
(but still well-defined) if is_null
returns true for the index.
§Panics
- if the index is out of bounds
- if the array value is null
If this is a shredded variant but has no value at the shredded location, it
will return [Variant::Null
].
§Performance Note
This is certainly not the most efficient way to access values in a
VariantArray
, but it is useful for testing and debugging.
Note: Does not do deep validation of the [Variant
], so it is up to the
caller to ensure that the metadata and value were constructed correctly.
Sourcepub fn metadata_field(&self) -> &BinaryViewArray
pub fn metadata_field(&self) -> &BinaryViewArray
Return a reference to the metadata field of the [StructArray
]
Sourcepub fn value_field(&self) -> Option<&BinaryViewArray>
pub fn value_field(&self) -> Option<&BinaryViewArray>
Return a reference to the value field of the StructArray
Sourcepub fn typed_value_field(&self) -> Option<&ArrayRef>
pub fn typed_value_field(&self) -> Option<&ArrayRef>
Return a reference to the typed_value field of the StructArray
, if present
Sourcepub fn field(&self, name: impl Into<String>) -> Field
pub fn field(&self, name: impl Into<String>) -> Field
Return a field to represent this VariantArray in a Schema
with
a particular name
Sourcepub fn data_type(&self) -> &DataType
pub fn data_type(&self) -> &DataType
Returns a new DataType representing this VariantArray’s inner type
pub fn slice(&self, offset: usize, length: usize) -> Self
pub fn len(&self) -> usize
pub fn is_empty(&self) -> bool
pub fn nulls(&self) -> Option<&NullBuffer>
Trait Implementations§
Source§impl Clone for VariantArray
impl Clone for VariantArray
Source§fn clone(&self) -> VariantArray
fn clone(&self) -> VariantArray
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read more