Struct VariantMetadata

pub struct VariantMetadata<'m> {
    pub(crate) bytes: &'m [u8],
    header: VariantMetadataHeader,
    dictionary_size: u32,
    first_value_byte: u32,
    validated: bool,
}

Variant Metadata

See the Variant Spec file for more information

Validation

Every instance of variant metadata is either valid or invalid. depending on whether the underlying bytes are a valid encoding of variant metadata (see below).

Instances produced by Self::try_new or Self::with_full_validation are fully validated. They always contain valid data, and infallible accesses such as iteration and indexing are panic-free. The validation cost is linear in the number of underlying bytes.

Instances produced by Self::new are unvalidated and so they may contain either valid or invalid data. Infallible accesses such as iteration and indexing will panic if the underlying bytes are invalid, and fallible alternatives such as Self::iter_try and Self::get are provided as panic-free alternatives. Self::with_full_validation can also be used to validate an unvalidated instance, if desired.

Unvalidated instances can be constructed in constant time. This can be useful if the caller knows the underlying bytes were already validated previously, or if the caller intends to perform a small number of (fallible) accesses to a large dictionary.

A validated variant [metadata instance guarantees that:

• header byte is valid
• dictionary size is in bounds
• offset array content is in-bounds
• first offset is zero
• last offset is in-bounds
• all other offsets are in-bounds (*)
• all offsets are monotonically increasing (*)
• all values are valid utf-8 (*)

NOTE: Self::new only skips expensive (non-constant cost) validation checks (marked by (*) in the list above); it panics any of the other checks fails.

Safety

Even an invalid variant metadata instance is still safe to use in the Rust sense. Accessing it with infallible methods may cause panics but will never lead to undefined behavior.

Fields

bytes: &'m [u8]

header: VariantMetadataHeader

dictionary_size: u32

first_value_byte: u32

validated: bool

Implementations

impl<'m> VariantMetadata<'m>

pub fn try_new(bytes: &'m [u8]) -> Result<Self, ArrowError>

Attempts to interpret bytes as a variant metadata instance, with full validation of all dictionary entries.

pub fn new(bytes: &'m [u8]) -> Self

Interprets bytes as a variant metadata instance, without attempting to validate dictionary entries. Panics if basic sanity checking fails, and subsequent infallible accesses such as indexing and iteration could also panic if the underlying bytes are invalid.

This constructor can be a useful lightweight alternative to Self::try_new if the bytes were already validated previously by other means, or if the caller expects a small number of accesses to a large dictionary (preferring to use a small number of fallible accesses as needed, instead of paying expensive full validation up front).

pub(crate) fn try_new_with_shallow_validation( bytes: &'m [u8], ) -> Result<Self, ArrowError>

pub fn len(&self) -> usize

The number of metadata dictionary entries

pub fn is_empty(&self) -> bool

True if this metadata dictionary contains no entries

pub fn is_fully_validated(&self) -> bool

True if this instance is fully validated for panic-free infallible accesses.

pub fn with_full_validation(self) -> Result<Self, ArrowError>

Performs a full validation of this metadata dictionary and returns the result.

pub fn is_sorted(&self) -> bool

Whether the dictionary keys are sorted and unique

pub const fn dictionary_size(&self) -> usize

Get the dictionary size

pub const fn version(&self) -> u8

The variant protocol version

fn get_offset(&self, i: usize) -> Result<u32, ArrowError>

Gets an offset array entry by index.

This offset is an index into the dictionary, at the boundary between string i-1 and string i. See Self::get to retrieve a specific dictionary entry.

pub fn get(&self, i: usize) -> Result<&'m str, ArrowError>

Attempts to retrieve a dictionary entry by index, failing if out of bounds or if the underlying bytes are invalid.

pub fn iter_try(&self) -> impl Iterator<Item = Result<&'m str, ArrowError>> + '_

Returns an iterator that attempts to visit all dictionary entries, producing Err if the iterator encounters invalid data.

pub fn iter(&self) -> impl Iterator<Item = &'m str> + '_

Iterates over all dictionary entries. When working with unvalidated input, consider Self::iter_try to avoid panics due to invalid data.

Trait Implementations

impl<'m> Clone for VariantMetadata<'m>

fn clone(&self) -> VariantMetadata<'m>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'m> Debug for VariantMetadata<'m>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Index<usize> for VariantMetadata<'_>

Retrieves the ith dictionary entry, panicking if the index is out of bounds. Accessing unvalidated input could also panic if the underlying bytes are invalid.

type Output = str

The returned type after indexing.

fn index(&self, i: usize) -> &str

Performs the indexing (container[index]) operation.

impl<'m> PartialEq for VariantMetadata<'m>

fn eq(&self, other: &VariantMetadata<'m>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<'m> StructuralPartialEq for VariantMetadata<'m>