Enum DictionaryBuffer

pub enum DictionaryBuffer<K: ArrowNativeType, V: OffsetSizeTrait> {
    Dict {
        keys: Vec<K>,
        values: ArrayRef,
    },
    Values {
        values: OffsetBuffer<V>,
    },
}

An array of variable length byte arrays that are potentially dictionary encoded and can be converted into a corresponding [ArrayRef]

Variants

Dict

Fields

keys: Vec<K>

values: ArrayRef

Values

Fields

values: OffsetBuffer<V>

Implementations

impl<K: ArrowNativeType + Ord, V: OffsetSizeTrait> DictionaryBuffer<K, V>

pub fn len(&self) -> usize

pub fn as_keys(&mut self, dictionary: &ArrayRef) -> Option<&mut Vec<K>>

Returns a mutable reference to a keys array

Returns None if the dictionary needs to be recomputed

Panic

Panics if the dictionary is too large for K

pub fn spill_values(&mut self) -> Result<&mut OffsetBuffer<V>>

Returns a mutable reference to a values array

If this is currently dictionary encoded, this will convert from the dictionary encoded representation

pub fn into_array( self, null_buffer: Option<Buffer>, data_type: &ArrowType, ) -> Result<ArrayRef>

Converts this into an [ArrayRef] with the provided data_type and null_buffer

Trait Implementations

impl<K: ArrowNativeType, V: OffsetSizeTrait> Default for DictionaryBuffer<K, V>

fn default() -> Self

Returns the “default value” for a type.

impl<K: ArrowNativeType, V: OffsetSizeTrait> ValuesBuffer for DictionaryBuffer<K, V>

fn pad_nulls( &mut self, read_offset: usize, values_read: usize, levels_read: usize, valid_mask: &[u8], )

If a column contains nulls, more level data may be read than value data, as null values are not encoded. Therefore, first the levels data is read, the null count determined, and then the corresponding number of values read to a ValuesBuffer.