Struct DictionaryTracker

pub struct DictionaryTracker {
    written: HashMap<i64, ArrayData>,
    dict_ids: Vec<i64>,
    error_on_replacement: bool,
    preserve_dict_id: bool,
}

Keeps track of dictionaries that have been written, to avoid emitting the same dictionary multiple times.

Can optionally error if an update to an existing dictionary is attempted, which isn’t allowed in the FileWriter.

Fields

written: HashMap<i64, ArrayData>

dict_ids: Vec<i64>

error_on_replacement: bool

preserve_dict_id: bool

Implementations

impl DictionaryTracker

pub fn new(error_on_replacement: bool) -> Self

Create a new DictionaryTracker.

If error_on_replacement is true, an error will be generated if an update to an existing dictionary is attempted.

If preserve_dict_id is true, the dictionary ID defined in the schema is used, otherwise a unique dictionary ID will be assigned by incrementing the last seen dictionary ID (or using 0 if no other dictionary IDs have been seen)

pub fn new_with_preserve_dict_id( error_on_replacement: bool, preserve_dict_id: bool, ) -> Self

Create a new DictionaryTracker.

If error_on_replacement is true, an error will be generated if an update to an existing dictionary is attempted.

pub fn set_dict_id(&mut self, field: &Field) -> i64

Set the dictionary ID for field.

If preserve_dict_id is true, this will return the dict_id in field (or panic if field does not have a dict_id defined).

If preserve_dict_id is false, this will return the value of the last dict_id assigned incremented by 1 or 0 in the case where no dictionary IDs have yet been assigned

pub fn dict_id(&mut self) -> &[i64]

Return the sequence of dictionary IDs in the order they should be observed while traversing the schema

pub fn insert( &mut self, dict_id: i64, column: &ArrayRef, ) -> Result<bool, ArrowError>

Keep track of the dictionary with the given ID and values. Behavior:

• If this ID has been written already and has the same data, return Ok(false) to indicate that the dictionary was not actually inserted (because it’s already been seen).
• If this ID has been written already but with different data, and this tracker is configured to return an error, return an error.
• If the tracker has not been configured to error on replacement or this dictionary has never been seen before, return Ok(true) to indicate that the dictionary was just inserted.

Trait Implementations

impl Debug for DictionaryTracker

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.