Skip to main content

merge_dictionary_values_with_masks

Function merge_dictionary_values_with_masks 

Source
pub(crate) fn merge_dictionary_values_with_masks<K: ArrowDictionaryKeyType>(
    values_arrays: &[&dyn Array],
    values_masks: &[BooleanBuffer],
) -> Result<MergedDictionaries<K>, ArrowError>
Expand description

Merges referenced entries from multiple dictionary values arrays.

values_masks[i] corresponds to values_arrays[i], with each set bit identifying a value to retain. Returns the merged values and a mapping from each source value index to its index in the merged array. Mapping entries for unreferenced values are zero and must not be used.

Referenced values are gathered without hashing when their count fits in K’s key space, preserving duplicates across sources. Otherwise, values are deduplicated on a best-effort basis to avoid key overflow.