Skip to main content

decode_dictionary_page

Function decode_dictionary_page 

Source
pub(crate) fn decode_dictionary_page(
    buffer: Bytes,
    parquet_meta_data: &ParquetMetaData,
    row_group_idx: usize,
    column_idx: usize,
) -> Result<ArrayRef>
Expand description

Decodes the dictionary page of a column chunk into an [ArrayRef].

buffer must contain the entire dictionary page, byte-for-byte, i.e. the range [dictionary_page_offset, data_page_offset) of the column chunk.

Only BYTE_ARRAY columns are currently supported; other physical types return an error. The returned array never contains nulls: dictionary pages only store the distinct non-null values, with nulls represented via definition levels in the data pages. The returned array has Binary values, regardless of the column’s logical type.

Note this only decodes whatever dictionary page is present – it does not verify that the entire column chunk is dictionary-encoded (i.e. that every value in the chunk is drawn from this dictionary). Callers that need that guarantee (for example, to use the dictionary as an exact membership index) must check that themselves, e.g. via crate::file::metadata::ColumnChunkMetaData::page_encoding_stats_mask.