Struct GenericColumnReader

pub struct GenericColumnReader<R, D, V> {
    descr: ColumnDescPtr,
    page_reader: Box<dyn PageReader>,
    num_buffered_values: usize,
    num_decoded_values: usize,
    has_record_delimiter: bool,
    def_level_decoder: Option<D>,
    rep_level_decoder: Option<R>,
    values_decoder: V,
}

Reads data for a given column chunk, using the provided decoders:

• R: ColumnLevelDecoder used to decode repetition levels
• D: ColumnLevelDecoder used to decode definition levels
• V: ColumnValueDecoder used to decode value data

Fields

descr: ColumnDescPtr

page_reader: Box<dyn PageReader>

num_buffered_values: usize

The total number of values stored in the data page.

num_decoded_values: usize

The number of values from the current data page that has been decoded into memory so far.

has_record_delimiter: bool

True if the end of the current data page denotes the end of a record

def_level_decoder: Option<D>

The decoder for the definition levels if any

rep_level_decoder: Option<R>

The decoder for the repetition levels if any

values_decoder: V

The decoder for the values

Implementations

impl<V> GenericColumnReader<RepetitionLevelDecoderImpl, DefinitionLevelDecoderImpl, V>

where V: ColumnValueDecoder,

pub fn new(descr: ColumnDescPtr, page_reader: Box<dyn PageReader>) -> Self

Creates new column reader based on column descriptor and page reader.

impl<R, D, V> GenericColumnReader<R, D, V>

where R: RepetitionLevelDecoder, D: DefinitionLevelDecoder, V: ColumnValueDecoder,

pub(crate) fn new_with_decoders( descr: ColumnDescPtr, page_reader: Box<dyn PageReader>, values_decoder: V, def_level_decoder: Option<D>, rep_level_decoder: Option<R>, ) -> Self

pub fn read_records( &mut self, max_records: usize, def_levels: Option<&mut D::Buffer>, rep_levels: Option<&mut R::Buffer>, values: &mut V::Buffer, ) -> Result<(usize, usize, usize)>

Read up to max_records whole records, returning the number of complete records, non-null values and levels decoded. All levels for a given record will be read, i.e. the next repetition level, if any, will be 0

If the max definition level is 0, def_levels will be ignored and the number of records, non-null values and levels decoded will all be equal, otherwise def_levels will be populated with the number of levels read, with an error returned if it is None.

If the max repetition level is 0, rep_levels will be ignored and the number of records and levels decoded will both be equal, otherwise rep_levels will be populated with the number of levels read, with an error returned if it is None.

values will be contiguously populated with the non-null values. Note that if the column is not required, this may be less than either max_records or the number of levels read

pub fn skip_records(&mut self, num_records: usize) -> Result<usize>

Skips over num_records records, where records are delimited by repetition levels of 0

Returns

Returns the number of records skipped

fn read_dictionary_page(&mut self) -> Result<()>

Read the next page as a dictionary page. If the next page is not a dictionary page, this will return an error.

fn read_new_page(&mut self) -> Result<bool>

Reads a new page and set up the decoders for levels, values or dictionary. Returns false if there’s no page left.

pub(crate) fn has_next(&mut self) -> Result<bool>

Check whether there is more data to read from this column, If the current page is fully decoded, this will load the next page (if it exists) into the buffer