Enum ParquetDecoderState 

enum ParquetDecoderState {
    ReadingRowGroup {
        remaining_row_groups: Box<RemainingRowGroups>,
    },
    DecodingRowGroup {
        record_batch_reader: Box<ParquetRecordBatchReader>,
        remaining_row_groups: Box<RemainingRowGroups>,
    },
    Finished,
}

Internal state machine for the ParquetPushDecoder

Variants

ReadingRowGroup

Waiting for data needed to decode the next RowGroup

Fields

remaining_row_groups: Box<RemainingRowGroups>

DecodingRowGroup

The decoder is actively decoding a RowGroup

Fields

record_batch_reader: Box<ParquetRecordBatchReader>

Current active reader

remaining_row_groups: Box<RemainingRowGroups>

Finished

The decoder has finished processing all data

Implementations

impl ParquetDecoderState

fn try_transition( self, ) -> Result<(Self, DecodeResult<RecordBatch>), ParquetError>

Current state –> next state + output

This function is called to check if the decoder has any RecordBatches and Self::push_data is called when new data is available.

Notes

This structure is used to reduce the indentation level of the main loop in try_build

pub fn push_data( self, ranges: Vec<Range<u64>>, data: Vec<Bytes>, ) -> Result<Self, ParquetError>

Push data, and transition state if needed

This should correspond to the data ranges requested by the decoder

fn buffered_bytes(&self) -> u64

How many bytes are currently buffered in the decoder?

Trait Implementations

impl Debug for ParquetDecoderState

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.