Struct RemainingRowGroups 

pub(crate) struct RemainingRowGroups {
    schema: SchemaRef,
    frontier: RowGroupFrontier,
    row_group_reader_builder: RowGroupReaderBuilder,
}

State machine that tracks the remaining high level chunks (row groups) of Parquet data left to read.

RowGroupFrontier owns cross-row-group scan state and selects the next work item. RowGroupReaderBuilder owns decoding for the active row group.

Fields

schema: SchemaRef

The arrow schema of the decoded output. Carried only so Self::into_parts can hand it to a rebuilt builder; unused while decoding.

frontier: RowGroupFrontier

Cross-row-group scan state for queued work.

row_group_reader_builder: RowGroupReaderBuilder

State for building the reader for the current row group

Implementations

impl RemainingRowGroups

pub fn new( schema: SchemaRef, parquet_metadata: Arc<ParquetMetaData>, row_groups: Vec<usize>, selection: Option<RowSelection>, budget: RowBudget, has_predicates: bool, row_group_reader_builder: RowGroupReaderBuilder, ) -> Self

pub(crate) fn into_parts(self) -> RemainingRowGroupsParts

Decompose into RemainingRowGroupsParts.

Must be called at a row-group boundary (see Self::is_at_row_group_boundary). The inner reader builder’s runtime decode state is discarded; its buffered bytes are carried through.

pub fn push_data(&mut self, ranges: Vec<Range<u64>>, buffers: Vec<Bytes>)

Push new data buffers that can be used to satisfy pending requests

pub fn buffered_bytes(&self) -> u64

Return the total number of bytes buffered so far

pub fn clear_all_ranges(&mut self)

Clear any staged ranges currently buffered for future decode work

pub fn is_at_row_group_boundary(&self) -> bool

True iff the inner row-group reader is between row groups (state Finished). Forward to RowGroupReaderBuilder::is_finished.

pub fn row_groups_remaining(&self) -> usize

Number of row groups remaining (not including the one currently being decoded).

pub fn try_next_reader( &mut self, ) -> Result<DecodeResult<ParquetRecordBatchReader>, ParquetError>

returns ParquetRecordBatchReader suitable for reading the next group of rows from the Parquet data, or the list of data ranges still needed to proceed

Trait Implementations

impl Debug for RemainingRowGroups

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.