Struct InMemoryRowGroup 

pub(crate) struct InMemoryRowGroup<'a> {
    pub(crate) offset_index: Option<&'a [OffsetIndexMetaData]>,
    pub(crate) column_chunks: Vec<Option<Arc<ColumnChunkData>>>,
    pub(crate) row_count: usize,
    pub(crate) row_group_idx: usize,
    pub(crate) metadata: &'a ParquetMetaData,
}

An in-memory collection of column chunks

Fields

offset_index: Option<&'a [OffsetIndexMetaData]>

column_chunks: Vec<Option<Arc<ColumnChunkData>>>

Column chunks for this row group

row_count: usize

row_group_idx: usize

metadata: &'a ParquetMetaData

Implementations

impl InMemoryRowGroup<'_>

pub(crate) async fn fetch<T: AsyncFileReader + Send>( &mut self, input: &mut T, projection: &ProjectionMask, selection: Option<&RowSelection>, batch_size: usize, cache_mask: Option<&ProjectionMask>, ) -> Result<()>

Fetches any additional column data specified in projection that is not already present in self.column_chunks.

If selection is provided, only the pages required for the selection are fetched. Otherwise, all pages are fetched.

impl InMemoryRowGroup<'_>

pub(crate) fn fetch_ranges( &self, projection: &ProjectionMask, selection: Option<&RowSelection>, batch_size: usize, cache_mask: Option<&ProjectionMask>, ) -> FetchRanges

Returns the byte ranges to fetch for the columns specified in projection and selection.

cache_mask indicates which columns, if any, are being cached by RowGroupCache. The selection for Cached columns is expanded to batch boundaries to simplify accounting for what data is cached.

pub(crate) fn fill_column_chunks<I>( &mut self, projection: &ProjectionMask, page_start_offsets: Option<Vec<Vec<u64>>>, chunk_data: I, )

where I: IntoIterator<Item = Bytes>,

Fills in self.column_chunks with the data fetched from chunk_data.

This function must be called with the data from the ranges returned by fetch_ranges and the corresponding page_start_offsets, with the exact same and selection.

Trait Implementations

impl<'a> Debug for InMemoryRowGroup<'a>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl RowGroups for InMemoryRowGroup<'_>

fn column_chunks(&self, i: usize) -> Result<Box<dyn PageIterator>>

Return chunks for column i

fn num_rows(&self) -> usize

Get the number of rows in this collection