Struct DataRequest 

pub(super) struct DataRequest {
    column_chunks: Vec<Option<Arc<ColumnChunkData>>>,
    ranges: Vec<Range<u64>>,
    page_start_offsets: Option<Vec<Vec<u64>>>,
}

Contains in-progress state to construct InMemoryRowGroups

See DataRequestBuilder for creating new requests

Fields

column_chunks: Vec<Option<Arc<ColumnChunkData>>>

Any previously read column chunk data

ranges: Vec<Range<u64>>

The ranges of data that are needed next

page_start_offsets: Option<Vec<Vec<u64>>>

Optional page start offsets for each requested range. This is used to create the relevant InMemoryRowGroup

Implementations

impl DataRequest

pub fn needed_ranges(&self, buffers: &PushBuffers) -> Vec<Range<u64>>

return what ranges are still needed to satisfy this request. Returns an empty vec if all ranges are satisfied

fn get_chunks(&self, buffers: &PushBuffers) -> Result<Vec<Bytes>, ParquetError>

Returns the chunks from the buffers that satisfy this request

pub fn try_into_in_memory_row_group<'a>( self, row_group_idx: usize, row_count: usize, parquet_metadata: &'a ParquetMetaData, projection: &ProjectionMask, buffers: &mut PushBuffers, ) -> Result<InMemoryRowGroup<'a>, ParquetError>

Create a new InMemoryRowGroup, and fill it with provided data

Assumes that all needed data is present in the buffers and clears any explicitly requested ranges

Trait Implementations

impl Debug for DataRequest

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.