pub struct MaskCursor {
mask: BooleanBuffer,
position: usize,
loaded_row_ranges: Option<Arc<LoadedRowRanges>>,
}Expand description
Cursor for iterating a mask-backed RowSelection
This is best for dense selections where there are many small skips or selections. For example, selecting every other row.
When page pruning produces sparse column data, loaded_row_ranges limits
each decoded chunk to rows whose pages are loaded for every projected leaf.
For example, two projected columns can have different page boundaries:
Row ranges: [0, 4) [4, 6) [6, 8) [8, 10) [10, 12)
Selection mask: 1000 00 00 00 01
Column A pages: loaded | missing [4, 8) | loaded [8, 12)
Column B pages: loaded [0, 6) | missing [6, 10) | loaded
LoadedRowRanges: [0, 4) [10, 12)The first chunk decodes [0, 4) with mask 1000. The next chunk skips to
row 11 and decodes [11, 12) with mask 1. The loaded ranges are decode
boundaries, not output batch boundaries: ParquetRecordBatchReader
accumulates both chunks and applies the combined mask 10001 once.
Fields§
§mask: BooleanBuffer§position: usizeCurrent absolute offset into the selection
loaded_row_ranges: Option<Arc<LoadedRowRanges>>Row ranges whose backing pages are loaded for every projected column.
Implementations§
Source§impl MaskCursor
impl MaskCursor
Sourcepub fn next_mask_chunk(&mut self, batch_size: usize) -> Option<MaskChunk>
pub fn next_mask_chunk(&mut self, batch_size: usize) -> Option<MaskChunk>
Advance through the mask representation, producing the next chunk summary
Sourcefn next_mask_chunk_non_empty(&mut self, batch_size: usize) -> MaskChunk
fn next_mask_chunk_non_empty(&mut self, batch_size: usize) -> MaskChunk
Produces the next chunk for a non-empty, trailing-skip-free mask.
Sourcepub(crate) fn next_chunk(
&mut self,
batch_size: usize,
) -> Result<MaskChunk, ParquetError>
pub(crate) fn next_chunk( &mut self, batch_size: usize, ) -> Result<MaskChunk, ParquetError>
Returns the next non-empty mask chunk without crossing an unloaded row range.
The ReadPlan removes trailing
skips before constructing this cursor. Callers therefore only invoke
this method for a non-empty mask that has another selected row.
Sourcepub fn mask_values_for(
&self,
chunk: &MaskChunk,
) -> Result<BooleanArray, ParquetError>
pub fn mask_values_for( &self, chunk: &MaskChunk, ) -> Result<BooleanArray, ParquetError>
Materialise the boolean values for a mask-backed chunk