fn prepare_selection_for_page_skipping(
plan_builder: ReadPlanBuilder,
projection_mask: &ProjectionMask,
offset_index: Option<&[OffsetIndexMetaData]>,
total_rows: usize,
) -> ReadPlanBuilderExpand description
Prepare row selection execution when page pruning produced sparse column data.
Some pages can be skipped during row-group construction if they are not read
by the selections. This means that the data pages for those rows are never
loaded and definition/repetition levels are never read. When using
RowSelections selection works because skip_records() handles this
case and skips the page accordingly.
However, with the current mask design, all values covered by a mask chunk are decoded before the mask filter is applied. Thus a chunk cannot cross a page that was skipped during row-group construction.
A simple example:
- the page size is 2, the mask is 100001, row selection should be read(1) skip(4) read(1)
- the
ColumnChunkDatawould be page1(10), page2(skipped), page3(01)
Mask execution records the row ranges loaded for every projected column, so
each mask chunk stays within loaded data and skip_records() crosses the
gaps. This applies both to an explicit mask policy and to Auto when it
resolves to mask execution.