Struct RowSelection

pub struct RowSelection {
    selectors: Vec<RowSelector>,
}

RowSelection allows selecting or skipping a provided number of rows when scanning the parquet file.

This is applied prior to reading column data, and can therefore be used to skip IO to fetch data into memory

A typical use-case would be using the PageIndex to filter out rows that don’t satisfy a predicate

Example

use parquet::arrow::arrow_reader::{RowSelection, RowSelector};

let selectors = vec![
    RowSelector::skip(5),
    RowSelector::select(5),
    RowSelector::select(5),
    RowSelector::skip(5),
];

// Creating a selection will combine adjacent selectors
let selection: RowSelection = selectors.into();

let expected = vec![
    RowSelector::skip(5),
    RowSelector::select(10),
    RowSelector::skip(5),
];

let actual: Vec<RowSelector> = selection.into();
assert_eq!(actual, expected);

// you can also create a selection from consecutive ranges
let ranges = vec![5..10, 10..15];
let selection =
  RowSelection::from_consecutive_ranges(ranges.into_iter(), 20);
let actual: Vec<RowSelector> = selection.into();
assert_eq!(actual, expected);

A RowSelection maintains the following invariants:

• It contains no RowSelector of 0 rows
• Consecutive RowSelectors alternate skipping or selecting rows

Fields

selectors: Vec<RowSelector>

Implementations

impl RowSelection

pub fn from_filters(filters: &[BooleanArray]) -> Self

Creates a RowSelection from a slice of [BooleanArray]

Panic

Panics if any of the [BooleanArray] contain nulls

pub fn from_consecutive_ranges<I: Iterator<Item = Range<usize>>>( ranges: I, total_rows: usize, ) -> Self

Creates a RowSelection from an iterator of consecutive ranges to keep

pub fn scan_ranges(&self, page_locations: &[PageLocation]) -> Vec<Range<usize>>

Given an offset index, return the byte ranges for all data pages selected by self

This is useful for determining what byte ranges to fetch from underlying storage

Note: this method does not make any effort to combine consecutive ranges, nor coalesce ranges that are close together. This is instead delegated to the IO subsystem to optimise, e.g. ObjectStore::get_ranges

pub fn split_off(&mut self, row_count: usize) -> Self

Splits off the first row_count from this RowSelection

pub fn and_then(&self, other: &Self) -> Self

returns a RowSelection representing rows that are selected in both input RowSelections.

This is equivalent to the logical AND / conjunction of the two selections.

Example

If N means the row is not selected, and Y means it is selected:

self:     NNNNNNNNNNNNYYYYYYYYYYYYYYYYYYYYYYNNNYYYYY
other:                YYYYYNNNNYYYYYYYYYYYYY   YYNNN

returned: NNNNNNNNNNNNYYYYYNNNNYYYYYYYYYYYYYNNNYYNNN

Panics

Panics if other does not have a length equal to the number of rows selected by this RowSelection

pub fn intersection(&self, other: &Self) -> Self

Compute the intersection of two RowSelection For example: self: NNYYYYNNYYNYN other: NYNNNNNNY

returned: NNNNNNNNYYNYN

pub fn union(&self, other: &Self) -> Self

Compute the union of two RowSelection For example: self: NNYYYYNNYYNYN other: NYNNNNNNN

returned: NYYYYYNNYYNYN

pub fn selects_any(&self) -> bool

Returns true if this RowSelection selects any rows

pub(crate) fn trim(self) -> Self

Trims this RowSelection removing any trailing skips

pub(crate) fn offset(self, offset: usize) -> Self

Applies an offset to this RowSelection, skipping the first offset selected rows

pub(crate) fn limit(self, limit: usize) -> Self

Limit this RowSelection to only select limit rows

pub fn iter(&self) -> impl Iterator<Item = &RowSelector>

Returns an iterator over the RowSelectors for this RowSelection.

pub fn row_count(&self) -> usize

Returns the number of selected rows

pub fn skipped_row_count(&self) -> usize

Returns the number of de-selected rows

Trait Implementations

impl Clone for RowSelection

fn clone(&self) -> RowSelection

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RowSelection

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for RowSelection

fn default() -> RowSelection

Returns the “default value” for a type.

impl From<RowSelection> for Vec<RowSelector>

fn from(r: RowSelection) -> Self

Converts to this type from the input type.

impl From<RowSelection> for VecDeque<RowSelector>

fn from(r: RowSelection) -> Self

Converts to this type from the input type.

impl From<Vec<RowSelector>> for RowSelection

fn from(selectors: Vec<RowSelector>) -> Self

Converts to this type from the input type.

impl FromIterator<RowSelector> for RowSelection

fn from_iter<T: IntoIterator<Item = RowSelector>>(iter: T) -> Self

Creates a value from an iterator.

impl PartialEq for RowSelection

fn eq(&self, other: &RowSelection) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for RowSelection

impl StructuralPartialEq for RowSelection