Struct ColumnChunkMask 

pub struct ColumnChunkMask {
    row_groups: Option<Arc<[u32]>>,
    columns: Option<Arc<[u32]>>,
}

A cheaply cloneable struct to specify column chunks for which metadata is required.

Column chunks are identified by row group index and leaf column index (the index of the column in SchemaDescriptor::columns, not the index of a root or Arrow field). This struct allows for specifying vertical slices of column chunk data (via Self::columns), horizontal slices (via Self::row_groups), or the intersection of the two (via Self::row_groups_and_columns).

Because Parquet collection cannot contain more than i32::MAX values, user provided indices outside that range will be discarded.

At present this is only used to select elements of the Page Index for decoding.

Examples

To select columns 0 and 1 from all row groups:

let mask = ColumnChunkMask::columns([0, 1]);

To select all columns from row group 2:

let mask = ColumnChunkMask::row_groups([2]);

To select columns 1 and 3 from row group 0:

let mask = ColumnChunkMask::row_groups_and_columns([0], [1, 3]);

Fields

row_groups: Option<Arc<[u32]>>

columns: Option<Arc<[u32]>>

Implementations

impl ColumnChunkMask

pub fn all() -> Self

Select all row groups and columns.

pub fn none() -> Self

Select no row groups or columns.

pub fn columns(columns: impl IntoIterator<Item = usize>) -> Self

Select only the listed columns.

Passing an empty iterator selects no columns.

pub fn row_groups(row_groups: impl IntoIterator<Item = usize>) -> Self

Select only the listed row groups.

Passing an empty iterator selects no row groups.

pub fn row_groups_and_columns( row_groups: impl IntoIterator<Item = usize>, columns: impl IntoIterator<Item = usize>, ) -> Self

Select only the listed row groups and columns.

An empty iterator for either dimension selects no column chunks.

pub fn includes_row_group(&self, idx: usize, num_row_groups: usize) -> bool

Returns whether row group idx is selected and within the bounds [0, num_row_groups).

pub fn includes_column(&self, idx: usize, num_columns: usize) -> bool

Returns whether column idx is selected and within the bounds [0, num_columns).

fn includes_index(keep: Option<&Arc<[u32]>>, idx: usize) -> bool

pub fn is_all(&self) -> bool

Returns true when this mask selects every column chunk.

pub fn is_none(&self) -> bool

Returns true when either configured axis is empty (and thus would select no column chunks).

This does not consider the dimensions of a particular Parquet file, so may return false when Self::column_indices or Self::row_group_indices returns an empty iterator due to all elements exceeding the provided upper bound.

pub fn from_projection( projection: &ProjectionMask, schema: &SchemaDescriptor, ) -> Self

Creates a mask selecting the leaf columns in an Arrow projection.

pub fn row_group_indices( &self, num_row_groups: usize, ) -> impl Iterator<Item = usize> + '_

Returns an iterator over the row group indices selected by this mask. If the mask is all then all indices in the range [0, num_row_groups) are returned.

pub fn column_indices( &self, num_columns: usize, ) -> impl Iterator<Item = usize> + '_

Returns an iterator over the column indices selected by this mask. If the mask is all then all indices in the range [0, num_columns) are returned.

fn axis_indices(axis: Option<&[u32]>, len: usize) -> AxisIndices<'_>

fn iter_to_set(indices: impl IntoIterator<Item = usize>) -> Option<Arc<[u32]>>

Trait Implementations

impl Clone for ColumnChunkMask

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ColumnChunkMask

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ColumnChunkMask

fn default() -> Self

Returns the “default value” for a type.

impl Eq for ColumnChunkMask

impl Hash for ColumnChunkMask

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for ColumnChunkMask

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for ColumnChunkMask