Struct ParquetMetaData 

pub struct ParquetMetaData {
    file_metadata: FileMetaData,
    row_groups: Vec<RowGroupMetaData>,
    page_index: Option<PageIndex>,
    file_decryptor: Option<Box<FileDecryptor>>,
}

Parsed metadata for a single Parquet file

This structure is stored in the footer of Parquet files, in the format defined by parquet.thrift.

Overview

The fields of this structure are:

• FileMetaData: Information about the overall file (such as the schema) (See Self::file_metadata)
• RowGroupMetaData: Information about each Row Group (see Self::row_groups)
• PageIndex: Optional “Page Index” structures (see Self::page_index)

This structure is read by the various readers in this crate or can be read directly from a file using the ParquetMetaDataReader struct.

See the ParquetMetaDataBuilder to create and modify this structure.

Fields

file_metadata: FileMetaData

File level metadata

row_groups: Vec<RowGroupMetaData>

Row group metadata

page_index: Option<PageIndex>

Page level index for each page in each column chunk

file_decryptor: Option<Box<FileDecryptor>>

Optional file decryptor

Implementations

impl ParquetMetaData

pub fn new( file_metadata: FileMetaData, row_groups: Vec<RowGroupMetaData>, ) -> Self

Creates Parquet metadata from file metadata and a list of row group metadata

pub(crate) fn with_file_decryptor( &mut self, file_decryptor: Option<FileDecryptor>, )

Adds [FileDecryptor] to this metadata instance to enable decryption of encrypted data.

pub fn into_builder(self) -> ParquetMetaDataBuilder

Convert this ParquetMetaData into a ParquetMetaDataBuilder

pub fn file_metadata(&self) -> &FileMetaData

Returns file metadata as reference.

pub(crate) fn file_decryptor(&self) -> Option<&FileDecryptor>

Returns file decryptor as reference.

pub fn num_row_groups(&self) -> usize

Returns number of row groups in this file.

pub fn row_group(&self, i: usize) -> &RowGroupMetaData

Returns row group metadata for ith position. Position should be less than number of row groups num_row_groups.

pub fn row_groups(&self) -> &[RowGroupMetaData]

Returns slice of row groups in this file.

pub fn row_group_num_rows(&self, row_group_idx: usize) -> Result<usize>

Returns the number of rows in row_group_idx.

Returns an error if the row group index is out of bounds or its row count cannot be represented as a usize.

pub fn page_index(&self) -> Option<&PageIndex>

Returns the page index for this file if loaded

Returns None if the parquet file lacks page indexes or ArrowReaderOptions::with_page_index was set to false.

pub fn memory_size(&self) -> usize

Estimate of the bytes allocated to store ParquetMetadata

Notes:

1. Includes size of self

2. Includes heap memory for sub fields such as FileMetaData and RowGroupMetaData.

3. Includes memory from shared pointers (e.g. SchemaDescPtr). This means memory_size will over estimate the memory size if such pointers are shared.

4. Does not include any allocator overheads

pub(crate) fn set_page_index(&mut self, index: Option<PageIndex>)

Override the page index

Trait Implementations

impl Clone for ParquetMetaData

fn clone(&self) -> ParquetMetaData

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ParquetMetaData

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<ParquetMetaData> for ParquetMetaDataBuilder

fn from(meta_data: ParquetMetaData) -> Self

Converts to this type from the input type.

impl PartialEq for ParquetMetaData

fn eq(&self, other: &ParquetMetaData) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for ParquetMetaData