Struct ArrowReaderMetadata

pub struct ArrowReaderMetadata {
    pub(crate) metadata: Arc<ParquetMetaData>,
    pub(crate) schema: SchemaRef,
    pub(crate) fields: Option<Arc<ParquetField>>,
}

The metadata necessary to construct a ArrowReaderBuilder

Note this structure is cheaply clone-able as it consists of several arcs.

This structure allows

1. Loading metadata for a file once and then using that same metadata to construct multiple separate readers, for example, to distribute readers across multiple threads

2. Using a cached copy of the ParquetMetadata rather than reading it from the file each time a reader is constructed.

Fields

metadata: Arc<ParquetMetaData>

The Parquet Metadata, if known aprior

schema: SchemaRef

The Arrow Schema

fields: Option<Arc<ParquetField>>

Implementations

impl ArrowReaderMetadata

pub fn load<T: ChunkReader>( reader: &T, options: ArrowReaderOptions, ) -> Result<Self>

Loads ArrowReaderMetadata from the provided ChunkReader, if necessary

See ParquetRecordBatchReaderBuilder::new_with_metadata for an example of how this can be used

Notes

If options has ArrowReaderOptions::with_page_index true, but Self::metadata is missing the page index, this function will attempt to load the page index by making an object store request.

pub fn try_new( metadata: Arc<ParquetMetaData>, options: ArrowReaderOptions, ) -> Result<Self>

Create a new ArrowReaderMetadata

Notes

This function does not attempt to load the PageIndex if not present in the metadata. See Self::load for more details.

fn with_supplied_schema( metadata: Arc<ParquetMetaData>, supplied_schema: SchemaRef, ) -> Result<Self>

pub fn metadata(&self) -> &Arc<ParquetMetaData>

Returns a reference to the ParquetMetaData for this parquet file

pub fn parquet_schema(&self) -> &SchemaDescriptor

Returns the parquet SchemaDescriptor for this parquet file

pub fn schema(&self) -> &SchemaRef

Returns the arrow [SchemaRef] for this parquet file

impl ArrowReaderMetadata

pub async fn load_async<T: AsyncFileReader>( input: &mut T, options: ArrowReaderOptions, ) -> Result<Self>

Returns a new ArrowReaderMetadata for this builder

See ParquetRecordBatchStreamBuilder::new_with_metadata for how this can be used

Notes

If options has ArrowReaderOptions::with_page_index true, but Self::metadata is missing the page index, this function will attempt to load the page index by making an object store request.

Trait Implementations

impl Clone for ArrowReaderMetadata

fn clone(&self) -> ArrowReaderMetadata

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ArrowReaderMetadata

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.