Trait AsyncFileReader

pub trait AsyncFileReader: Send {
    // Required methods
    fn get_bytes(&mut self, range: Range<u64>) -> BoxFuture<'_, Result<Bytes>>;
    fn get_metadata<'a>(
        &'a mut self,
        options: Option<&'a ArrowReaderOptions>,
    ) -> BoxFuture<'a, Result<Arc<ParquetMetaData>>>;

    // Provided method
    fn get_byte_ranges(
        &mut self,
        ranges: Vec<Range<u64>>,
    ) -> BoxFuture<'_, Result<Vec<Bytes>>> { ... }
}

The asynchronous interface used by ParquetRecordBatchStream to read parquet files

Notes:

1. There is a default implementation for types that implement [AsyncRead] and [AsyncSeek], for example tokio::fs::File.

2. ParquetObjectReader, available when the object_store crate feature is enabled, implements this interface for ObjectStore.

Required Methods

fn get_bytes(&mut self, range: Range<u64>) -> BoxFuture<'_, Result<Bytes>>

Retrieve the bytes in range

fn get_metadata<'a>( &'a mut self, options: Option<&'a ArrowReaderOptions>, ) -> BoxFuture<'a, Result<Arc<ParquetMetaData>>>

Return a future which results in the ParquetMetaData for this Parquet file.

This is an asynchronous operation as it may involve reading the file footer and potentially other metadata from disk or a remote source.

Reading data from Parquet requires the metadata to understand the schema, row groups, and location of pages within the file. This metadata is stored primarily in the footer of the Parquet file, and can be read using ParquetMetaDataReader.

However, implementations can significantly speed up reading Parquet by supplying cached metadata or pre-fetched metadata via this API.

Parameters

• options: Optional ArrowReaderOptions that may contain decryption and other options that affect how the metadata is read.

Provided Methods

fn get_byte_ranges( &mut self, ranges: Vec<Range<u64>>, ) -> BoxFuture<'_, Result<Vec<Bytes>>>

Retrieve multiple byte ranges. The default implementation will call get_bytes sequentially

Trait Implementations

impl AsyncFileReader for Box<dyn AsyncFileReader + '_>

This allows Box<dyn AsyncFileReader + ’_> to be used as an AsyncFileReader,

fn get_bytes(&mut self, range: Range<u64>) -> BoxFuture<'_, Result<Bytes>>

Retrieve the bytes in range

fn get_byte_ranges( &mut self, ranges: Vec<Range<u64>>, ) -> BoxFuture<'_, Result<Vec<Bytes>>>

Retrieve multiple byte ranges. The default implementation will call get_bytes sequentially

fn get_metadata<'a>( &'a mut self, options: Option<&'a ArrowReaderOptions>, ) -> BoxFuture<'a, Result<Arc<ParquetMetaData>>>

Return a future which results in the ParquetMetaData for this Parquet file.

Implementations on Foreign Types

impl AsyncFileReader for Box<dyn AsyncFileReader + '_>

This allows Box<dyn AsyncFileReader + ’_> to be used as an AsyncFileReader,

fn get_bytes(&mut self, range: Range<u64>) -> BoxFuture<'_, Result<Bytes>>

fn get_byte_ranges( &mut self, ranges: Vec<Range<u64>>, ) -> BoxFuture<'_, Result<Vec<Bytes>>>

fn get_metadata<'a>( &'a mut self, options: Option<&'a ArrowReaderOptions>, ) -> BoxFuture<'a, Result<Arc<ParquetMetaData>>>

Implementors

impl AsyncFileReader for ParquetObjectReader

impl<T: AsyncRead + AsyncSeek + Unpin + Send> AsyncFileReader for T