Trait MetadataFetch

pub trait MetadataFetch {
    // Required method
    fn fetch(&mut self, range: Range<u64>) -> BoxFuture<'_, Result<Bytes>>;
}

A data source that can be used with MetadataLoader to load ParquetMetaData

Note that implementation is provided for AsyncFileReader.

Example MetadataFetch for a custom async data source

// Adapter that implements the API for reading bytes from an async source (in
// this case a tokio::fs::File)
struct TokioFileMetadata {
    file: tokio::fs::File,
}
impl MetadataFetch for TokioFileMetadata {
    fn fetch(&mut self, range: Range<u64>) -> BoxFuture<'_, Result<Bytes>> {
        // return a future that fetches data in range
        async move {
            let len = (range.end - range.start).try_into().unwrap();
            let mut buf = vec![0; len]; // target buffer
            // seek to the start of the range and read the data
            self.file.seek(SeekFrom::Start(range.start)).await?;
            self.file.read_exact(&mut buf).await?;
            Ok(Bytes::from(buf)) // convert to Bytes
        }
            .boxed() // turn into BoxedFuture, using FutureExt::boxed
    }
}

Required Methods

fn fetch(&mut self, range: Range<u64>) -> BoxFuture<'_, Result<Bytes>>

Return a future that fetches the specified range of bytes asynchronously

Note the returned type is a boxed future, often created by [FutureExt::boxed]. See the trait documentation for an example

Implementations on Foreign Types

impl<T: AsyncFileReader> MetadataFetch for &mut T

fn fetch(&mut self, range: Range<u64>) -> BoxFuture<'_, Result<Bytes>>

Implementors

impl<F, Fut> MetadataFetch for MetadataFetchFn<F>

where F: FnMut(Range<usize>) -> Fut + Send, Fut: Future<Output = Result<Bytes>> + Send,