Trait AsyncFileReader 

pub trait AsyncFileReader: Send {
    // Required method
    fn get_bytes(
        &mut self,
        range: Range<u64>,
    ) -> BoxFuture<'_, Result<Bytes, AvroError>>;

    // Provided method
    fn get_byte_ranges(
        &mut self,
        ranges: Vec<Range<u64>>,
    ) -> BoxFuture<'_, Result<Vec<Bytes>, AvroError>> { ... }
}

The asynchronous interface used by super::AsyncAvroFileReader to read avro files

Notes:

1. There is a default implementation for types that implement [AsyncRead] and [AsyncSeek], for example tokio::fs::File.

2. Implementations for remote storage, such as the object_store crate, can implement this interface directly, typically by pairing a store handle with an object path and delegating Self::get_bytes and Self::get_byte_ranges to ranged reads. super::SpawnedReader can wrap such a reader to perform its I/O on a dedicated tokio runtime.

Example: implementing AsyncFileReader for the object_store crate

use arrow_avro::errors::AvroError;
use arrow_avro::reader::AsyncFileReader;
use bytes::Bytes;
use futures::FutureExt;
use futures::future::BoxFuture;
use object_store::path::Path;
use object_store::{ObjectStore, ObjectStoreExt};

#[derive(Clone, Debug)]
struct ObjectStoreReader {
    store: Arc<dyn ObjectStore>,
    path: Path,
}

impl AsyncFileReader for ObjectStoreReader {
    fn get_bytes(&mut self, range: Range<u64>) -> BoxFuture<'_, Result<Bytes, AvroError>> {
        async move {
            self.store
                .get_range(&self.path, range)
                .await
                .map_err(|e| AvroError::General(e.to_string()))
        }
        .boxed()
    }

    fn get_byte_ranges(
        &mut self,
        ranges: Vec<Range<u64>>,
    ) -> BoxFuture<'_, Result<Vec<Bytes>, AvroError>> {
        async move {
            self.store
                .get_ranges(&self.path, &ranges)
                .await
                .map_err(|e| AvroError::General(e.to_string()))
        }
        .boxed()
    }
}

Required Methods

fn get_bytes( &mut self, range: Range<u64>, ) -> BoxFuture<'_, Result<Bytes, AvroError>>

Retrieve the bytes in range

Provided Methods

fn get_byte_ranges( &mut self, ranges: Vec<Range<u64>>, ) -> BoxFuture<'_, Result<Vec<Bytes>, AvroError>>

Retrieve multiple byte ranges. The default implementation will call get_bytes sequentially

Trait Implementations

impl AsyncFileReader for Box<dyn AsyncFileReader + '_>

This allows Box<dyn AsyncFileReader + ’_> to be used as an AsyncFileReader,

fn get_bytes( &mut self, range: Range<u64>, ) -> BoxFuture<'_, Result<Bytes, AvroError>>

Retrieve the bytes in range

fn get_byte_ranges( &mut self, ranges: Vec<Range<u64>>, ) -> BoxFuture<'_, Result<Vec<Bytes>, AvroError>>

Retrieve multiple byte ranges. The default implementation will call get_bytes sequentially

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementations on Foreign Types

impl AsyncFileReader for Box<dyn AsyncFileReader + '_>

This allows Box<dyn AsyncFileReader + ’_> to be used as an AsyncFileReader,

fn get_bytes( &mut self, range: Range<u64>, ) -> BoxFuture<'_, Result<Bytes, AvroError>>

fn get_byte_ranges( &mut self, ranges: Vec<Range<u64>>, ) -> BoxFuture<'_, Result<Vec<Bytes>, AvroError>>

Implementors

impl AsyncFileReader for AvroObjectReader

impl<R> AsyncFileReader for SpawnedReader<R>

where R: AsyncFileReader + Clone + Send + 'static,

impl<T: AsyncRead + AsyncSeek + Unpin + Send> AsyncFileReader for T