Skip to main content

PageReader

parquet::column::page

Trait PageReader 

Source 
pub trait PageReader: Iterator<Item = Result<Page>> + Send {
    // Required methods
    fn get_next_page(&mut self) -> Result<Option<Page>>;
    fn peek_next_page(&mut self) -> Result<Option<PageMetadata>>;
    fn skip_next_page(&mut self) -> Result<()>;

    // Provided methods
    fn supports_deferred_dictionary(&self) -> bool { ... }
    fn take_deferred_dictionary(&mut self) -> Result<Option<Page>> { ... }
    fn at_record_boundary(&mut self) -> Result<bool> { ... }
}
Expand description

API for reading pages from a column chunk. This offers a iterator like API to get the next page.

Required Methods§

Source

fn get_next_page(&mut self) -> Result<Option<Page>>

Gets the next page in the column chunk associated with this reader. Returns None if there are no pages left.

Source

fn peek_next_page(&mut self) -> Result<Option<PageMetadata>>

Gets metadata about the next page, returns an error if no column index information

Source

fn skip_next_page(&mut self) -> Result<()>

Skips reading the next page, returns an error if no column index information

Provided Methods§

Source

fn supports_deferred_dictionary(&self) -> bool

Returns whether skipped dictionary pages can later be recovered with Self::take_deferred_dictionary.

Implementations returning true must retain dictionary pages consumed by Self::skip_next_page.

Source

fn take_deferred_dictionary(&mut self) -> Result<Option<Page>>

Returns the decoded dictionary Page this reader has previously skipped past, if any, removing it from the reader.

Self::skip_next_page may skip a dictionary page without decoding it, since skipping rows never needs dictionary contents. If decoding later reaches a dictionary-encoded data page, the reader is asked for the dictionary through this method, which pays the deferred decompression exactly once. A chunk skipped end to end never pays it.

The default implementation returns Ok(None), meaning the reader never defers.

Source

fn at_record_boundary(&mut self) -> Result<bool>

Returns true if the next page can be assumed to contain the start of a new record

Prior to parquet V2 the specification was ambiguous as to whether a single record could be split across multiple pages, and prior to (#4327) the Rust writer would do this in certain situations. However, correctly interpreting the offset index relies on this assumption holding (#4943), and so this mechanism is provided for a PageReader to signal this to the calling context

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§