Module reader

Arrow IPC File and Stream Readers

Notes

The FileReader and StreamReader have similar interfaces, however the FileReader expects a reader that supports Seeking

Modules

stream 🔒

Structs

FileDecoder

A low-level, push-based interface for reading an IPC file

FileReader

Arrow File Reader

FileReaderBuilder

Build an Arrow FileReader with custom options.

RecordBatchDecoder 🔒

State for decoding Arrow arrays from an IPC RecordBatch structure to [RecordBatch]

StreamDecoder

A low-level interface for reading [RecordBatch] data from a stream of bytes

StreamReader

Arrow Stream Reader

Functions

parse_message 🔒

Parse an encapsulated message

read_block 🔒

Read the data for a given block

read_buffer 🔒

Read a buffer based on offset and length From https://github.com/apache/arrow/blob/6a936c4ff5007045e86f65f1a6b6c3c955ad5103/format/Message.fbs#L58 Each constituent buffer is first compressed with the indicated compressor, and then written with the uncompressed length in the first 8 bytes as a 64-bit little-endian signed integer followed by the compressed buffer bytes (and then padding as required by the protocol). The uncompressed length may be set to -1 to indicate that the data that follows is not compressed, which can be useful for cases where compression does not yield appreciable savings.

read_dictionary

Read the dictionary from the buffer and provided metadata, updating the dictionaries_by_id with the resulting dictionary

read_dictionary_impl 🔒

read_footer_length

Read the footer length from the last 10 bytes of an Arrow IPC file

read_record_batch

Creates a record batch from binary data using the crate::RecordBatch indexes and the Schema.