Module reader

Arrow IPC File and Stream Readers

The FileReader and StreamReader have similar interfaces, however the FileReader expects a reader that supports Seeking

Modules

• stream 🔒

Structs

• ArrayReader 🔒
  State for decoding arrays from an encoded [RecordBatch]
• FileDecoder
  A low-level, push-based interface for reading an IPC file
• FileReader
  Arrow File reader
• FileReaderBuilder
  Build an Arrow FileReader with custom options.
• StreamDecoder
  A low-level interface for reading [RecordBatch] data from a stream of bytes
• StreamReader
  Arrow Stream reader

Functions

• create_array 🔒
  Coordinates reading arrays based on data types.
• create_dictionary_array 🔒
  Reads the correct number of buffers based on list type and null_count, and creates a list array ref
• create_list_array 🔒
  Reads the correct number of buffers based on list type and null_count, and creates a list array ref
• create_primitive_array 🔒
  Reads the correct number of buffers based on data type and null_count, and creates a primitive array ref
• parse_message 🔒
  Parse an encapsulated message
• read_block 🔒
  Read the data for a given block
• read_buffer 🔒
  Read a buffer based on offset and length From https://github.com/apache/arrow/blob/6a936c4ff5007045e86f65f1a6b6c3c955ad5103/format/Message.fbs#L58 Each constituent buffer is first compressed with the indicated compressor, and then written with the uncompressed length in the first 8 bytes as a 64-bit little-endian signed integer followed by the compressed buffer bytes (and then padding as required by the protocol). The uncompressed length may be set to -1 to indicate that the data that follows is not compressed, which can be useful for cases where compression does not yield appreciable savings.
• read_dictionary
  Read the dictionary from the buffer and provided metadata, updating the dictionaries_by_id with the resulting dictionary
• read_dictionary_impl 🔒
• read_footer_length
  Read the footer length from the last 10 bytes of an Arrow IPC file
• read_record_batch
  Creates a record batch from binary data using the crate::RecordBatch indexes and the Schema.
• read_record_batch_impl 🔒