arrow_ipc::reader

Function create_array

Source
fn create_array(
    reader: &mut ArrayReader<'_>,
    field: &Field,
    variadic_counts: &mut VecDeque<i64>,
    require_alignment: bool,
) -> Result<ArrayRef, ArrowError>
Expand description

Coordinates reading arrays based on data types.

variadic_counts encodes the number of buffers to read for variadic types (e.g., Utf8View, BinaryView) When encounter such types, we pop from the front of the queue to get the number of buffers to read.

Notes:

  • In the IPC format, null buffers are always set, but may be empty. We discard them if an array has 0 nulls
  • Numeric values inside list arrays are often stored as 64-bit values regardless of their data type size. We thus:
    • check if the bit width of non-64-bit numbers is 64, and
    • read the buffer as 64-bit (signed integer or float), and
    • cast the 64-bit array to the appropriate data type