Skip to main content

concat_batches

Function concat_batches 

pub fn concat_batches<'a>(
    schema: &Arc<Schema>,
    input_batches: impl IntoIterator<Item = &'a RecordBatch>,
) -> Result<RecordBatch, ArrowError>
Expand description

Concatenates batches together into a single RecordBatch.

The output batch has the specified schemas; The schema of the input are ignored.

§Notes

  • Callers should budget for peak memory use to approach 2x the input size, as the input batches and output arrays co-exist during construction.
  • Arrays with i32 offsets, such as StringArray and BinaryArray, only support up to ~2GiB of payloads. Concatenating large arrays of these types can cause offset overflows.

§Errors

Returns an error if the types of underlying arrays are different.