Struct MutableArrayData

pub struct MutableArrayData<'a> {
    arrays: Vec<&'a ArrayData>,
    data: _MutableArrayData<'a>,
    dictionary: Option<ArrayData>,
    variadic_data_buffers: Vec<Buffer>,
    extend_values: Vec<Box<dyn Fn(&mut _MutableArrayData<'_>, usize, usize, usize) -> Result<(), ArrowError> + 'a>>,
    extend_null_bits: Vec<Box<dyn Fn(&mut _MutableArrayData<'_>, usize, usize) + 'a>>,
    extend_nulls: Box<dyn Fn(&mut _MutableArrayData<'_>, usize) -> Result<(), ArrowError>>,
}

Expand description

Efficiently create an ArrayData from one or more existing ArrayDatas by copying chunks.

The main use case of this struct is to perform unary operations to arrays of arbitrary types, such as filter and take.

§Example

use arrow_buffer::Buffer;
use arrow_data::ArrayData;
use arrow_data::transform::MutableArrayData;
use arrow_schema::DataType;
fn i32_array(values: &[i32]) -> ArrayData {
  ArrayData::try_new(DataType::Int32, values.len(), None, 0, vec![Buffer::from_slice_ref(values)], vec![]).unwrap()
}
let arr1  = i32_array(&[1, 2, 3, 4, 5]);
let arr2  = i32_array(&[6, 7, 8, 9, 10]);
// Create a mutable array for copying values from arr1 and arr2, with a capacity for 6 elements
let capacity = 3 * std::mem::size_of::<i32>();
let mut mutable = MutableArrayData::new(vec![&arr1, &arr2], false, 10);
// Copy the first 3 elements from arr1
mutable.extend(0, 0, 3);
// Copy the last 3 elements from arr2
mutable.extend(1, 2, 5);
// Complete the MutableArrayData into a new ArrayData
let frozen = mutable.freeze();
assert_eq!(frozen, i32_array(&[1, 2, 3, 8, 9, 10]));

Fields§

§arrays: Vec<&'a ArrayData>§data: _MutableArrayData<'a>§dictionary: Option<ArrayData>§variadic_data_buffers: Vec<Buffer>§extend_values: Vec<Box<dyn Fn(&mut _MutableArrayData<'_>, usize, usize, usize) -> Result<(), ArrowError> + 'a>>§extend_null_bits: Vec<Box<dyn Fn(&mut _MutableArrayData<'_>, usize, usize) + 'a>>§extend_nulls: Box<dyn Fn(&mut _MutableArrayData<'_>, usize) -> Result<(), ArrowError>>

Implementations§

§

impl<'a> MutableArrayData<'a>

pub fn new( arrays: Vec<&'a ArrayData>, use_nulls: bool, capacity: usize, ) -> MutableArrayData<'a>

Returns a new MutableArrayData with capacity to capacity slots and specialized to create an ArrayData from multiple arrays.

§Arguments

arrays - the source arrays to copy from
use_nulls - a flag indicating whether the caller intends to call extend_nulls. Note: null-handling is enabled automatically if any source array contains nulls.
capacity - the preallocated capacity of the output array, in slots (number of elements)

if use_nulls is false and no source arrays contains nulls, calling MutableArrayData::extend_nulls or MutableArrayData::try_extend_nulls will panic.

pub fn try_new( arrays: Vec<&'a ArrayData>, use_nulls: bool, capacity: usize, ) -> Result<MutableArrayData<'a>, ArrowError>

Fallible variant of MutableArrayData::new.

Unlike MutableArrayData::new, this does not panic when merging dictionary arrays whose combined values would overflow the dictionary key type. Instead, it returns an error, letting callers (e.g. interleave / concat) surface it as a normal error.

pub fn with_capacities( arrays: Vec<&'a ArrayData>, use_nulls: bool, capacities: Capacities, ) -> MutableArrayData<'a>

Similar to MutableArrayData::new, but lets users define the preallocated capacities of the array with more granularity.

See MutableArrayData::new for more information on the arguments.

§Panics

if the given capacities don’t match the data type of arrays
if a Capacities variant is not yet supported
when merging dictionary arrays whose combined values overflow the dictionary key type — see MutableArrayData::try_with_capacities for a fallible variant

pub fn try_with_capacities( arrays: Vec<&'a ArrayData>, use_nulls: bool, capacities: Capacities, ) -> Result<MutableArrayData<'a>, ArrowError>

Fallible variant of MutableArrayData::with_capacities.

Returns an error instead of panicking when merging dictionary arrays whose combined values would overflow the dictionary key type. Still panics for other unsupported combinations (inconsistent input types, unsupported Capacities variants) as documented on MutableArrayData::with_capacities.

pub fn try_extend( &mut self, index: usize, start: usize, end: usize, ) -> Result<(), ArrowError>

Extends the in progress array with a region of the input arrays, returning an error on overflow.

§Arguments

index - the index of array that you want to copy values from
start - the start index of the chunk (inclusive)
end - the end index of the chunk (exclusive)

§Errors

Returns an error if

index >= the number of source arrays,
start..end is not a valid range within the indexth array, or
offset arithmetic overflows the underlying integer type.

pub fn extend(&mut self, index: usize, start: usize, end: usize)

👎Deprecated since 59.0.0:

Use try_extend which returns an error on overflow instead of panicking

Extends the in progress array with a region of the input arrays.

§Panics

This function panics if

index >= the number of source arrays,
start..end is not a valid range within the indexth array, or
the offset type overflows (e.g. more than 2 GiB in a StringArray).

pub fn try_extend_nulls(&mut self, len: usize) -> Result<(), ArrowError>

Extends the in progress array with null elements, ignoring the input arrays, returning an error on overflow.

Prefer this over extend_nulls to handle cases where the run-end counter overflows (relevant for RunEndEncoded arrays).

§Errors

Returns an error if this MutableArrayData was not created with use_nulls and none of the source arrays are nullable, or if the run-end counter overflows.