arrow_data::transform

Struct MutableArrayData

Source
pub struct MutableArrayData<'a> {
    arrays: Vec<&'a ArrayData>,
    data: _MutableArrayData<'a>,
    dictionary: Option<ArrayData>,
    variadic_data_buffers: Vec<Buffer>,
    extend_values: Vec<Box<dyn Fn(&mut _MutableArrayData<'_>, usize, usize, usize) + 'a>>,
    extend_null_bits: Vec<Box<dyn Fn(&mut _MutableArrayData<'_>, usize, usize) + 'a>>,
    extend_nulls: Box<dyn Fn(&mut _MutableArrayData<'_>, usize)>,
}
Expand description

Efficiently create an ArrayData from one or more existing ArrayDatas by copying chunks.

The main use case of this struct is to perform unary operations to arrays of arbitrary types, such as filter and take.

§Example

use arrow_buffer::Buffer;
use arrow_data::ArrayData;
use arrow_data::transform::MutableArrayData;
use arrow_schema::DataType;
fn i32_array(values: &[i32]) -> ArrayData {
  ArrayData::try_new(DataType::Int32, 5, None, 0, vec![Buffer::from_slice_ref(values)], vec![]).unwrap()
}
let arr1  = i32_array(&[1, 2, 3, 4, 5]);
let arr2  = i32_array(&[6, 7, 8, 9, 10]);
// Create a mutable array for copying values from arr1 and arr2, with a capacity for 6 elements
let capacity = 3 * std::mem::size_of::<i32>();
let mut mutable = MutableArrayData::new(vec![&arr1, &arr2], false, 10);
// Copy the first 3 elements from arr1
mutable.extend(0, 0, 3);
// Copy the last 3 elements from arr2
mutable.extend(1, 2, 4);
// Complete the MutableArrayData into a new ArrayData
let frozen = mutable.freeze();
assert_eq!(frozen, i32_array(&[1, 2, 3, 8, 9, 10]));

Fields§

§arrays: Vec<&'a ArrayData>

Input arrays: the data being read FROM.

Note this is “dead code” because all actual references to the arrays are stored in closures for extending values and nulls.

§data: _MutableArrayData<'a>

In progress output array: The data being written TO

Note these fields are in a separate struct, _MutableArrayData, as they cannot be in MutableArrayData itself due to mutability invariants (interior mutability): MutableArrayData contains a function that can only mutate _MutableArrayData, not MutableArrayData itself

§dictionary: Option<ArrayData>

The child data of the Array in Dictionary arrays.

This is not stored in _MutableArrayData because these values are constant and only needed at the end, when freezing _MutableArrayData.

§variadic_data_buffers: Vec<Buffer>

Variadic data buffers referenced by views.

Note this this is not stored in _MutableArrayData because these values are constant and only needed at the end, when freezing _MutableArrayData

§extend_values: Vec<Box<dyn Fn(&mut _MutableArrayData<'_>, usize, usize, usize) + 'a>>

function used to extend output array with values from input arrays.

This function’s lifetime is bound to the input arrays because it reads values from them.

§extend_null_bits: Vec<Box<dyn Fn(&mut _MutableArrayData<'_>, usize, usize) + 'a>>

function used to extend the output array with nulls from input arrays.

This function’s lifetime is bound to the input arrays because it reads nulls from it.

§extend_nulls: Box<dyn Fn(&mut _MutableArrayData<'_>, usize)>

function used to extend the output array with null elements.

This function is independent of the arrays and therefore has no lifetime.

Implementations§

Source§

impl<'a> MutableArrayData<'a>

Source

pub fn new(arrays: Vec<&'a ArrayData>, use_nulls: bool, capacity: usize) -> Self

Returns a new MutableArrayData with capacity to capacity slots and specialized to create an ArrayData from multiple arrays.

§Arguments
  • arrays - the source arrays to copy from
  • use_nulls - a flag used to optimize insertions
  • capacity - the preallocated capacity of the output array, in bytes

Thus, if use_nulls is false, calling MutableArrayData::extend_nulls should not be used.

Source

pub fn with_capacities( arrays: Vec<&'a ArrayData>, use_nulls: bool, capacities: Capacities, ) -> Self

Similar to MutableArrayData::new, but lets users define the preallocated capacities of the array with more granularity.

See MutableArrayData::new for more information on the arguments.

§Panics

This function panics if the given capacities don’t match the data type of arrays. Or when a Capacities variant is not yet supported.

Source

pub fn extend(&mut self, index: usize, start: usize, end: usize)

Extends the in progress array with a region of the input arrays

§Arguments
  • index - the index of array that you what to copy values from
  • start - the start index of the chunk (inclusive)
  • end - the end index of the chunk (exclusive)
§Panic

This function panics if there is an invalid index, i.e. index >= the number of source arrays or end > the length of the indexth array

Source

pub fn extend_nulls(&mut self, len: usize)

Extends the in progress array with null elements, ignoring the input arrays.

§Panics

Panics if MutableArrayData not created with use_nulls or nullable source arrays

Source

pub fn len(&self) -> usize

Returns the current length

Source

pub fn is_empty(&self) -> bool

Returns true if len is 0

Source

pub fn null_count(&self) -> usize

Returns the current null count

Source

pub fn freeze(self) -> ArrayData

Creates a ArrayData from the in progress array, consuming self.

Source

pub fn into_builder(self) -> ArrayDataBuilder

Consume self and returns the in progress array as ArrayDataBuilder.

This is useful for extending the default behavior of MutableArrayData.

Trait Implementations§

Source§

impl Debug for MutableArrayData<'_>

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

§

impl<'a> Freeze for MutableArrayData<'a>

§

impl<'a> !RefUnwindSafe for MutableArrayData<'a>

§

impl<'a> !Send for MutableArrayData<'a>

§

impl<'a> !Sync for MutableArrayData<'a>

§

impl<'a> Unpin for MutableArrayData<'a>

§

impl<'a> !UnwindSafe for MutableArrayData<'a>

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.