Struct GenericListArray
pub struct GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,{
data_type: DataType,
nulls: Option<NullBuffer>,
values: Arc<dyn Array>,
value_offsets: OffsetBuffer<OffsetSize>,
}
Expand description
An array of variable length lists, similar to JSON arrays
(e.g. ["A", "B", "C"]
).
Lists are represented using offsets
into a values
child
array. Offsets are stored in two adjacent entries of an
OffsetBuffer
.
Arrow defines ListArray
with i32
offsets and
LargeListArray
with i64
offsets.
Use GenericListBuilder
to construct a GenericListArray
.
§Representation
A ListArray
can represent a list of values of any other
supported Arrow type. Each element of the ListArray
itself is
a list which may be empty, may contain NULL and non-null values,
or may itself be NULL.
For example, the ListArray
shown in the following diagram stores
lists of strings. Note that []
represents an empty (length
0), but non NULL list.
┌─────────────┐
│ [A,B,C] │
├─────────────┤
│ [] │
├─────────────┤
│ NULL │
├─────────────┤
│ [D] │
├─────────────┤
│ [NULL, F] │
└─────────────┘
The values
are stored in a child StringArray
and the offsets
are stored in an OffsetBuffer
as shown in the following
diagram. The logical values and offsets are shown on the left, and
the actual ListArray
encoding on the right.
┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─
┌ ─ ─ ─ ─ ─ ─ ┐ │
┌─────────────┐ ┌───────┐ │ ┌───┐ ┌───┐ ┌───┐ ┌───┐
│ [A,B,C] │ │ (0,3) │ │ 1 │ │ 0 │ │ │ 1 │ │ A │ │ 0 │
├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
│ [] │ │ (3,3) │ │ 1 │ │ 3 │ │ │ 1 │ │ B │ │ 1 │
├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
│ NULL │ │ (3,4) │ │ 0 │ │ 3 │ │ │ 1 │ │ C │ │ 2 │
├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
│ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ ? │ │ ? │ │ 3 │
├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
│ [NULL, F] │ │ (5,7) │ │ 1 │ │ 5 │ │ │ 1 │ │ D │ │ 4 │
└─────────────┘ └───────┘ │ └───┘ ├───┤ ├───┤ ├───┤
│ 7 │ │ │ 0 │ │ ? │ │ 5 │
│ Validity └───┘ ├───┤ ├───┤
Logical Logical (nulls) Offsets │ │ 1 │ │ F │ │ 6 │
Values Offsets │ └───┘ └───┘
│ Values │ │
(offsets[i], │ ListArray (Array)
offsets[i+1]) └ ─ ─ ─ ─ ─ ─ ┘ │
└ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─
Fields§
§data_type: DataType
§nulls: Option<NullBuffer>
§values: Arc<dyn Array>
§value_offsets: OffsetBuffer<OffsetSize>
Implementations§
§impl<OffsetSize> GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,
impl<OffsetSize> GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,
pub const DATA_TYPE_CONSTRUCTOR: fn(_: Arc<Field>) -> DataType = _
pub const DATA_TYPE_CONSTRUCTOR: fn(_: Arc<Field>) -> DataType = _
The data type constructor of list array.
The input is the schema of the child array and
the output is the DataType
, List or LargeList.
pub fn try_new(
field: Arc<Field>,
offsets: OffsetBuffer<OffsetSize>,
values: Arc<dyn Array>,
nulls: Option<NullBuffer>,
) -> Result<GenericListArray<OffsetSize>, ArrowError>
pub fn try_new( field: Arc<Field>, offsets: OffsetBuffer<OffsetSize>, values: Arc<dyn Array>, nulls: Option<NullBuffer>, ) -> Result<GenericListArray<OffsetSize>, ArrowError>
Create a new GenericListArray
from the provided parts
§Errors
Errors if
offsets.len() - 1 != nulls.len()
offsets.last() > values.len()
!field.is_nullable() && values.is_nullable()
field.data_type() != values.data_type()
pub fn new(
field: Arc<Field>,
offsets: OffsetBuffer<OffsetSize>,
values: Arc<dyn Array>,
nulls: Option<NullBuffer>,
) -> GenericListArray<OffsetSize>
pub fn new( field: Arc<Field>, offsets: OffsetBuffer<OffsetSize>, values: Arc<dyn Array>, nulls: Option<NullBuffer>, ) -> GenericListArray<OffsetSize>
Create a new GenericListArray
from the provided parts
§Panics
Panics if Self::try_new
returns an error
pub fn new_null(field: Arc<Field>, len: usize) -> GenericListArray<OffsetSize>
pub fn new_null(field: Arc<Field>, len: usize) -> GenericListArray<OffsetSize>
Create a new GenericListArray
of length len
where all values are null
pub fn into_parts(
self,
) -> (Arc<Field>, OffsetBuffer<OffsetSize>, Arc<dyn Array>, Option<NullBuffer>)
pub fn into_parts( self, ) -> (Arc<Field>, OffsetBuffer<OffsetSize>, Arc<dyn Array>, Option<NullBuffer>)
Deconstruct this array into its constituent parts
pub fn offsets(&self) -> &OffsetBuffer<OffsetSize>
pub fn offsets(&self) -> &OffsetBuffer<OffsetSize>
Returns a reference to the offsets of this list
Unlike Self::value_offsets
this returns the OffsetBuffer
allowing for zero-copy cloning
pub fn value_type(&self) -> DataType
pub fn value_type(&self) -> DataType
Returns a clone of the value type of this list.
pub unsafe fn value_unchecked(&self, i: usize) -> Arc<dyn Array>
pub unsafe fn value_unchecked(&self, i: usize) -> Arc<dyn Array>
Returns ith value of this list array.
§Safety
Caller must ensure that the index is within the array bounds
pub fn value_offsets(&self) -> &[OffsetSize]
pub fn value_offsets(&self) -> &[OffsetSize]
Returns the offset values in the offsets buffer
pub fn value_length(&self, i: usize) -> OffsetSize
pub fn value_length(&self, i: usize) -> OffsetSize
Returns the length for value at index i
.
pub fn iter<'a>(&'a self) -> ArrayIter<&'a GenericListArray<OffsetSize>> ⓘ
pub fn iter<'a>(&'a self) -> ArrayIter<&'a GenericListArray<OffsetSize>> ⓘ
constructs a new iterator
pub fn slice(
&self,
offset: usize,
length: usize,
) -> GenericListArray<OffsetSize>
pub fn slice( &self, offset: usize, length: usize, ) -> GenericListArray<OffsetSize>
Returns a zero-copy slice of this array with the indicated offset and length.
pub fn from_iter_primitive<T, P, I>(iter: I) -> GenericListArray<OffsetSize>where
T: ArrowPrimitiveType,
P: IntoIterator<Item = Option<<T as ArrowPrimitiveType>::Native>>,
I: IntoIterator<Item = Option<P>>,
pub fn from_iter_primitive<T, P, I>(iter: I) -> GenericListArray<OffsetSize>where
T: ArrowPrimitiveType,
P: IntoIterator<Item = Option<<T as ArrowPrimitiveType>::Native>>,
I: IntoIterator<Item = Option<P>>,
Creates a GenericListArray
from an iterator of primitive values
§Example
let data = vec![
Some(vec![Some(0), Some(1), Some(2)]),
None,
Some(vec![Some(3), None, Some(5)]),
Some(vec![Some(6), Some(7)]),
];
let list_array = ListArray::from_iter_primitive::<Int32Type, _, _>(data);
println!("{:?}", list_array);
Trait Implementations§
§impl<OffsetSize> Array for GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,
impl<OffsetSize> Array for GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,
§fn slice(&self, offset: usize, length: usize) -> Arc<dyn Array>
fn slice(&self, offset: usize, length: usize) -> Arc<dyn Array>
§fn shrink_to_fit(&mut self)
fn shrink_to_fit(&mut self)
§fn offset(&self) -> usize
fn offset(&self) -> usize
0
. Read more§fn nulls(&self) -> Option<&NullBuffer>
fn nulls(&self) -> Option<&NullBuffer>
§fn logical_null_count(&self) -> usize
fn logical_null_count(&self) -> usize
§fn get_buffer_memory_size(&self) -> usize
fn get_buffer_memory_size(&self) -> usize
§fn get_array_memory_size(&self) -> usize
fn get_array_memory_size(&self) -> usize
get_buffer_memory_size()
and
includes the overhead of the data structures that contain the pointers to the various buffers.§fn logical_nulls(&self) -> Option<NullBuffer>
fn logical_nulls(&self) -> Option<NullBuffer>
NullBuffer
that represents the logical
null values of this array, if any. Read more§fn null_count(&self) -> usize
fn null_count(&self) -> usize
§fn is_nullable(&self) -> bool
fn is_nullable(&self) -> bool
false
if the array is guaranteed to not contain any logical nulls Read more§impl<OffsetSize> ArrayAccessor for &GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,
impl<OffsetSize> ArrayAccessor for &GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,
§fn value(
&self,
index: usize,
) -> <&GenericListArray<OffsetSize> as ArrayAccessor>::Item
fn value( &self, index: usize, ) -> <&GenericListArray<OffsetSize> as ArrayAccessor>::Item
i
Read more§unsafe fn value_unchecked(
&self,
index: usize,
) -> <&GenericListArray<OffsetSize> as ArrayAccessor>::Item
unsafe fn value_unchecked( &self, index: usize, ) -> <&GenericListArray<OffsetSize> as ArrayAccessor>::Item
i
Read more§impl<OffsetSize> Clone for GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,
impl<OffsetSize> Clone for GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,
§fn clone(&self) -> GenericListArray<OffsetSize>
fn clone(&self) -> GenericListArray<OffsetSize>
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read more