Struct VariableShapeTensor

pub struct VariableShapeTensor {
    value_type: DataType,
    dimensions: usize,
    metadata: VariableShapeTensorMetadata,
}

The extension type for VariableShapeTensor.

Extension name: arrow.variable_shape_tensor.

The storage type of the extension is: StructArray where struct is composed of data and shape fields describing a single tensor per row:

• data is a List holding tensor elements (each list element is a single tensor). The List’s value type is the value type of the tensor, such as an integer or floating-point type.
• shape is a FixedSizeList<int32>[ndim] of the tensor shape where the size of the list ndim is equal to the number of dimensions of the tensor.

Extension type parameters: value_type: the Arrow data type of individual tensor elements.

Optional parameters describing the logical layout:

• dim_names: explicit names to tensor dimensions as an array. The length of it should be equal to the shape length and equal to the number of dimensions. dim_names can be used if the dimensions have well-known names and they map to the physical layout (row-major).
• permutation: indices of the desired ordering of the original dimensions, defined as an array. The indices contain a permutation of the values [0, 1, .., N-1] where N is the number of dimensions. The permutation indicates which dimension of the logical layout corresponds to which dimension of the physical tensor (the i-th dimension of the logical view corresponds to the dimension with number permutations[i] of the physical tensor). Permutation can be useful in case the logical order of the tensor is a permutation of the physical order (row-major). When logical and physical layout are equal, the permutation will always be ([0, 1, .., N-1]) and can therefore be left out.
• uniform_shape: sizes of individual tensor’s dimensions which are guaranteed to stay constant in uniform dimensions and can vary in non- uniform dimensions. This holds over all tensors in the array. Sizes in uniform dimensions are represented with int32 values, while sizes of the non-uniform dimensions are not known in advance and are represented with null. If uniform_shape is not provided it is assumed that all dimensions are non-uniform. An array containing a tensor with shape (2, 3, 4) and whose first and last dimensions are uniform would have uniform_shape (2, null, 4). This allows for interpreting the tensor correctly without accounting for uniform dimensions while still permitting optional optimizations that take advantage of the uniformity.

https://arrow.apache.org/docs/format/CanonicalExtensions.html#variable-shape-tensor

Fields

value_type: DataType

The data type of individual tensor elements.

dimensions: usize

The number of dimensions of the tensor.

metadata: VariableShapeTensorMetadata

The metadata of this extension type.

Implementations

impl VariableShapeTensor

pub fn try_new( value_type: DataType, dimensions: usize, dimension_names: Option<Vec<String>>, permutations: Option<Vec<usize>>, uniform_shapes: Option<Vec<Option<i32>>>, ) -> Result<Self, ArrowError>

Returns a new variable shape tensor extension type.

Error

Return an error if the provided dimension names, permutations or uniform shapes are invalid.

pub fn value_type(&self) -> &DataType

Returns the value type of the individual tensor elements.

pub fn dimensions(&self) -> usize

Returns the number of dimensions in this variable shape tensor.

pub fn dimension_names(&self) -> Option<&[String]>

Returns the names of the dimensions in this variable shape tensor, if set.

pub fn permutations(&self) -> Option<&[usize]>

Returns the indices of the desired ordering of the original dimensions, if set.

pub fn uniform_shapes(&self) -> Option<&[Option<i32>]>

Returns sizes of individual tensor’s dimensions which are guaranteed to stay constant in uniform dimensions and can vary in non-uniform dimensions.

Trait Implementations

impl Clone for VariableShapeTensor

fn clone(&self) -> VariableShapeTensor

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for VariableShapeTensor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl ExtensionType for VariableShapeTensor

const NAME: &'static str = "arrow.variable_shape_tensor"

The name identifying this extension type.

type Metadata = VariableShapeTensorMetadata

The metadata type of this extension type.

fn metadata(&self) -> &Self::Metadata

Returns a reference to the metadata of this extension type, or &() if if this extension type defines no metadata (Self::Metadata=()).

fn serialize_metadata(&self) -> Option<String>

Returns the serialized representation of the metadata of this extension type, or None if this extension type defines no metadata (Self::Metadata=()).

fn deserialize_metadata( metadata: Option<&str>, ) -> Result<Self::Metadata, ArrowError>

Deserialize the metadata of this extension type from the serialized representation of the metadata. An extension type that defines no metadata should expect None for the serialized metadata and return Ok(()).

fn supports_data_type(&self, data_type: &DataType) -> Result<(), ArrowError>

Returns OK()) iff the given data type is supported by this extension type.

fn try_new( data_type: &DataType, metadata: Self::Metadata, ) -> Result<Self, ArrowError>

Construct this extension type for a field with the given data type and metadata.

impl From<VariableShapeTensor> for CanonicalExtensionType

fn from(value: VariableShapeTensor) -> Self

Converts to this type from the input type.

impl PartialEq for VariableShapeTensor

fn eq(&self, other: &VariableShapeTensor) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for VariableShapeTensor