Module org.apache.arrow.variant

Package org.apache.arrow.variant.extension

Class VariantVector

java.lang.Object

org.apache.arrow.vector.BaseValueVector

org.apache.arrow.vector.ExtensionTypeVector<StructVector>

org.apache.arrow.variant.extension.VariantVector

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<ValueVector>, FieldVector, ValueVector

public class VariantVector
extends ExtensionTypeVector<StructVector>

Arrow vector for storing VariantType values.

Stores semi-structured data (like JSON) as metadata + value binary pairs, allowing type-flexible columnar storage within Arrow's type system.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static class	VariantVector.VariantTransferPair	VariantTransferPair is a transfer pair for VariantVector.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	METADATA_VECTOR_NAME
static final String	VALUE_VECTOR_NAME

Fields inherited from class org.apache.arrow.vector.BaseValueVector

allocator, fieldReader, INITIAL_VALUE_ALLOCATION, MAX_ALLOCATION_SIZE, MAX_ALLOCATION_SIZE_PROPERTY, validityBuffer, valueCount

Constructor Summary

Constructors

Constructor	Description
VariantVector(String name, BufferAllocator allocator)	Constructs a new VariantVector with the given name and allocator.

Method Summary

Modifier and Type	Method	Description
void	copyFrom(int fromIndex, int thisIndex, ValueVector from)	Copy a cell value from a particular index in source vector to a particular position in this vector.
void	copyFromSafe(int fromIndex, int thisIndex, ValueVector from)	Same as ValueVector.copyFrom(int, int, ValueVector) except that it handles the case when the capacity of the vector needs to be expanded before copy.
static List<Field>	createVariantChildFields()	Creates the child fields for the VariantVector.
static Field	createVariantField(String name)	Creates a new VariantVector with the given name.
void	get(int index, NullableVariantHolder holder)	Retrieves the variant value at the specified index into the provided holder.
void	get(int index, VariantHolder holder)	Retrieves the variant value at the specified index into the provided non-nullable holder.
Field	getField()	Get information about how this field is materialized.
VarBinaryVector	getMetadataVector()
Object	getObject(int index)	Get the extension object at the specified index.
protected FieldReader	getReaderImpl()	Each vector has a different reader that implements the FieldReader interface.
TransferPair	getTransferPair(String ref, BufferAllocator allocator)	To transfer quota responsibility.
TransferPair	getTransferPair(String ref, BufferAllocator allocator, CallBack callBack)	To transfer quota responsibility.
TransferPair	getTransferPair(BufferAllocator allocator)	To transfer quota responsibility.
TransferPair	getTransferPair(Field field, BufferAllocator allocator)	To transfer quota responsibility.
TransferPair	getTransferPair(Field field, BufferAllocator allocator, CallBack callBack)	To transfer quota responsibility.
VarBinaryVector	getValueVector()
int	hashCode(int index)	Returns hashCode of element in index with the default hasher.
int	hashCode(int index, ArrowBufHasher hasher)	Returns hashCode of element in index with the given hasher.
void	initializeChildrenFromFields(List<Field> children)	Initializes the child vectors to be later loaded with loadBuffers.
TransferPair	makeTransferPair(ValueVector target)	Makes a new transfer pair used to transfer underlying buffers.
void	set(int index, NullableVariantHolder holder)	Sets the variant value at the specified index from the provided nullable holder.
void	set(int index, VariantHolder holder)	Sets the variant value at the specified index from the provided holder.
void	setSafe(int index, NullableVariantHolder holder)	Sets the variant value at the specified index from the provided nullable holder, with bounds checking.
void	setSafe(int index, VariantHolder holder)	Sets the variant value at the specified index from the provided holder, with bounds checking.
void	setSafe(int index, Variant variant)	Sets the value at the given index from the provided Variant.

Methods inherited from class org.apache.arrow.vector.ExtensionTypeVector

accept, allocateNew, allocateNewSafe, clear, close, getAllocator, getBuffers, getBufferSize, getBufferSizeFor, getChildrenFromFields, getDataBuffer, getDataBufferAddress, getFieldBuffers, getFieldInnerVectors, getMinorType, getName, getNullCount, getOffsetBuffer, getOffsetBufferAddress, getUnderlyingVector, getValidityBuffer, getValidityBufferAddress, getValueCapacity, getValueCount, isNull, iterator, loadFieldBuffers, reAlloc, reset, setInitialCapacity, setNull, setValueCount

Methods inherited from class org.apache.arrow.vector.BaseValueVector

allocateValidityBuffer, checkBufRefs, copyValidityBuffer, getReader, getValidityBufferSizeFromCount, releaseBuffer, sliceAndTransferValidityBuffer, splitAndTransferValidityBuffer, toString, transferBuffer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.arrow.vector.FieldVector

exportBuffer, exportCDataBuffers, getExportedCDataBufferCount

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Methods inherited from interface org.apache.arrow.vector.ValueVector

getReader, validate, validateFull

Field Details

METADATA_VECTOR_NAME

public static final String METADATA_VECTOR_NAME

See Also:

• Constant Field Values

VALUE_VECTOR_NAME

public static final String VALUE_VECTOR_NAME

See Also:

• Constant Field Values

Constructor Details

VariantVector

public VariantVector(String name,
 BufferAllocator allocator)

Constructs a new VariantVector with the given name and allocator.

Parameters:

name - the name of the vector

allocator - the buffer allocator for memory management

Method Details

createVariantField

public static Field createVariantField(String name)

Creates a new VariantVector with the given name. The Variant Field schema has to be the same everywhere, otherwise ArrowBuffer loading might fail during serialization/deserialization and schema mismatches can occur. This includes CompleteType's VARIANT and VARIANT_REQUIRED types.

createVariantChildFields

public static List<Field> createVariantChildFields()

Creates the child fields for the VariantVector. Metadata vector will be index 0 and value vector will be index 1.

initializeChildrenFromFields

public void initializeChildrenFromFields(List<Field> children)

Description copied from interface: FieldVector

Initializes the child vectors to be later loaded with loadBuffers.

Specified by:

initializeChildrenFromFields in interface FieldVector

Overrides:

initializeChildrenFromFields in class ExtensionTypeVector<StructVector>

Parameters:

children - the schema

getField

public Field getField()

Description copied from interface: ValueVector

Get information about how this field is materialized.

Specified by:

getField in interface ValueVector

Overrides:

getField in class ExtensionTypeVector<StructVector>

Returns:

the field corresponding to this vector

getMetadataVector

public VarBinaryVector getMetadataVector()

getValueVector

public VarBinaryVector getValueVector()

makeTransferPair

public TransferPair makeTransferPair(ValueVector target)

Description copied from interface: ValueVector

Makes a new transfer pair used to transfer underlying buffers.

Specified by:

makeTransferPair in interface ValueVector

Overrides:

makeTransferPair in class ExtensionTypeVector<StructVector>

Parameters:

target - the target for the transfer

Returns:

a new transfer pair that is used to transfer underlying buffers into the target vector.

getTransferPair

public TransferPair getTransferPair(Field field,
 BufferAllocator allocator)

Description copied from interface: ValueVector

To transfer quota responsibility.

Specified by:

getTransferPair in interface ValueVector

Overrides:

getTransferPair in class ExtensionTypeVector<StructVector>

Parameters:

field - the Field object used by the target vector

allocator - the target allocator

Returns:

a transfer pair, creating a new target vector of the same type.

getTransferPair

public TransferPair getTransferPair(Field field,
 BufferAllocator allocator,
 CallBack callBack)

Description copied from interface: ValueVector

To transfer quota responsibility.

Specified by:

getTransferPair in interface ValueVector

Overrides:

getTransferPair in class ExtensionTypeVector<StructVector>

Parameters:

field - the Field object used by the target vector

allocator - the target allocator

callBack - A schema change callback.

Returns:

a transfer pair, creating a new target vector of the same type.

getTransferPair

public TransferPair getTransferPair(String ref,
 BufferAllocator allocator)

Description copied from interface: ValueVector

To transfer quota responsibility.

Specified by:

getTransferPair in interface ValueVector

Overrides:

getTransferPair in class ExtensionTypeVector<StructVector>

Parameters:

ref - the name of the vector

allocator - the target allocator

Returns:

a transfer pair, creating a new target vector of the same type.

getTransferPair

public TransferPair getTransferPair(String ref,
 BufferAllocator allocator,
 CallBack callBack)

Description copied from interface: ValueVector

To transfer quota responsibility.

Specified by:

getTransferPair in interface ValueVector

Overrides:

getTransferPair in class ExtensionTypeVector<StructVector>

Parameters:

ref - the name of the vector

allocator - the target allocator

callBack - A schema change callback.

Returns:

a transfer pair, creating a new target vector of the same type.

getTransferPair

public TransferPair getTransferPair(BufferAllocator allocator)

Description copied from interface: ValueVector

To transfer quota responsibility.

Specified by:

getTransferPair in interface ValueVector

Overrides:

getTransferPair in class ExtensionTypeVector<StructVector>

Parameters:

allocator - the target allocator

Returns:

a transfer pair, creating a new target vector of the same type.

copyFrom

public void copyFrom(int fromIndex,
 int thisIndex,
 ValueVector from)

Description copied from interface: ValueVector

Copy a cell value from a particular index in source vector to a particular position in this vector.

Specified by:

copyFrom in interface ValueVector

Overrides:

copyFrom in class BaseValueVector

Parameters:

fromIndex - position to copy from in source vector

thisIndex - position to copy to in this vector

from - source vector

copyFromSafe

public void copyFromSafe(int fromIndex,
 int thisIndex,
 ValueVector from)

Description copied from interface: ValueVector

Same as ValueVector.copyFrom(int, int, ValueVector) except that it handles the case when the capacity of the vector needs to be expanded before copy.

Specified by:

copyFromSafe in interface ValueVector

Overrides:

copyFromSafe in class BaseValueVector

Parameters:

fromIndex - position to copy from in source vector

thisIndex - position to copy to in this vector

from - source vector

getObject

public Object getObject(int index)

Description copied from class: ExtensionTypeVector

Get the extension object at the specified index.

Generally, this should access the underlying vector and construct the corresponding Java object from the raw data.

Specified by:

getObject in interface ValueVector

Specified by:

getObject in class ExtensionTypeVector<StructVector>

Parameters:

index - index of object to get

Returns:

friendly type object, null if value is unset

get

public void get(int index,
 NullableVariantHolder holder)

Retrieves the variant value at the specified index into the provided holder.

Parameters:

index - the index of the value to retrieve

holder - the holder to populate with the variant data

get

public void get(int index,
 VariantHolder holder)

Retrieves the variant value at the specified index into the provided non-nullable holder.

Parameters:

index - the index of the value to retrieve

holder - the holder to populate with the variant data

set

public void set(int index,
 VariantHolder holder)

Sets the variant value at the specified index from the provided holder.

Parameters:

index - the index at which to set the value

holder - the holder containing the variant data to set

set

public void set(int index,
 NullableVariantHolder holder)

Sets the variant value at the specified index from the provided nullable holder.

Parameters:

index - the index at which to set the value

holder - the nullable holder containing the variant data to set

setSafe

public void setSafe(int index,
 VariantHolder holder)

Sets the variant value at the specified index from the provided holder, with bounds checking.

Parameters:

index - the index at which to set the value

holder - the holder containing the variant data to set

setSafe

public void setSafe(int index,
 NullableVariantHolder holder)

Sets the variant value at the specified index from the provided nullable holder, with bounds checking.

Parameters:

index - the index at which to set the value

holder - the nullable holder containing the variant data to set

setSafe

public void setSafe(int index,
 Variant variant)

Sets the value at the given index from the provided Variant.

getReaderImpl

protected FieldReader getReaderImpl()

Description copied from class: BaseValueVector

Each vector has a different reader that implements the FieldReader interface. Overridden methods must make sure to return the correct concrete reader implementation.

Overrides:

getReaderImpl in class ExtensionTypeVector<StructVector>

Returns:

Returns a lambda that initializes a reader when called.

hashCode

public int hashCode(int index)

Description copied from interface: ValueVector

Returns hashCode of element in index with the default hasher.

hashCode

public int hashCode(int index,
 ArrowBufHasher hasher)

Description copied from interface: ValueVector

Returns hashCode of element in index with the given hasher.