Module org.apache.arrow.vector

Package org.apache.arrow.vector

Class LargeVarBinaryVector

java.lang.Object

org.apache.arrow.vector.BaseValueVector

org.apache.arrow.vector.BaseLargeVariableWidthVector

org.apache.arrow.vector.LargeVarBinaryVector

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<ValueVector>, DensityAwareVector, ElementAddressableVector, FieldVector, ValueIterableVector<byte[]>, ValueVector, VariableWidthFieldVector, VariableWidthVector, VectorDefinitionSetter

public final class LargeVarBinaryVector
extends BaseLargeVariableWidthVector
implements ValueIterableVector<byte[]>

LargeVarBinaryVector implements a large variable width vector of binary values which could be NULL. A validity buffer (bit vector) is maintained to track which elements in the vector are null. The size of the underlying buffer can be over 2GB.

Field Summary

Fields inherited from class org.apache.arrow.vector.BaseLargeVariableWidthVector

emptyByteArray, field, lastSet, OFFSET_WIDTH, offsetBuffer, valueBuffer

Fields inherited from class org.apache.arrow.vector.BaseValueVector

allocator, fieldReader, INITIAL_VALUE_ALLOCATION, MAX_ALLOCATION_SIZE, MAX_ALLOCATION_SIZE_PROPERTY, validityBuffer, valueCount

Constructor Summary

Constructors

Constructor	Description
LargeVarBinaryVector(String name, BufferAllocator allocator)	Instantiate a LargeVarBinaryVector.
LargeVarBinaryVector(String name, FieldType fieldType, BufferAllocator allocator)	Instantiate a LargeVarBinaryVector.
LargeVarBinaryVector(Field field, BufferAllocator allocator)	Instantiate a LargeVarBinaryVector.

Method Summary

Modifier and Type	Method	Description
byte[]	get(int index)	Get the variable length element at specified index as byte array.
void	get(int index, NullableLargeVarBinaryHolder holder)	Get the variable length element at specified index and sets the state in provided holder.
Types.MinorType	getMinorType()	Get minor type for this vector.
byte[]	getObject(int index)	Get the variable length element at specified index as Text.
protected FieldReader	getReaderImpl()	Each vector has a different reader that implements the FieldReader interface.
TransferPair	getTransferPair(String ref, BufferAllocator allocator)	Construct a TransferPair comprising of this and a target vector of the same type.
TransferPair	getTransferPair(Field field, BufferAllocator allocator)	Construct a transfer pair of this vector and another vector of same type.
TransferPair	makeTransferPair(ValueVector to)	Construct a TransferPair with a desired target vector of the same type.
void	read(int index, ReusableBuffer<?> buffer)	Read the value at the given position to the given output buffer.
void	set(int index, LargeVarBinaryHolder holder)	Set the variable length element at the specified index to the data buffer supplied in the holder.
void	set(int index, NullableLargeVarBinaryHolder holder)	Set the variable length element at the specified index to the data buffer supplied in the holder.
void	setSafe(int index, LargeVarBinaryHolder holder)	Same as set(int, LargeVarBinaryHolder) except that it handles the case where index and length of new element are beyond the existing capacity of the vector.
void	setSafe(int index, NullableLargeVarBinaryHolder holder)	Same as set(int, NullableLargeVarBinaryHolder) except that it handles the case where index and length of new element are beyond the existing capacity of the vector.

Methods inherited from class org.apache.arrow.vector.BaseLargeVariableWidthVector

accept, allocateNew, allocateNew, allocateNew, allocateNewSafe, allocateValidityBuffer, clear, close, copyFrom, copyFromSafe, exportCDataBuffers, fillEmpties, fillHoles, get, getBuffers, getBufferSize, getBufferSizeFor, getByteCapacity, getChildrenFromFields, getDataBuffer, getDataBufferAddress, getDataPointer, getDataPointer, getDensity, getEndOffset, getField, getFieldBuffers, getFieldInnerVectors, getLastSet, getName, getNullCount, getOffsetBuffer, getOffsetBufferAddress, getStartOffset, getTransferPair, getTransferPair, getTransferPair, getValidityBuffer, getValidityBufferAddress, getValueCapacity, getValueCount, getValueLength, handleSafe, hashCode, hashCode, initializeChildrenFromFields, isNull, isSafe, isSet, loadFieldBuffers, reAlloc, reallocDataBuffer, reallocValidityAndOffsetBuffers, reset, set, set, set, set, set, set, setBytes, setIndexDefined, setInitialCapacity, setInitialCapacity, setLastSet, setNull, setSafe, setSafe, setSafe, setSafe, setSafe, setValueCount, setValueLengthSafe, sizeOfValueBuffer, sliceAndTransferValidityBuffer, splitAndTransferTo, transferTo, validateScalars, zeroVector

Methods inherited from class org.apache.arrow.vector.BaseValueVector

checkBufRefs, copyValidityBuffer, getAllocator, getReader, getValidityBufferSizeFromCount, iterator, releaseBuffer, splitAndTransferValidityBuffer, toString, transferBuffer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.arrow.vector.FieldVector

exportBuffer, getExportedCDataBufferCount

Methods inherited from interface java.lang.Iterable

forEach, iterator, spliterator

Methods inherited from interface org.apache.arrow.vector.ValueIterableVector

getValueIterable, getValueIterator

Methods inherited from interface org.apache.arrow.vector.ValueVector

accept, allocateNew, allocateNewSafe, clear, close, copyFrom, copyFromSafe, getAllocator, getBuffers, getBufferSize, getBufferSizeFor, getDataBuffer, getField, getName, getNullCount, getOffsetBuffer, getReader, getTransferPair, getTransferPair, getTransferPair, getValidityBuffer, getValueCapacity, getValueCount, hashCode, hashCode, isNull, reAlloc, reset, setInitialCapacity, setValueCount, validate, validateFull

Constructor Details

LargeVarBinaryVector

public LargeVarBinaryVector(String name,
 BufferAllocator allocator)

Instantiate a LargeVarBinaryVector. This doesn't allocate any memory for the data in vector.

Parameters:

name - name of the vector

allocator - allocator for memory management.

LargeVarBinaryVector

public LargeVarBinaryVector(String name,
 FieldType fieldType,
 BufferAllocator allocator)

Instantiate a LargeVarBinaryVector. This doesn't allocate any memory for the data in vector.

Parameters:

name - name of the vector

fieldType - type of Field materialized by this vector

allocator - allocator for memory management.

LargeVarBinaryVector

public LargeVarBinaryVector(Field field,
 BufferAllocator allocator)

Instantiate a LargeVarBinaryVector. This doesn't allocate any memory for the data in vector.

Parameters:

field - field materialized by this vector

allocator - allocator for memory management.

Method Details

getReaderImpl

protected FieldReader getReaderImpl()

Description copied from class: BaseValueVector

Each vector has a different reader that implements the FieldReader interface. Overridden methods must make sure to return the correct concrete reader implementation.

Specified by:

getReaderImpl in class BaseValueVector

Returns:

Returns a lambda that initializes a reader when called.

getMinorType

public Types.MinorType getMinorType()

Get minor type for this vector. The vector holds values belonging to a particular type.

Specified by:

getMinorType in interface ValueVector

Returns:

Types.MinorType

get

public byte[] get(int index)

Get the variable length element at specified index as byte array.

Specified by:

get in interface VariableWidthFieldVector

Parameters:

index - position of element to get

Returns:

array of bytes for non-null element, null otherwise

read

public void read(int index,
 ReusableBuffer<?> buffer)

Read the value at the given position to the given output buffer. The caller is responsible for checking for nullity first.

Specified by:

read in interface VariableWidthFieldVector

Parameters:

index - position of element.

buffer - the buffer to write into.

getObject

public byte[] getObject(int index)

Get the variable length element at specified index as Text.

Specified by:

getObject in interface ValueVector

Parameters:

index - position of element to get

Returns:

byte array for non-null element, null otherwise

get

public void get(int index,
 NullableLargeVarBinaryHolder holder)

Get the variable length element at specified index and sets the state in provided holder.

Parameters:

index - position of element to get

holder - data holder to be populated by this function

set

public void set(int index,
 LargeVarBinaryHolder holder)

Set the variable length element at the specified index to the data buffer supplied in the holder.

Parameters:

index - position of the element to set

holder - holder that carries data buffer.

setSafe

public void setSafe(int index,
 LargeVarBinaryHolder holder)

Same as set(int, LargeVarBinaryHolder) except that it handles the case where index and length of new element are beyond the existing capacity of the vector.

Parameters:

index - position of the element to set

holder - holder that carries data buffer.

set

public void set(int index,
 NullableLargeVarBinaryHolder holder)

Set the variable length element at the specified index to the data buffer supplied in the holder.

Parameters:

index - position of the element to set

holder - holder that carries data buffer.

setSafe

public void setSafe(int index,
 NullableLargeVarBinaryHolder holder)

Same as set(int, NullableLargeVarBinaryHolder) except that it handles the case where index and length of new element are beyond the existing capacity of the vector.

Parameters:

index - position of the element to set

holder - holder that carries data buffer.

getTransferPair

public TransferPair getTransferPair(String ref,
 BufferAllocator allocator)

Construct a TransferPair comprising of this and a target vector of the same type.

Specified by:

getTransferPair in interface ValueVector

Specified by:

getTransferPair in class BaseLargeVariableWidthVector

Parameters:

ref - name of the target vector

allocator - allocator for the target vector

Returns:

TransferPair

getTransferPair

public TransferPair getTransferPair(Field field,
 BufferAllocator allocator)

Description copied from class: BaseLargeVariableWidthVector

Construct a transfer pair of this vector and another vector of same type.

Specified by:

getTransferPair in interface ValueVector

Specified by:

getTransferPair in class BaseLargeVariableWidthVector

Parameters:

field - The field materialized by this vector

allocator - allocator for the target vector

Returns:

TransferPair

makeTransferPair

public TransferPair makeTransferPair(ValueVector to)

Construct a TransferPair with a desired target vector of the same type.

Specified by:

makeTransferPair in interface ValueVector

Parameters:

to - target vector

Returns:

TransferPair