Module org.apache.arrow.vector
Package org.apache.arrow.vector.complex
Class BaseRepeatedValueViewVector
java.lang.Object
org.apache.arrow.vector.BaseValueVector
org.apache.arrow.vector.complex.BaseRepeatedValueViewVector
- All Implemented Interfaces:
Closeable
,AutoCloseable
,Iterable<ValueVector>
,BaseListVector
,RepeatedValueVector
,DensityAwareVector
,FieldVector
,ValueVector
- Direct Known Subclasses:
ListViewVector
public abstract class BaseRepeatedValueViewVector
extends BaseValueVector
implements RepeatedValueVector, BaseListVector
-
Field Summary
Modifier and TypeFieldDescriptionstatic final String
static final FieldVector
protected String
static final byte
protected long
protected ArrowBuf
protected final CallBack
static final byte
protected long
protected ArrowBuf
protected int
protected FieldVector
Fields inherited from class org.apache.arrow.vector.BaseValueVector
allocator, fieldReader, INITIAL_VALUE_ALLOCATION, MAX_ALLOCATION_SIZE, MAX_ALLOCATION_SIZE_PROPERTY
Fields inherited from interface org.apache.arrow.vector.complex.RepeatedValueVector
DEFAULT_REPEAT_PER_RECORD
-
Constructor Summary
ModifierConstructorDescriptionprotected
BaseRepeatedValueViewVector
(String name, BufferAllocator allocator, FieldVector vector, CallBack callBack) protected
BaseRepeatedValueViewVector
(String name, BufferAllocator allocator, CallBack callBack) -
Method Summary
Modifier and TypeMethodDescription<T extends ValueVector>
AddOrGetResult<T>addOrGetVector
(FieldType fieldType) Initialize the data vector (and execute callback) if it hasn't already been done, returns the data vector.protected ArrowBuf
allocateBuffers
(long size) boolean
Allocates new buffers.void
clear()
Release any owned ArrowBuf and reset the ValueVector to the initial state.ArrowBuf[]
getBuffers
(boolean clear) Return the underlying buffers associated with this vector.int
Get the number of bytes used by this vector.int
getBufferSizeFor
(int valueCount) Returns the number of bytes that is used by this vector if it holds the given number of values.Get the data vector.protected int
Get the end of the child vector via the maximum view length.protected int
getMaxViewEndChildVectorByIndex
(int index) Get the end of the child vector via the maximum view length of the child vector by index.getName()
Gets the name of the vector.protected int
Deprecated.protected int
int
Returns the maximum number of values that can be stored in this vector instance.int
Gets the number of values.abstract boolean
isEmpty
(int index) iterator()
void
reAlloc()
Allocate new buffer with double capacity, and copy data into the new buffer.protected void
protected void
void
reset()
Reset the ValueVector to the initial state without releasing any owned ArrowBuf.void
setInitialCapacity
(int numRecords) Set the initial record capacity.void
setInitialCapacity
(int numRecords, double density) Set value with density.void
setInitialTotalCapacity
(int numRecords, int totalNumberOfElements) Specialized version of setInitialTotalCapacity() for ListViewVector.void
setValueCount
(int valueCount) Set number of values in the vector.int
startNewValue
(int index) Start a new value at the given index.Methods inherited from class org.apache.arrow.vector.BaseValueVector
checkBufRefs, close, copyFrom, copyFromSafe, getAllocator, getReader, getReaderImpl, getTransferPair, getValidityBufferSizeFromCount, releaseBuffer, toString, transferBuffer
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface org.apache.arrow.vector.complex.BaseListVector
getElementEndIndex, getElementStartIndex
Methods inherited from interface org.apache.arrow.vector.FieldVector
exportBuffer, exportCDataBuffers, getChildrenFromFields, getDataBufferAddress, getExportedCDataBufferCount, getFieldBuffers, getFieldInnerVectors, getOffsetBufferAddress, getValidityBufferAddress, initializeChildrenFromFields, loadFieldBuffers, setNull
Methods inherited from interface java.lang.Iterable
forEach, spliterator
Methods inherited from interface org.apache.arrow.vector.ValueVector
accept, allocateNew, close, copyFrom, copyFromSafe, getAllocator, getDataBuffer, getField, getMinorType, getNullCount, getObject, getOffsetBuffer, getReader, getTransferPair, getTransferPair, getTransferPair, getTransferPair, getTransferPair, getValidityBuffer, hashCode, hashCode, isNull, makeTransferPair, validate, validateFull
-
Field Details
-
DEFAULT_DATA_VECTOR
-
DATA_VECTOR_NAME
- See Also:
-
OFFSET_WIDTH
public static final byte OFFSET_WIDTH- See Also:
-
SIZE_WIDTH
public static final byte SIZE_WIDTH- See Also:
-
offsetBuffer
-
sizeBuffer
-
vector
-
repeatedCallBack
-
valueCount
protected int valueCount -
offsetAllocationSizeInBytes
protected long offsetAllocationSizeInBytes -
sizeAllocationSizeInBytes
protected long sizeAllocationSizeInBytes -
defaultDataVectorName
-
-
Constructor Details
-
BaseRepeatedValueViewVector
-
BaseRepeatedValueViewVector
protected BaseRepeatedValueViewVector(String name, BufferAllocator allocator, FieldVector vector, CallBack callBack)
-
-
Method Details
-
getName
Description copied from interface:ValueVector
Gets the name of the vector.- Specified by:
getName
in interfaceValueVector
- Specified by:
getName
in classBaseValueVector
- Returns:
- the name of the vector.
-
allocateNewSafe
public boolean allocateNewSafe()Description copied from interface:ValueVector
Allocates new buffers. ValueVector implements logic to determine how much to allocate.- Specified by:
allocateNewSafe
in interfaceValueVector
- Returns:
- Returns true if allocation was successful.
-
allocateBuffers
-
reAlloc
public void reAlloc()Description copied from interface:ValueVector
Allocate new buffer with double capacity, and copy data into the new buffer. Replace vector's buffer with new buffer, and release old one- Specified by:
reAlloc
in interfaceValueVector
-
reallocateBuffers
protected void reallocateBuffers() -
getDataVector
Description copied from interface:RepeatedValueVector
Get the data vector.- Specified by:
getDataVector
in interfaceRepeatedValueVector
- Returns:
- the underlying data vector or null if none exists.
-
setInitialCapacity
public void setInitialCapacity(int numRecords) Description copied from interface:ValueVector
Set the initial record capacity.- Specified by:
setInitialCapacity
in interfaceValueVector
- Parameters:
numRecords
- the initial record capacity.
-
setInitialCapacity
public void setInitialCapacity(int numRecords, double density) Description copied from interface:DensityAwareVector
Set value with density.- Specified by:
setInitialCapacity
in interfaceDensityAwareVector
- Parameters:
numRecords
- the number of values in this vectordensity
- the density of the vector
-
setInitialTotalCapacity
public void setInitialTotalCapacity(int numRecords, int totalNumberOfElements) Specialized version of setInitialTotalCapacity() for ListViewVector. This is used by some callers when they want to explicitly control and be conservative about memory allocated for inner data vector. This is very useful when we are working with memory constraints for a query and have a fixed amount of memory reserved for the record batch. In such cases, we are likely to face OOM or related problems when we reserve memory for a record batch with value count x and do setInitialCapacity(x) such that each vector allocates only what is necessary and not the default amount, but the multiplier forces the memory requirement to go beyond what was needed.- Parameters:
numRecords
- value counttotalNumberOfElements
- the total number of elements to allow for in this vector across all records.
-
getValueCapacity
public int getValueCapacity()Description copied from interface:ValueVector
Returns the maximum number of values that can be stored in this vector instance.- Specified by:
getValueCapacity
in interfaceValueVector
- Returns:
- the maximum number of values that can be stored in this vector instance.
-
getOffsetBufferValueCapacity
protected int getOffsetBufferValueCapacity() -
getSizeBufferValueCapacity
protected int getSizeBufferValueCapacity() -
getBufferSize
public int getBufferSize()Description copied from interface:ValueVector
Get the number of bytes used by this vector.- Specified by:
getBufferSize
in interfaceValueVector
- Returns:
- the number of bytes that is used by this vector instance.
-
getBufferSizeFor
public int getBufferSizeFor(int valueCount) Description copied from interface:ValueVector
Returns the number of bytes that is used by this vector if it holds the given number of values. The result will be the same as if setValueCount() were called, followed by calling getBufferSize(), but without any of the closing side-effects that setValueCount() implies wrt finishing off the population of a vector. Some operations might wish to use this to determine how much memory has been used by a vector so far, even though it is not finished being populated.- Specified by:
getBufferSizeFor
in interfaceValueVector
- Parameters:
valueCount
- the number of values to assume this vector contains- Returns:
- the buffer size if this vector is holding valueCount values
-
iterator
- Specified by:
iterator
in interfaceIterable<ValueVector>
- Overrides:
iterator
in classBaseValueVector
-
clear
public void clear()Description copied from interface:ValueVector
Release any owned ArrowBuf and reset the ValueVector to the initial state. If the vector has any child vectors, they will also be cleared.- Specified by:
clear
in interfaceValueVector
- Overrides:
clear
in classBaseValueVector
-
reset
public void reset()Description copied from interface:ValueVector
Reset the ValueVector to the initial state without releasing any owned ArrowBuf. Buffer capacities will remain unchanged and any previous data will be zeroed out. This includes buffers for data, validity, offset, etc. If the vector has any child vectors, they will also be reset.- Specified by:
reset
in interfaceValueVector
-
getBuffers
Description copied from interface:ValueVector
Return the underlying buffers associated with this vector. Note that this doesn't impact the reference counts for this buffer so it only should be used for in-context access. Also note that this buffer changes regularly thus external classes shouldn't hold a reference to it (unless they change it).- Specified by:
getBuffers
in interfaceValueVector
- Parameters:
clear
- Whether to clear vector before returning; the buffers will still be refcounted; but the returned array will be the only reference to them- Returns:
- The underlying
buffers
that is used by this vector instance.
-
getValueCount
public int getValueCount()Description copied from interface:ValueVector
Gets the number of values.- Specified by:
getValueCount
in interfaceValueVector
- Returns:
- number of values in the vector
-
setValueCount
public void setValueCount(int valueCount) Description copied from interface:ValueVector
Set number of values in the vector.- Specified by:
setValueCount
in interfaceValueVector
-
getMaxViewEndChildVector
protected int getMaxViewEndChildVector()Get the end of the child vector via the maximum view length. This method deduces the length by considering the condition i.e., argmax_i(offsets[i] + size[i]).- Returns:
- the end of the child vector.
-
getMaxViewEndChildVectorByIndex
protected int getMaxViewEndChildVectorByIndex(int index) Get the end of the child vector via the maximum view length of the child vector by index.- Returns:
- the end of the child vector by index
-
addOrGetVector
Initialize the data vector (and execute callback) if it hasn't already been done, returns the data vector. -
replaceDataVector
-
isEmpty
public abstract boolean isEmpty(int index) -
startNewValue
public int startNewValue(int index) Start a new value at the given index.- Parameters:
index
- the index to start the new value at- Returns:
- the offset in the data vector where the new value starts
-
getOffsetVector
Deprecated.Description copied from interface:RepeatedValueVector
Get the offset vector.- Specified by:
getOffsetVector
in interfaceRepeatedValueVector
- Returns:
- the underlying offset vector or null if none exists.
-