org.apache.arrow.memory

Class BufferLedger

java.lang.Object

org.apache.arrow.memory.BufferLedger

All Implemented Interfaces:

ReferenceManager, ValueWithKeyIncluded<BufferAllocator>

public class BufferLedger
extends Object
implements ValueWithKeyIncluded<BufferAllocator>, ReferenceManager

The reference manager that binds an AllocationManager to BufferAllocator and a set of ArrowBuf. The set of ArrowBufs managed by this reference manager share a common fate (same reference count).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	BufferLedger.TransferResult The outcome of a Transfer.

Field Summary

Fields inherited from interface org.apache.arrow.memory.ReferenceManager

NO_OP, NO_OP_ERROR_MESSAGE

Method Summary

Modifier and Type	Method and Description
ArrowBuf	deriveBuffer(ArrowBuf sourceBuffer, long index, long length) Derive a new ArrowBuf from a given source ArrowBuf.
long	getAccountedSize() How much memory is accounted for by this ledger.
AllocationManager	getAllocationManager() Get the AllocationManager used by this BufferLedger.
BufferAllocator	getAllocator() Get the buffer allocator associated with this reference manager.
BufferAllocator	getKey()
int	getRefCount() Get this ledger's reference count.
long	getSize() Total size (in bytes) of memory underlying this reference manager.
boolean	release() Decrement the ledger's reference count by 1 for the associated underlying memory chunk.
boolean	release(int decrement) Decrement the ledger's reference count for the associated underlying memory chunk.
void	retain() Increment the ledger's reference count for associated underlying memory chunk by 1.
ArrowBuf	retain(ArrowBuf srcBuffer, BufferAllocator target) Create a new ArrowBuf that is associated with an alternative allocator for the purposes of memory ownership and accounting.
void	retain(int increment) Increment the ledger's reference count for associated underlying memory chunk by the given amount.
BufferLedger.TransferResult	transferOwnership(ArrowBuf srcBuffer, BufferAllocator target) Transfer the memory accounting ownership of this ArrowBuf to another allocator.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getKey

public BufferAllocator getKey()

Specified by:

getKey in interface ValueWithKeyIncluded<BufferAllocator>

getAllocator

public BufferAllocator getAllocator()

Get the buffer allocator associated with this reference manager.

Specified by:

getAllocator in interface ReferenceManager

Returns:

buffer allocator

getRefCount

public int getRefCount()

Get this ledger's reference count.

Specified by:

getRefCount in interface ReferenceManager

Returns:

reference count

release

public boolean release()

Decrement the ledger's reference count by 1 for the associated underlying memory chunk. If the reference count drops to 0, it implies that no ArrowBufs managed by this reference manager need access to the memory chunk. In that case, the ledger should inform the allocation manager about releasing its ownership for the chunk. Whether or not the memory chunk will be released is something that AllocationManager will decide since tracks the usage of memory chunk across multiple reference managers and allocators.

Specified by:

release in interface ReferenceManager

Returns:

true if the new ref count has dropped to 0, false otherwise

release

public boolean release(int decrement)

Decrement the ledger's reference count for the associated underlying memory chunk. If the reference count drops to 0, it implies that no ArrowBufs managed by this reference manager need access to the memory chunk. In that case, the ledger should inform the allocation manager about releasing its ownership for the chunk. Whether or not the memory chunk will be released is something that AllocationManager will decide since tracks the usage of memory chunk across multiple reference managers and allocators.

Specified by:

release in interface ReferenceManager

Parameters:

decrement - amount to decrease the reference count by

Returns:

true if the new ref count has dropped to 0, false otherwise

retain

public void retain()

Increment the ledger's reference count for associated underlying memory chunk by 1.

Specified by:

retain in interface ReferenceManager

retain

public void retain(int increment)

Increment the ledger's reference count for associated underlying memory chunk by the given amount.

Specified by:

retain in interface ReferenceManager

Parameters:

increment - amount to increase the reference count by

deriveBuffer

public ArrowBuf deriveBuffer(ArrowBuf sourceBuffer,
                             long index,
                             long length)

Derive a new ArrowBuf from a given source ArrowBuf. The new derived ArrowBuf will share the same reference count as rest of the ArrowBufs associated with this ledger. This operation is typically used for slicing -- creating new ArrowBufs from a compound ArrowBuf starting at a particular index in the underlying memory and having access to a particular length (in bytes) of data in memory chunk.

This method is also used as a helper for transferring ownership and retain to target allocator.

Specified by:

deriveBuffer in interface ReferenceManager

Parameters:

sourceBuffer - source ArrowBuf

index - index (relative to source ArrowBuf) new ArrowBuf should be derived from

length - length (bytes) of data in underlying memory that derived buffer will have access to in underlying memory

Returns:

derived buffer

retain

public ArrowBuf retain(ArrowBuf srcBuffer,
                       BufferAllocator target)

Create a new ArrowBuf that is associated with an alternative allocator for the purposes of memory ownership and accounting. This has no impact on the reference counting for the current ArrowBuf except in the situation where the passed in Allocator is the same as the current buffer.

This operation has no impact on the reference count of this ArrowBuf. The newly created ArrowBuf with either have a reference count of 1 (in the case that this is the first time this memory is being associated with the target allocator or in other words allocation manager currently doesn't hold a mapping for the target allocator) or the current value of the reference count for the target allocator-reference manager combination + 1 in the case that the provided allocator already had an association to this underlying memory.

Specified by:

retain in interface ReferenceManager

Parameters:

srcBuffer - source ArrowBuf

target - The target allocator to create an association with.

Returns:

A new ArrowBuf which shares the same underlying memory as the provided ArrowBuf.

transferOwnership

public BufferLedger.TransferResult transferOwnership(ArrowBuf srcBuffer,
                                                     BufferAllocator target)

Transfer the memory accounting ownership of this ArrowBuf to another allocator. This will generate a new ArrowBuf that carries an association with the underlying memory of this ArrowBuf. If this ArrowBuf is connected to the owning BufferLedger of this memory, that memory ownership/accounting will be transferred to the target allocator. If this ArrowBuf does not currently own the memory underlying it (and is only associated with it), this does not transfer any ownership to the newly created ArrowBuf.

This operation has no impact on the reference count of this ArrowBuf. The newly created ArrowBuf with either have a reference count of 1 (in the case that this is the first time this memory is being associated with the new allocator) or the current value of the reference count for the other AllocationManager/BufferLedger combination + 1 in the case that the provided allocator already had an association to this underlying memory.

Transfers will always succeed, even if that puts the other allocator into an overlimit situation. This is possible due to the fact that the original owning allocator may have allocated this memory out of a local reservation whereas the target allocator may need to allocate new memory from a parent or RootAllocator. This operation is done n a mostly-lockless but consistent manner. As such, the overlimit==true situation could occur slightly prematurely to an actual overlimit==true condition. This is simply conservative behavior which means we may return overlimit slightly sooner than is necessary.

Specified by:

transferOwnership in interface ReferenceManager

Parameters:

target - The allocator to transfer ownership to.

srcBuffer - source ArrowBuf

Returns:

A new transfer result with the impact of the transfer (whether it was overlimit) as well as the newly created ArrowBuf.

getSize

public long getSize()

Total size (in bytes) of memory underlying this reference manager.

Specified by:

getSize in interface ReferenceManager

Returns:

Size (in bytes) of the memory chunk

getAccountedSize

public long getAccountedSize()

How much memory is accounted for by this ledger. This is either getSize() if this is the owning ledger for the memory or zero in the case that this is not the owning ledger associated with this memory.

Specified by:

getAccountedSize in interface ReferenceManager

Returns:

Amount of accounted(owned) memory associated with this ledger.

getAllocationManager

public AllocationManager getAllocationManager()

Get the AllocationManager used by this BufferLedger.

Returns:

The AllocationManager used by this BufferLedger.