Package org.apache.arrow.memory

Class BufferLedger

java.lang.Object
org.apache.arrow.memory.BufferLedger
All Implemented Interfaces:
ReferenceManager, ValueWithKeyIncluded<BufferAllocator>
public class BufferLedger extends Object implements ValueWithKeyIncluded<BufferAllocator>, ReferenceManager
The reference manager that binds an AllocationManager to BufferAllocator and a set of ArrowBuf. The set of ArrowBufs managed by this reference manager share a common fate (same reference count).

  • Method Details

    • getKey

      public BufferAllocator getKey()
      Specified by:
      getKey in interface ValueWithKeyIncluded<BufferAllocator>

    • getAllocator

      public BufferAllocator getAllocator()
      Get the buffer allocator associated with this reference manager.
      Specified by:
      getAllocator in interface ReferenceManager
      Returns:
      buffer allocator

    • getRefCount

      public int getRefCount()
      Get this ledger's reference count.
      Specified by:
      getRefCount in interface ReferenceManager
      Returns:
      reference count

    • release

      public boolean release()
      Decrement the ledger's reference count by 1 for the associated underlying memory chunk. If the reference count drops to 0, it implies that no ArrowBufs managed by this reference manager need access to the memory chunk. In that case, the ledger should inform the allocation manager about releasing its ownership for the chunk. Whether or not the memory chunk will be released is something that AllocationManager will decide since tracks the usage of memory chunk across multiple reference managers and allocators.
      Specified by:
      release in interface ReferenceManager
      Returns:
      true if the new ref count has dropped to 0, false otherwise

    • release

      public boolean release(int decrement)
      Decrement the ledger's reference count for the associated underlying memory chunk. If the reference count drops to 0, it implies that no ArrowBufs managed by this reference manager need access to the memory chunk. In that case, the ledger should inform the allocation manager about releasing its ownership for the chunk. Whether or not the memory chunk will be released is something that AllocationManager will decide since tracks the usage of memory chunk across multiple reference managers and allocators.
      Specified by:
      release in interface ReferenceManager
      Parameters:
      decrement - amount to decrease the reference count by
      Returns:
      true if the new ref count has dropped to 0, false otherwise

    • retain

      public void retain()
      Increment the ledger's reference count for associated underlying memory chunk by 1.
      Specified by:
      retain in interface ReferenceManager

    • retain

      public void retain(int increment)
      Increment the ledger's reference count for associated underlying memory chunk by the given amount.
      Specified by:
      retain in interface ReferenceManager
      Parameters:
      increment - amount to increase the reference count by

    • deriveBuffer

      public ArrowBuf deriveBuffer(ArrowBuf sourceBuffer, long index, long length)
      Derive a new ArrowBuf from a given source ArrowBuf. The new derived ArrowBuf will share the same reference count as rest of the ArrowBufs associated with this ledger. This operation is typically used for slicing -- creating new ArrowBufs from a compound ArrowBuf starting at a particular index in the underlying memory and having access to a particular length (in bytes) of data in memory chunk.

      This method is also used as a helper for transferring ownership and retain to target allocator.

      Specified by:
      deriveBuffer in interface ReferenceManager
      Parameters:
      sourceBuffer - source ArrowBuf
      index - index (relative to source ArrowBuf) new ArrowBuf should be derived from
      length - length (bytes) of data in underlying memory that derived buffer will have access to in underlying memory
      Returns:
      derived buffer

    • retain

      public ArrowBuf retain(ArrowBuf srcBuffer, BufferAllocator target)
      Create a new ArrowBuf that is associated with an alternative allocator for the purposes of memory ownership and accounting. This has no impact on the reference counting for the current ArrowBuf except in the situation where the passed in Allocator is the same as the current buffer.

      This operation has no impact on the reference count of this ArrowBuf. The newly created ArrowBuf with either have a reference count of 1 (in the case that this is the first time this memory is being associated with the target allocator or in other words allocation manager currently doesn't hold a mapping for the target allocator) or the current value of the reference count for the target allocator-reference manager combination + 1 in the case that the provided allocator already had an association to this underlying memory.

      Specified by:
      retain in interface ReferenceManager
      Parameters:
      srcBuffer - source ArrowBuf
      target - The target allocator to create an association with.
      Returns:
      A new ArrowBuf which shares the same underlying memory as the provided ArrowBuf.

    • transferOwnership

      public BufferLedger.TransferResult transferOwnership(ArrowBuf srcBuffer, BufferAllocator target)
      Transfer the memory accounting ownership of this ArrowBuf to another allocator. This will generate a new ArrowBuf that carries an association with the underlying memory of this ArrowBuf. If this ArrowBuf is connected to the owning BufferLedger of this memory, that memory ownership/accounting will be transferred to the target allocator. If this ArrowBuf does not currently own the memory underlying it (and is only associated with it), this does not transfer any ownership to the newly created ArrowBuf.

      This operation has no impact on the reference count of this ArrowBuf. The newly created ArrowBuf with either have a reference count of 1 (in the case that this is the first time this memory is being associated with the new allocator) or the current value of the reference count for the other AllocationManager/BufferLedger combination + 1 in the case that the provided allocator already had an association to this underlying memory.

      Transfers will always succeed, even if that puts the other allocator into an overlimit situation. This is possible due to the fact that the original owning allocator may have allocated this memory out of a local reservation whereas the target allocator may need to allocate new memory from a parent or RootAllocator. This operation is done n a mostly-lockless but consistent manner. As such, the overlimit==true situation could occur slightly prematurely to an actual overlimit==true condition. This is simply conservative behavior which means we may return overlimit slightly sooner than is necessary.

      Specified by:
      transferOwnership in interface ReferenceManager
      Parameters:
      target - The allocator to transfer ownership to.
      srcBuffer - source ArrowBuf
      Returns:
      A new transfer result with the impact of the transfer (whether it was overlimit) as well as the newly created ArrowBuf.

    • getSize

      public long getSize()
      Total size (in bytes) of memory underlying this reference manager.
      Specified by:
      getSize in interface ReferenceManager
      Returns:
      Size (in bytes) of the memory chunk

    • getAccountedSize

      public long getAccountedSize()
      How much memory is accounted for by this ledger. This is either getSize() if this is the owning ledger for the memory or zero in the case that this is not the owning ledger associated with this memory.
      Specified by:
      getAccountedSize in interface ReferenceManager
      Returns:
      Amount of accounted(owned) memory associated with this ledger.

    • getAllocationManager

      public AllocationManager getAllocationManager()
      Get the AllocationManager used by this BufferLedger.
      Returns:
      The AllocationManager used by this BufferLedger.