Module org.apache.arrow.memory.core
Package org.apache.arrow.memory

Class RootAllocator

java.lang.Object
org.apache.arrow.memory.RootAllocator
All Implemented Interfaces:
AutoCloseable, BufferAllocator
public class RootAllocator extends Object
A root allocator for using direct memory for Arrow Vectors/Arrays. Supports creating a tree of descendant child allocators to facilitate better instrumentation of memory allocations.

  • Field Details

    • DEBUG_ALLOCATOR

      public static final String DEBUG_ALLOCATOR
      See Also:

    • DEBUG_LOG_LENGTH

      public static final int DEBUG_LOG_LENGTH
      See Also:

    • DEBUG

      public static final boolean DEBUG

    • DEFAULT_CONFIG

      public static final org.apache.arrow.memory.BaseAllocator.Config DEFAULT_CONFIG

    • parent

      protected final @Nullable org.apache.arrow.memory.Accountant parent
      The parent allocator.

    • reservation

      protected final long reservation
      The amount of memory reserved for this allocator. Releases below this amount of memory will not be returned to the parent Accountant until this Accountant is closed.

  • Constructor Details

    • RootAllocator

      public RootAllocator()

    • RootAllocator

      public RootAllocator(long limit)

    • RootAllocator

      public RootAllocator(AllocationListener listener, long limit)

    • RootAllocator

      public RootAllocator(AllocationListener listener, long limit, RoundingPolicy roundingPolicy)
      Constructor.
      Parameters:
      listener - the allocation listener
      limit - max allocation size in bytes
      roundingPolicy - the policy for rounding the buffer size

    • RootAllocator

      public RootAllocator(org.apache.arrow.memory.BaseAllocator.Config config)

  • Method Details

    • verify

      public void verify()
      Verify the accounting state of the allocation system.

    • getListener

      public AllocationListener getListener()
      Description copied from interface: BufferAllocator
      Returns the allocation listener used by this allocator.
      Specified by:
      getListener in interface BufferAllocator
      Returns:
      the AllocationListener instance. Or AllocationListener.NOOP by default if no listener is configured when this allocator was created.

    • getParentAllocator

      public @Nullable org.apache.arrow.memory.BaseAllocator getParentAllocator()
      Description copied from interface: BufferAllocator
      Returns the parent allocator.
      Specified by:
      getParentAllocator in interface BufferAllocator
      Returns:
      parent allocator

    • getChildAllocators

      public Collection<BufferAllocator> getChildAllocators()
      Description copied from interface: BufferAllocator
      Returns the set of child allocators.
      Specified by:
      getChildAllocators in interface BufferAllocator
      Returns:
      set of child allocators

    • isDebug

      public static boolean isDebug()

    • assertOpen

      public void assertOpen()
      Description copied from interface: BufferAllocator
      Asserts (using java assertions) that the provided allocator is currently open. If assertions are disabled, this is a no-op.
      Specified by:
      assertOpen in interface BufferAllocator

    • getName

      public String getName()
      Return the name of the accountant.
      Specified by:
      getName in interface BufferAllocator
      Returns:
      name of accountant

    • getEmpty

      public ArrowBuf getEmpty()
      Description copied from interface: BufferAllocator
      Get a reference to the empty buffer associated with this allocator. Empty buffers are special because we don't worry about them leaking or managing reference counts on them since they don't actually point to any memory.
      Specified by:
      getEmpty in interface BufferAllocator
      Returns:
      the empty buffer

    • wrapForeignAllocation

      public ArrowBuf wrapForeignAllocation(ForeignAllocation allocation)
      Description copied from interface: BufferAllocator
      EXPERIMENTAL: Wrap an allocation created outside this BufferAllocator.

      This is useful to integrate allocations from native code into the same memory management framework as Java-allocated buffers, presenting users a consistent API. The created buffer will be tracked by this allocator and can be transferred like Java-allocated buffers.

      The underlying allocation will be closed when all references to the buffer are released. If this method throws, the underlying allocation will also be closed.

      Specified by:
      wrapForeignAllocation in interface BufferAllocator
      Parameters:
      allocation - The underlying allocation.

    • buffer

      public ArrowBuf buffer(long initialRequestSize)
      Description copied from interface: BufferAllocator
      Allocate a new or reused buffer of the provided size. The buffer may be larger than the requested size for rounding purposes (e.g. to a power of two), and the buffer's capacity will reflect the actual allocated size. Use ArrowBuf.capacity(long) to set the capacity to the requested size if needed.
      Specified by:
      buffer in interface BufferAllocator
      Parameters:
      initialRequestSize - The size in bytes.
      Returns:
      a new ArrowBuf, or null if the request can't be satisfied

    • buffer

      public ArrowBuf buffer(long initialRequestSize, @Nullable BufferManager manager)
      Description copied from interface: BufferAllocator
      Allocate a new or reused buffer of the provided size. The buffer may be larger than the requested size for rounding purposes (e.g. to a power of two), and the buffer's capacity will reflect the actual allocated size. Use ArrowBuf.capacity(long) to set the capacity to the requested size if needed.
      Specified by:
      buffer in interface BufferAllocator
      Parameters:
      initialRequestSize - The size in bytes.
      manager - A buffer manager to manage reallocation.
      Returns:
      a new ArrowBuf, or null if the request can't be satisfied

    • getRoot

      public BufferAllocator getRoot()
      Description copied from interface: BufferAllocator
      Get the root allocator of this allocator. If this allocator is already a root, return this directly.
      Specified by:
      getRoot in interface BufferAllocator
      Returns:
      The root allocator

    • newChildAllocator

      public BufferAllocator newChildAllocator(String name, long initReservation, long maxAllocation)
      Description copied from interface: BufferAllocator
      Create a new child allocator.
      Specified by:
      newChildAllocator in interface BufferAllocator
      Parameters:
      name - the name of the allocator.
      initReservation - the initial space reservation (obtained from this allocator)
      maxAllocation - maximum amount of space the new allocator can allocate
      Returns:
      the new allocator, or null if it can't be created

    • newChildAllocator

      public BufferAllocator newChildAllocator(String name, AllocationListener listener, long initReservation, long maxAllocation)
      Description copied from interface: BufferAllocator
      Create a new child allocator.
      Specified by:
      newChildAllocator in interface BufferAllocator
      Parameters:
      name - the name of the allocator.
      listener - allocation listener for the newly created child
      initReservation - the initial space reservation (obtained from this allocator)
      maxAllocation - maximum amount of space the new allocator can allocate
      Returns:
      the new allocator, or null if it can't be created

    • newReservation

      public AllocationReservation newReservation()
      Description copied from interface: BufferAllocator
      Create an allocation reservation. A reservation is a way of building up a request for a buffer whose size is not known in advance. See
      Specified by:
      newReservation in interface BufferAllocator
      Returns:
      the newly created reservation
      See Also:

    • close

      public void close()
      Close this Accountant. This will release any reservation bytes back to a parent Accountant.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface BufferAllocator

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • toVerboseString

      public String toVerboseString()
      Provide a verbose string of the current allocator state. Includes the state of all child allocators, along with historical logs of each object and including stacktraces.
      Specified by:
      toVerboseString in interface BufferAllocator
      Returns:
      A Verbose string of current allocator state.

    • defaultConfig

      public static org.apache.arrow.memory.BaseAllocator.Config defaultConfig()
      Returns a default BaseAllocator.Config instance.
      See Also:
      • ImmutableConfig.Builder

    • configBuilder

      public static org.apache.arrow.memory.ImmutableConfig.Builder configBuilder()
      Returns a builder class for configuring BaseAllocator's options.

    • getRoundingPolicy

      public RoundingPolicy getRoundingPolicy()
      Description copied from interface: BufferAllocator
      Gets the rounding policy of the allocator.
      Specified by:
      getRoundingPolicy in interface BufferAllocator

    • forceAllocate

      public boolean forceAllocate(long size)
      Increase the accounting. Returns whether the allocation fit within limits.
      Parameters:
      size - to increase
      Returns:
      Whether the allocation fit within limits.

    • releaseBytes

      public void releaseBytes(long size)

    • isOverLimit

      public boolean isOverLimit()

    • getLimit

      public long getLimit()
      Return the current limit of this Accountant.
      Returns:
      Limit in bytes.

    • getInitReservation

      public long getInitReservation()
      Return the initial reservation.
      Returns:
      reservation in bytes.

    • setLimit

      public void setLimit(long newLimit)
      Set the maximum amount of memory that can be allocated in the this Accountant before failing an allocation.
      Parameters:
      newLimit - The limit in bytes.

    • getAllocatedMemory

      public long getAllocatedMemory()
      Return the current amount of allocated memory that this Accountant is managing accounting for. Note this does not include reservation memory that hasn't been allocated.
      Returns:
      Currently allocate memory in bytes.

    • getPeakMemoryAllocation

      public long getPeakMemoryAllocation()
      The peak memory allocated by this Accountant.
      Returns:
      The peak allocated memory in bytes.

    • getHeadroom

      public long getHeadroom()