java.lang.Object
org.apache.arrow.memory.RootAllocator
- All Implemented Interfaces:
AutoCloseable
,BufferAllocator
A root allocator for using direct memory for Arrow Vectors/Arrays. Supports creating a tree of
descendant child allocators to facilitate better instrumentation of memory allocations.
-
Field Summary
Modifier and TypeFieldDescriptionstatic final boolean
static final String
static final int
static final org.apache.arrow.memory.BaseAllocator.Config
protected final @Nullable org.apache.arrow.memory.Accountant
The parent allocator.protected final long
The amount of memory reserved for this allocator. -
Constructor Summary
ConstructorDescriptionRootAllocator
(long limit) RootAllocator
(AllocationListener listener, long limit) RootAllocator
(AllocationListener listener, long limit, RoundingPolicy roundingPolicy) Constructor.RootAllocator
(org.apache.arrow.memory.BaseAllocator.Config config) -
Method Summary
Modifier and TypeMethodDescriptionvoid
Asserts (using java assertions) that the provided allocator is currently open.buffer
(long initialRequestSize) Allocate a new or reused buffer of the provided size.buffer
(long initialRequestSize, @Nullable BufferManager manager) Allocate a new or reused buffer of the provided size.void
close()
Close this Accountant.static org.apache.arrow.memory.ImmutableConfig.Builder
Returns a builder class for configuring BaseAllocator's options.static org.apache.arrow.memory.BaseAllocator.Config
Returns a defaultBaseAllocator.Config
instance.boolean
forceAllocate
(long size) Increase the accounting.long
Return the current amount of allocated memory that this Accountant is managing accounting for.Returns the set of child allocators.getEmpty()
Get a reference to the empty buffer associated with this allocator.long
long
Return the initial reservation.long
getLimit()
Return the current limit of this Accountant.Returns the allocation listener used by this allocator.getName()
Return the name of the accountant.@Nullable org.apache.arrow.memory.BaseAllocator
Returns the parent allocator.long
The peak memory allocated by this Accountant.getRoot()
Get the root allocator of this allocator.Gets the rounding policy of the allocator.static boolean
isDebug()
boolean
newChildAllocator
(String name, long initReservation, long maxAllocation) Create a new child allocator.newChildAllocator
(String name, AllocationListener listener, long initReservation, long maxAllocation) Create a new child allocator.Create an allocation reservation.void
releaseBytes
(long size) void
setLimit
(long newLimit) Set the maximum amount of memory that can be allocated in the this Accountant before failing an allocation.toString()
Provide a verbose string of the current allocator state.void
verify()
Verify the accounting state of the allocation system.wrapForeignAllocation
(ForeignAllocation allocation) EXPERIMENTAL: Wrap an allocation created outside this BufferAllocator.Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface org.apache.arrow.memory.BufferAllocator
forceAllocate, getAllocatedMemory, getHeadroom, getInitReservation, getLimit, getPeakMemoryAllocation, isOverLimit, releaseBytes, setLimit
-
Field Details
-
DEBUG_ALLOCATOR
- See Also:
-
DEBUG_LOG_LENGTH
public static final int DEBUG_LOG_LENGTH- See Also:
-
DEBUG
public static final boolean DEBUG -
DEFAULT_CONFIG
public static final org.apache.arrow.memory.BaseAllocator.Config DEFAULT_CONFIG -
parent
protected final @Nullable org.apache.arrow.memory.Accountant parentThe parent allocator. -
reservation
protected final long reservationThe amount of memory reserved for this allocator. Releases below this amount of memory will not be returned to the parent Accountant until this Accountant is closed.
-
-
Constructor Details
-
RootAllocator
public RootAllocator() -
RootAllocator
public RootAllocator(long limit) -
RootAllocator
-
RootAllocator
Constructor.- Parameters:
listener
- the allocation listenerlimit
- max allocation size in bytesroundingPolicy
- the policy for rounding the buffer size
-
RootAllocator
public RootAllocator(org.apache.arrow.memory.BaseAllocator.Config config)
-
-
Method Details
-
verify
public void verify()Verify the accounting state of the allocation system. -
getListener
Description copied from interface:BufferAllocator
Returns the allocation listener used by this allocator.- Specified by:
getListener
in interfaceBufferAllocator
- Returns:
- the
AllocationListener
instance. OrAllocationListener.NOOP
by default if no listener is configured when this allocator was created.
-
getParentAllocator
public @Nullable org.apache.arrow.memory.BaseAllocator getParentAllocator()Description copied from interface:BufferAllocator
Returns the parent allocator.- Specified by:
getParentAllocator
in interfaceBufferAllocator
- Returns:
- parent allocator
-
getChildAllocators
Description copied from interface:BufferAllocator
Returns the set of child allocators.- Specified by:
getChildAllocators
in interfaceBufferAllocator
- Returns:
- set of child allocators
-
isDebug
public static boolean isDebug() -
assertOpen
public void assertOpen()Description copied from interface:BufferAllocator
Asserts (using java assertions) that the provided allocator is currently open. If assertions are disabled, this is a no-op.- Specified by:
assertOpen
in interfaceBufferAllocator
-
getName
Return the name of the accountant.- Specified by:
getName
in interfaceBufferAllocator
- Returns:
- name of accountant
-
getEmpty
Description copied from interface:BufferAllocator
Get a reference to the empty buffer associated with this allocator. Empty buffers are special because we don't worry about them leaking or managing reference counts on them since they don't actually point to any memory.- Specified by:
getEmpty
in interfaceBufferAllocator
- Returns:
- the empty buffer
-
wrapForeignAllocation
Description copied from interface:BufferAllocator
EXPERIMENTAL: Wrap an allocation created outside this BufferAllocator.This is useful to integrate allocations from native code into the same memory management framework as Java-allocated buffers, presenting users a consistent API. The created buffer will be tracked by this allocator and can be transferred like Java-allocated buffers.
The underlying allocation will be closed when all references to the buffer are released. If this method throws, the underlying allocation will also be closed.
- Specified by:
wrapForeignAllocation
in interfaceBufferAllocator
- Parameters:
allocation
- The underlying allocation.
-
buffer
Description copied from interface:BufferAllocator
Allocate a new or reused buffer of the provided size. Note that the buffer may technically be larger than the requested size for rounding purposes. However, the buffer's capacity will be set to the configured size.- Specified by:
buffer
in interfaceBufferAllocator
- Parameters:
initialRequestSize
- The size in bytes.- Returns:
- a new ArrowBuf, or null if the request can't be satisfied
-
buffer
Description copied from interface:BufferAllocator
Allocate a new or reused buffer of the provided size. Note that the buffer may technically be larger than the requested size for rounding purposes. However, the buffer's capacity will be set to the configured size.- Specified by:
buffer
in interfaceBufferAllocator
- Parameters:
initialRequestSize
- The size in bytes.manager
- A buffer manager to manage reallocation.- Returns:
- a new ArrowBuf, or null if the request can't be satisfied
-
getRoot
Description copied from interface:BufferAllocator
Get the root allocator of this allocator. If this allocator is already a root, return this directly.- Specified by:
getRoot
in interfaceBufferAllocator
- Returns:
- The root allocator
-
newChildAllocator
Description copied from interface:BufferAllocator
Create a new child allocator.- Specified by:
newChildAllocator
in interfaceBufferAllocator
- Parameters:
name
- the name of the allocator.initReservation
- the initial space reservation (obtained from this allocator)maxAllocation
- maximum amount of space the new allocator can allocate- Returns:
- the new allocator, or null if it can't be created
-
newChildAllocator
public BufferAllocator newChildAllocator(String name, AllocationListener listener, long initReservation, long maxAllocation) Description copied from interface:BufferAllocator
Create a new child allocator.- Specified by:
newChildAllocator
in interfaceBufferAllocator
- Parameters:
name
- the name of the allocator.listener
- allocation listener for the newly created childinitReservation
- the initial space reservation (obtained from this allocator)maxAllocation
- maximum amount of space the new allocator can allocate- Returns:
- the new allocator, or null if it can't be created
-
newReservation
Description copied from interface:BufferAllocator
Create an allocation reservation. A reservation is a way of building up a request for a buffer whose size is not known in advance. See- Specified by:
newReservation
in interfaceBufferAllocator
- Returns:
- the newly created reservation
- See Also:
-
close
public void close()Close this Accountant. This will release any reservation bytes back to a parent Accountant.- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceBufferAllocator
-
toString
-
toVerboseString
Provide a verbose string of the current allocator state. Includes the state of all child allocators, along with historical logs of each object and including stacktraces.- Specified by:
toVerboseString
in interfaceBufferAllocator
- Returns:
- A Verbose string of current allocator state.
-
defaultConfig
public static org.apache.arrow.memory.BaseAllocator.Config defaultConfig()Returns a defaultBaseAllocator.Config
instance.- See Also:
-
ImmutableConfig.Builder
-
configBuilder
public static org.apache.arrow.memory.ImmutableConfig.Builder configBuilder()Returns a builder class for configuring BaseAllocator's options. -
getRoundingPolicy
Description copied from interface:BufferAllocator
Gets the rounding policy of the allocator.- Specified by:
getRoundingPolicy
in interfaceBufferAllocator
-
forceAllocate
public boolean forceAllocate(long size) Increase the accounting. Returns whether the allocation fit within limits.- Parameters:
size
- to increase- Returns:
- Whether the allocation fit within limits.
-
releaseBytes
public void releaseBytes(long size) -
isOverLimit
public boolean isOverLimit() -
getLimit
public long getLimit()Return the current limit of this Accountant.- Returns:
- Limit in bytes.
-
getInitReservation
public long getInitReservation()Return the initial reservation.- Returns:
- reservation in bytes.
-
setLimit
public void setLimit(long newLimit) Set the maximum amount of memory that can be allocated in the this Accountant before failing an allocation.- Parameters:
newLimit
- The limit in bytes.
-
getAllocatedMemory
public long getAllocatedMemory()Return the current amount of allocated memory that this Accountant is managing accounting for. Note this does not include reservation memory that hasn't been allocated.- Returns:
- Currently allocate memory in bytes.
-
getPeakMemoryAllocation
public long getPeakMemoryAllocation()The peak memory allocated by this Accountant.- Returns:
- The peak allocated memory in bytes.
-
getHeadroom
public long getHeadroom()
-