Memory (management)¶
Devices¶
-
class Device : public std::enable_shared_from_this<Device>, public arrow::util::EqualityComparable<Device>¶
EXPERIMENTAL: Abstract interface for hardware devices.
This object represents a device with access to some memory spaces. When handling a Buffer or raw memory address, it allows deciding in which context the raw memory address should be interpreted (e.g. CPU-accessible memory, or embedded memory on some particular GPU).
Subclassed by arrow::CPUDevice, arrow::cuda::CudaDevice
Public Functions
-
virtual const char *type_name() const = 0¶
A shorthand for this device’s type.
The returned value is different for each device class, but is the same for all instances of a given class. It can be used as a replacement for RTTI.
-
virtual std::string ToString() const = 0¶
A human-readable description of the device.
The returned value should be detailed enough to distinguish between different instances, where necessary.
-
virtual bool Equals(const Device&) const = 0¶
Whether this instance points to the same device as another one.
-
inline bool is_cpu() const¶
Whether this device is the main CPU device.
This shorthand method is very useful when deciding whether a memory address is CPU-accessible.
-
virtual std::shared_ptr<MemoryManager> default_memory_manager() = 0¶
Return a MemoryManager instance tied to this device.
The returned instance uses default parameters for this device type’s MemoryManager implementation. Some devices also allow constructing MemoryManager instances with non-default parameters.
-
virtual const char *type_name() const = 0¶
-
class CPUDevice : public arrow::Device¶
Public Functions
-
virtual const char *type_name() const override¶
A shorthand for this device’s type.
The returned value is different for each device class, but is the same for all instances of a given class. It can be used as a replacement for RTTI.
-
virtual std::string ToString() const override¶
A human-readable description of the device.
The returned value should be detailed enough to distinguish between different instances, where necessary.
-
virtual bool Equals(const Device&) const override¶
Whether this instance points to the same device as another one.
-
virtual std::shared_ptr<MemoryManager> default_memory_manager() override¶
Return a MemoryManager instance tied to this device.
The returned instance uses default parameters for this device type’s MemoryManager implementation. Some devices also allow constructing MemoryManager instances with non-default parameters.
Public Static Functions
-
static std::shared_ptr<MemoryManager> memory_manager(MemoryPool *pool)¶
Create a MemoryManager.
The returned MemoryManager will use the given MemoryPool for allocations.
-
virtual const char *type_name() const override¶
-
std::shared_ptr<MemoryManager> arrow::default_cpu_memory_manager()¶
Return the default CPU MemoryManager instance.
The returned singleton instance uses the default MemoryPool. This function is a faster spelling of
CPUDevice::Instance()->default_memory_manager()
.
Memory Managers¶
-
class MemoryManager : public std::enable_shared_from_this<MemoryManager>¶
EXPERIMENTAL: An object that provides memory management primitives.
A MemoryManager is always tied to a particular Device instance. It can also have additional parameters (such as a MemoryPool to allocate CPU memory).
Subclassed by arrow::CPUMemoryManager, arrow::cuda::CudaMemoryManager
Public Functions
-
inline const std::shared_ptr<Device> &device() const¶
The device this MemoryManager is tied to.
-
inline bool is_cpu() const¶
Whether this MemoryManager is tied to the main CPU device.
This shorthand method is very useful when deciding whether a memory address is CPU-accessible.
Create a RandomAccessFile to read a particular buffer.
The given buffer must be tied to this MemoryManager.
See also the Buffer::GetReader shorthand.
Create a OutputStream to write to a particular buffer.
The given buffer must be mutable and tied to this MemoryManager. The returned stream object writes into the buffer’s underlying memory (but it won’t resize it).
See also the Buffer::GetWriter shorthand.
Public Static Functions
Copy a Buffer to a destination MemoryManager.
See also the Buffer::Copy shorthand.
Copy a non-owned Buffer to a destination MemoryManager.
This is useful for cases where the source memory area is externally managed (its lifetime not tied to the source Buffer), otherwise please use CopyBuffer().
Make a no-copy Buffer view in a destination MemoryManager.
See also the Buffer::View shorthand.
-
inline const std::shared_ptr<Device> &device() const¶
-
class CPUMemoryManager : public arrow::MemoryManager¶
Public Functions
Create a RandomAccessFile to read a particular buffer.
The given buffer must be tied to this MemoryManager.
See also the Buffer::GetReader shorthand.
Create a OutputStream to write to a particular buffer.
The given buffer must be mutable and tied to this MemoryManager. The returned stream object writes into the buffer’s underlying memory (but it won’t resize it).
See also the Buffer::GetWriter shorthand.
-
virtual Result<std::unique_ptr<Buffer>> AllocateBuffer(int64_t size) override¶
Allocate a (mutable) Buffer.
The buffer will be allocated in the device’s memory.
-
inline MemoryPool *pool() const¶
Return the MemoryPool associated with this MemoryManager.
Buffers¶
-
class Buffer¶
Object containing a pointer to a piece of contiguous memory with a particular size.
Buffers have two related notions of length: size and capacity. Size is the number of bytes that might have valid data. Capacity is the number of bytes that were allocated for the buffer in total.
The Buffer base class does not own its memory, but subclasses often do.
The following invariant is always true: Size <= Capacity
Subclassed by arrow::cuda::CudaBuffer, arrow::MutableBuffer, arrow::py::NumPyBuffer, arrow::py::PyBuffer, arrow::py::PyForeignBuffer
Public Functions
-
inline Buffer(const uint8_t *data, int64_t size)¶
Construct from buffer and size without copying memory.
Note
The passed memory must be kept alive through some other means
- Parameters:
data – [in] a memory buffer
size – [in] buffer size
-
inline explicit Buffer(util::string_view data)¶
Construct from string_view without copying memory.
Note
The memory viewed by data must not be deallocated in the lifetime of the Buffer; temporary rvalue strings must be stored in an lvalue somewhere
- Parameters:
data – [in] a string_view object
An offset into data that is owned by another buffer, but we want to be able to retain a valid pointer to it even after other shared_ptr’s to the parent buffer have been destroyed.
This method makes no assertions about alignment or padding of the buffer but in general we expected buffers to be aligned and padded to 64 bytes. In the future we might add utility methods to help determine if a buffer satisfies this contract.
-
std::string ToHexString()¶
Construct a new std::string with a hexadecimal representation of the buffer.
- Returns:
std::string
-
bool Equals(const Buffer &other, int64_t nbytes) const¶
Return true if both buffers are the same size and contain the same bytes up to the number of compared bytes.
-
bool Equals(const Buffer &other) const¶
Return true if both buffers are the same size and contain the same bytes.
-
Result<std::shared_ptr<Buffer>> CopySlice(const int64_t start, const int64_t nbytes, MemoryPool *pool = default_memory_pool()) const¶
Copy a section of the buffer into a new Buffer.
-
inline void ZeroPadding()¶
Zero bytes in padding, i.e. bytes between size_ and capacity_.
-
std::string ToString() const¶
Copy buffer contents into a new std::string.
Note
Can throw std::bad_alloc if buffer is large
- Returns:
std::string
-
inline explicit operator util::string_view() const¶
View buffer contents as a util::string_view.
- Returns:
util::string_view
-
inline explicit operator util::bytes_view() const¶
View buffer contents as a util::bytes_view.
- Returns:
util::bytes_view
-
inline const uint8_t *data() const¶
Return a pointer to the buffer’s data.
The buffer has to be a CPU buffer (
is_cpu()
is true). Otherwise, an assertion may be thrown or a null pointer may be returned.To get the buffer’s data address regardless of its device, call
address()
.
-
inline uint8_t *mutable_data()¶
Return a writable pointer to the buffer’s data.
The buffer has to be a mutable CPU buffer (
is_cpu()
andis_mutable()
are true). Otherwise, an assertion may be thrown or a null pointer may be returned.To get the buffer’s mutable data address regardless of its device, call
mutable_address()
.
-
inline uintptr_t address() const¶
Return the device address of the buffer’s data.
-
inline uintptr_t mutable_address() const¶
Return a writable device address to the buffer’s data.
The buffer has to be a mutable buffer (
is_mutable()
is true). Otherwise, an assertion may be thrown or 0 may be returned.
-
inline int64_t size() const¶
Return the buffer’s size in bytes.
-
inline int64_t capacity() const¶
Return the buffer’s capacity (number of allocated bytes)
-
inline bool is_cpu() const¶
Whether the buffer is directly CPU-accessible.
If this function returns true, you can read directly from the buffer’s
data()
pointer. Otherwise, you’ll have toView()
orCopy()
it.
-
inline bool is_mutable() const¶
Whether the buffer is mutable.
If this function returns true, you are allowed to modify buffer contents using the pointer returned by
mutable_data()
ormutable_address()
.
Public Static Functions
-
static std::shared_ptr<Buffer> FromString(std::string data)¶
Construct an immutable buffer that takes ownership of the contents of an std::string (without copying it).
- Parameters:
data – [in] a string to own
- Returns:
a new Buffer instance
Create buffer referencing typed memory with some length without copying.
- Parameters:
data – [in] the typed memory as C array
length – [in] the number of values in the array
- Returns:
a new shared_ptr<Buffer>
Create buffer referencing std::vector with some length without copying.
- Parameters:
data – [in] the vector to be referenced. If this vector is changed, the buffer may become invalid
- Returns:
a new shared_ptr<Buffer>
Get a RandomAccessFile for reading a buffer.
The returned file object reads from this buffer’s underlying memory.
Get a OutputStream for writing to a buffer.
The buffer must be mutable. The returned stream object writes into the buffer’s underlying memory (but it won’t resize it).
Copy buffer.
The buffer contents will be copied into a new buffer allocated by the given MemoryManager. This function supports cross-device copies.
Copy a non-owned buffer.
This is useful for cases where the source memory area is externally managed (its lifetime not tied to the source Buffer), otherwise please use Copy().
View buffer.
Return a Buffer that reflects this buffer, seen potentially from another device, without making an explicit copy of the contents. The underlying mechanism is typically implemented by the kernel or device driver, and may involve lazy caching of parts of the buffer contents on the destination device’s memory.
If a non-copy view is unsupported for the buffer on the given device, nullptr is returned. An error can be returned if some low-level operation fails (such as an out-of-memory condition).
View or copy buffer.
Try to view buffer contents on the given MemoryManager’s device, but fall back to copying if a no-copy view isn’t supported.
-
inline Buffer(const uint8_t *data, int64_t size)¶
-
class MutableBuffer : public arrow::Buffer¶
A Buffer whose contents can be mutated. May or may not own its data.
Subclassed by arrow::cuda::CudaHostBuffer, arrow::ResizableBuffer
Public Static Functions
Create buffer referencing typed memory with some length.
- Parameters:
data – [in] the typed memory as C array
length – [in] the number of values in the array
- Returns:
a new shared_ptr<Buffer>
-
class ResizableBuffer : public arrow::MutableBuffer¶
A mutable buffer that can be resized.
Public Functions
-
virtual Status Resize(const int64_t new_size, bool shrink_to_fit) = 0¶
Change buffer reported size to indicated size, allocating memory if necessary.
This will ensure that the capacity of the buffer is a multiple of 64 bytes as defined in Layout.md. Consider using ZeroPadding afterwards, to conform to the Arrow layout specification.
- Parameters:
new_size – The new size for the buffer.
shrink_to_fit – Whether to shrink the capacity if new size < current size
-
virtual Status Resize(const int64_t new_size, bool shrink_to_fit) = 0¶
Memory Pools¶
-
MemoryPool *arrow::default_memory_pool()¶
Return the process-wide default memory pool.
-
Status arrow::jemalloc_memory_pool(MemoryPool **out)¶
Return a process-wide memory pool based on jemalloc.
May return NotImplemented if jemalloc is not available.
-
Status arrow::mimalloc_memory_pool(MemoryPool **out)¶
Return a process-wide memory pool based on mimalloc.
May return NotImplemented if mimalloc is not available.
-
MemoryPool *arrow::system_memory_pool()¶
Return a process-wide memory pool based on the system allocator.
-
class MemoryPool¶
Base class for memory allocation on the CPU.
Besides tracking the number of allocated bytes, the allocator also should take care of the required 64-byte alignment.
Subclassed by arrow::LoggingMemoryPool, arrow::ProxyMemoryPool, arrow::stl::STLMemoryPool< Allocator >
Public Functions
-
virtual Status Allocate(int64_t size, uint8_t **out) = 0¶
Allocate a new memory region of at least size bytes.
The allocated region shall be 64-byte aligned.
-
virtual Status Reallocate(int64_t old_size, int64_t new_size, uint8_t **ptr) = 0¶
Resize an already allocated memory section.
As by default most default allocators on a platform don’t support aligned reallocation, this function can involve a copy of the underlying data.
-
virtual void Free(uint8_t *buffer, int64_t size) = 0¶
Free an allocated region.
- Parameters:
buffer – Pointer to the start of the allocated memory region
size – Allocated size located at buffer. An allocator implementation may use this for tracking the amount of allocated bytes as well as for faster deallocation if supported by its backend.
-
inline virtual void ReleaseUnused()¶
Return unused memory to the OS.
Only applies to allocators that hold onto unused memory. This will be best effort, a memory pool may not implement this feature or may be unable to fulfill the request due to fragmentation.
-
virtual int64_t bytes_allocated() const = 0¶
The number of bytes that were allocated and not yet free’d through this allocator.
-
virtual int64_t max_memory() const¶
Return peak memory allocation in this memory pool.
- Returns:
Maximum bytes allocated. If not known (or not implemented), returns -1
-
virtual std::string backend_name() const = 0¶
The name of the backend used by this MemoryPool (e.g. “system” or “jemalloc”).
Public Static Functions
-
static std::unique_ptr<MemoryPool> CreateDefault()¶
EXPERIMENTAL. Create a new instance of the default MemoryPool.
-
virtual Status Allocate(int64_t size, uint8_t **out) = 0¶
-
class LoggingMemoryPool : public arrow::MemoryPool¶
Public Functions
-
virtual Status Allocate(int64_t size, uint8_t **out) override¶
Allocate a new memory region of at least size bytes.
The allocated region shall be 64-byte aligned.
-
virtual Status Reallocate(int64_t old_size, int64_t new_size, uint8_t **ptr) override¶
Resize an already allocated memory section.
As by default most default allocators on a platform don’t support aligned reallocation, this function can involve a copy of the underlying data.
-
virtual void Free(uint8_t *buffer, int64_t size) override¶
Free an allocated region.
- Parameters:
buffer – Pointer to the start of the allocated memory region
size – Allocated size located at buffer. An allocator implementation may use this for tracking the amount of allocated bytes as well as for faster deallocation if supported by its backend.
-
virtual int64_t bytes_allocated() const override¶
The number of bytes that were allocated and not yet free’d through this allocator.
-
virtual int64_t max_memory() const override¶
Return peak memory allocation in this memory pool.
- Returns:
Maximum bytes allocated. If not known (or not implemented), returns -1
-
virtual std::string backend_name() const override¶
The name of the backend used by this MemoryPool (e.g. “system” or “jemalloc”).
-
virtual Status Allocate(int64_t size, uint8_t **out) override¶
-
class ProxyMemoryPool : public arrow::MemoryPool¶
Derived class for memory allocation.
Tracks the number of bytes and maximum memory allocated through its direct calls. Actual allocation is delegated to MemoryPool class.
Public Functions
-
virtual Status Allocate(int64_t size, uint8_t **out) override¶
Allocate a new memory region of at least size bytes.
The allocated region shall be 64-byte aligned.
-
virtual Status Reallocate(int64_t old_size, int64_t new_size, uint8_t **ptr) override¶
Resize an already allocated memory section.
As by default most default allocators on a platform don’t support aligned reallocation, this function can involve a copy of the underlying data.
-
virtual void Free(uint8_t *buffer, int64_t size) override¶
Free an allocated region.
- Parameters:
buffer – Pointer to the start of the allocated memory region
size – Allocated size located at buffer. An allocator implementation may use this for tracking the amount of allocated bytes as well as for faster deallocation if supported by its backend.
-
virtual int64_t bytes_allocated() const override¶
The number of bytes that were allocated and not yet free’d through this allocator.
-
virtual int64_t max_memory() const override¶
Return peak memory allocation in this memory pool.
- Returns:
Maximum bytes allocated. If not known (or not implemented), returns -1
-
virtual std::string backend_name() const override¶
The name of the backend used by this MemoryPool (e.g. “system” or “jemalloc”).
-
virtual Status Allocate(int64_t size, uint8_t **out) override¶
-
std::vector<std::string> arrow::SupportedMemoryBackendNames()¶
Return the names of the backends supported by this Arrow build.
Allocation Functions¶
These functions allocate a buffer from a particular memory pool.
-
Result<std::unique_ptr<Buffer>> AllocateBuffer(const int64_t size, MemoryPool *pool = NULLPTR)¶
Allocate a fixed size mutable buffer from a memory pool, zero its padding.
- Parameters:
size – [in] size of buffer to allocate
pool – [in] a memory pool
-
Result<std::unique_ptr<ResizableBuffer>> AllocateResizableBuffer(const int64_t size, MemoryPool *pool = NULLPTR)¶
Allocate a resizeable buffer from a memory pool, zero its padding.
- Parameters:
size – [in] size of buffer to allocate
pool – [in] a memory pool
-
Result<std::shared_ptr<Buffer>> AllocateBitmap(int64_t length, MemoryPool *pool = NULLPTR)¶
Allocate a bitmap buffer from a memory pool no guarantee on values is provided.
- Parameters:
length – [in] size in bits of bitmap to allocate
pool – [in] memory pool to allocate memory from
-
Result<std::shared_ptr<Buffer>> AllocateEmptyBitmap(int64_t length, MemoryPool *pool = NULLPTR)¶
Allocate a zero-initialized bitmap buffer from a memory pool.
- Parameters:
length – [in] size in bits of bitmap to allocate
pool – [in] memory pool to allocate memory from
-
Result<std::shared_ptr<Buffer>> ConcatenateBuffers(const BufferVector &buffers, MemoryPool *pool = NULLPTR)¶
Concatenate multiple buffers into a single buffer.
- Parameters:
buffers – [in] to be concatenated
pool – [in] memory pool to allocate the new buffer from
Slicing¶
Construct a view on a buffer at the given offset and length.
This function cannot fail and does not check for errors (except in debug builds)
Construct a view on a buffer at the given offset, up to the buffer’s end.
This function cannot fail and does not check for errors (except in debug builds)
Input-checking version of SliceBuffer.
An Invalid Status is returned if the requested slice falls out of bounds.
Input-checking version of SliceBuffer.
An Invalid Status is returned if the requested slice falls out of bounds. Note that unlike SliceBuffer,
length
isn’t clamped to the available buffer size.
Like SliceBuffer, but construct a mutable buffer slice.
If the parent buffer is not mutable, behavior is undefined (it may abort in debug builds).
Like SliceBuffer, but construct a mutable buffer slice.
If the parent buffer is not mutable, behavior is undefined (it may abort in debug builds).
Input-checking version of SliceMutableBuffer.
An Invalid Status is returned if the requested slice falls out of bounds.
Input-checking version of SliceMutableBuffer.
An Invalid Status is returned if the requested slice falls out of bounds. Note that unlike SliceBuffer,
length
isn’t clamped to the available buffer size.
Buffer Builders¶
-
class BufferBuilder¶
A class for incrementally building a contiguous chunk of in-memory data.
Public Functions
Constructs new Builder that will start using the provided buffer until Finish/Reset are called.
The buffer is not resized.
-
inline Status Resize(const int64_t new_capacity, bool shrink_to_fit = true)¶
Resize the buffer to the nearest multiple of 64 bytes.
- Parameters:
new_capacity – the new capacity of the of the builder. Will be rounded up to a multiple of 64 bytes for padding
shrink_to_fit – if new capacity is smaller than the existing, reallocate internal buffer. Set to false to avoid reallocations when shrinking the builder.
- Returns:
-
inline Status Reserve(const int64_t additional_bytes)¶
Ensure that builder can accommodate the additional number of bytes without the need to perform allocations.
- Parameters:
additional_bytes – [in] number of additional bytes to make space for
- Returns:
-
inline Status Append(const void *data, const int64_t length)¶
Append the given data to the buffer.
The buffer is automatically expanded if necessary.
-
inline Status Append(const int64_t num_copies, uint8_t value)¶
Append copies of a value to the buffer.
The buffer is automatically expanded if necessary.
Return result of builder as a Buffer object.
The builder is reset and can be reused afterwards.
-
inline Result<std::shared_ptr<Buffer>> FinishWithLength(int64_t final_length, bool shrink_to_fit = true)¶
Like Finish, but override the final buffer size.
This is useful after writing data directly into the builder memory without calling the Append methods (basically, when using BufferBuilder mostly for memory allocation).
-
inline void Rewind(int64_t position)¶
Set size to a smaller value without modifying builder contents.
For reusable BufferBuilder classes
- Parameters:
position – [in] must be non-negative and less than or equal to the current length()
Public Static Functions
-
static inline int64_t GrowByFactor(int64_t current_capacity, int64_t new_capacity)¶
Return a capacity expanded by the desired growth factor.
-
template<typename T, typename Enable = void>
class TypedBufferBuilder¶
STL Integration¶
-
template<class T>
class allocator¶ A STL allocator delegating allocations to a Arrow MemoryPool.
Public Functions
-
inline allocator() noexcept¶
Construct an allocator from the default MemoryPool.
-
inline explicit allocator(MemoryPool *pool) noexcept¶
Construct an allocator from the given MemoryPool.
-
template<class U>
struct rebind¶
-
inline allocator() noexcept¶
-
template<typename Allocator = std::allocator<uint8_t>>
class STLMemoryPool : public arrow::MemoryPool¶ A MemoryPool implementation delegating allocations to a STL allocator.
Note that STL allocators don’t provide a resizing operation, and therefore any buffer resizes will do a full reallocation and copy.
Public Functions
-
inline explicit STLMemoryPool(const Allocator &alloc)¶
Construct a memory pool from the given allocator.
-
inline virtual Status Allocate(int64_t size, uint8_t **out) override¶
Allocate a new memory region of at least size bytes.
The allocated region shall be 64-byte aligned.
-
inline virtual Status Reallocate(int64_t old_size, int64_t new_size, uint8_t **ptr) override¶
Resize an already allocated memory section.
As by default most default allocators on a platform don’t support aligned reallocation, this function can involve a copy of the underlying data.
-
inline virtual void Free(uint8_t *buffer, int64_t size) override¶
Free an allocated region.
- Parameters:
buffer – Pointer to the start of the allocated memory region
size – Allocated size located at buffer. An allocator implementation may use this for tracking the amount of allocated bytes as well as for faster deallocation if supported by its backend.
-
inline virtual int64_t bytes_allocated() const override¶
The number of bytes that were allocated and not yet free’d through this allocator.
-
inline virtual int64_t max_memory() const override¶
Return peak memory allocation in this memory pool.
- Returns:
Maximum bytes allocated. If not known (or not implemented), returns -1
-
inline virtual std::string backend_name() const override¶
The name of the backend used by this MemoryPool (e.g. “system” or “jemalloc”).
-
inline explicit STLMemoryPool(const Allocator &alloc)¶