Memory (management)¶

Devices¶

class arrow::Device : public std::enable_shared_from_this<Device>, public arrow::util::EqualityComparable<Device>¶

EXPERIMENTAL: Abstract interface for hardware devices.

This object represents a device with access to some memory spaces. When handling a Buffer or raw memory address, it allows deciding in which context the raw memory address should be interpreted (e.g. CPU-accessible memory, or embedded memory on some particular GPU).

Subclassed by arrow::CPUDevice, arrow::cuda::CudaDevice

Public Functions

virtual const char *type_name() const = 0¶

A shorthand for this device’s type.

The returned value is different for each device class, but is the same for all instances of a given class. It can be used as a replacement for RTTI.

virtual std::string ToString() const = 0¶

A human-readable description of the device.

The returned value should be detailed enough to distinguish between different instances, where necessary.

virtual bool Equals(const Device&) const = 0¶: Whether this instance points to the same device as another one.

inline bool is_cpu() const¶

Whether this device is the main CPU device.

This shorthand method is very useful when deciding whether a memory address is CPU-accessible.

virtual std::shared_ptr<MemoryManager> default_memory_manager() = 0¶

Return a MemoryManager instance tied to this device.

The returned instance uses default parameters for this device type’s MemoryManager implementation. Some devices also allow constructing MemoryManager instances with non-default parameters.

class arrow::CPUDevice : public arrow::Device ¶

Public Functions

virtual const char *type_name() const override¶

A shorthand for this device’s type.

The returned value is different for each device class, but is the same for all instances of a given class. It can be used as a replacement for RTTI.

virtual std::string ToString() const override¶

A human-readable description of the device.

The returned value should be detailed enough to distinguish between different instances, where necessary.

virtual bool Equals(const Device&) const override¶: Whether this instance points to the same device as another one.

virtual std::shared_ptr<MemoryManager> default_memory_manager() override¶

Return a MemoryManager instance tied to this device.

The returned instance uses default parameters for this device type’s MemoryManager implementation. Some devices also allow constructing MemoryManager instances with non-default parameters.

Public Static Functions

static std::shared_ptr<Device> Instance()¶: Return the global CPUDevice instance.

static std::shared_ptr<MemoryManager> memory_manager(MemoryPool *pool)¶

Create a MemoryManager.

The returned MemoryManager will use the given MemoryPool for allocations.

std::shared_ptr<MemoryManager> arrow::default_cpu_memory_manager()¶

Return the default CPU MemoryManager instance.

The returned singleton instance uses the default MemoryPool. This function is a faster spelling of CPUDevice::Instance()->default_memory_manager().

Memory Managers¶

class arrow::MemoryManager : public std::enable_shared_from_this<MemoryManager>¶

EXPERIMENTAL: An object that provides memory management primitives.

A MemoryManager is always tied to a particular Device instance. It can also have additional parameters (such as a MemoryPool to allocate CPU memory).

Subclassed by arrow::CPUMemoryManager, arrow::cuda::CudaMemoryManager

Public Functions

inline const std::shared_ptr<Device> &device() const¶: The device this MemoryManager is tied to.

inline bool is_cpu() const¶

Whether this MemoryManager is tied to the main CPU device.

This shorthand method is very useful when deciding whether a memory address is CPU-accessible.

virtual Result<std::shared_ptr<io::RandomAccessFile>> GetBufferReader(std::shared_ptr<Buffer> buf) = 0¶

Create a RandomAccessFile to read a particular buffer.

The given buffer must be tied to this MemoryManager.

See also the Buffer::GetReader shorthand.

virtual Result<std::shared_ptr<io::OutputStream>> GetBufferWriter(std::shared_ptr<Buffer> buf) = 0¶

Create a OutputStream to write to a particular buffer.

The given buffer must be mutable and tied to this MemoryManager. The returned stream object writes into the buffer’s underlying memory (but it won’t resize it).

See also the Buffer::GetWriter shorthand.

virtual Result<std::shared_ptr<Buffer>> AllocateBuffer(int64_t size) = 0¶

Allocate a (mutable) Buffer.

The buffer will be allocated in the device’s memory.

Public Static Functions

static Result<std::shared_ptr<Buffer>> CopyBuffer(const std::shared_ptr<Buffer> &source, const std::shared_ptr<MemoryManager> &to)¶

Copy a Buffer to a destination MemoryManager.

Buffers¶

class arrow::Buffer¶

Object containing a pointer to a piece of contiguous memory with a particular size.

Buffers have two related notions of length: size and capacity. Size is the number of bytes that might have valid data. Capacity is the number of bytes that were allocated for the buffer in total.

The Buffer base class does not own its memory, but subclasses often do.

The following invariant is always true: Size <= Capacity

Subclassed by arrow::cuda::CudaBuffer, arrow::MutableBuffer, arrow::py::NumPyBuffer, arrow::py::PyBuffer, arrow::py::PyForeignBuffer

Public Functions

inline Buffer(const uint8_t *data, int64_t size)¶

Construct from buffer and size without copying memory.

Note

The passed memory must be kept alive through some other means

Parameters

[in] data – a memory buffer
[in] size – buffer size

inline explicit Buffer(util::string_view data)¶

Construct from string_view without copying memory.

Note

The memory viewed by data must not be deallocated in the lifetime of the Buffer; temporary rvalue strings must be stored in an lvalue somewhere

Parameters: [in] data – a string_view object

inline Buffer(const std::shared_ptr<Buffer> &parent, const int64_t offset, const int64_t size)¶

An offset into data that is owned by another buffer, but we want to be able to retain a valid pointer to it even after other shared_ptr’s to the parent buffer have been destroyed.

This method makes no assertions about alignment or padding of the buffer but in general we expected buffers to be aligned and padded to 64 bytes. In the future we might add utility methods to help determine if a buffer satisfies this contract.

std::string ToHexString()¶

Construct a new std::string with a hexadecimal representation of the buffer.

Returns: std::string

bool Equals(const Buffer &other, int64_t nbytes) const¶: Return true if both buffers are the same size and contain the same bytes up to the number of compared bytes.

bool Equals(const Buffer &other) const¶: Return true if both buffers are the same size and contain the same bytes.

Result<std::shared_ptr<Buffer>> CopySlice(const int64_t start, const int64_t nbytes, MemoryPool *pool = default_memory_pool()) const¶: Copy a section of the buffer into a new Buffer.

inline void ZeroPadding()¶: Zero bytes in padding, i.e. bytes between size_ and capacity_.

std::string ToString() const¶

Copy buffer contents into a new std::string.

Note

Can throw std::bad_alloc if buffer is large

Returns: std::string

inline explicit operator util::string_view() const¶

View buffer contents as a util::string_view.

Returns: util::string_view

inline explicit operator util::bytes_view() const¶

View buffer contents as a util::bytes_view.

Returns: util::bytes_view

inline const uint8_t *data() const¶

Return a pointer to the buffer’s data.

The buffer has to be a CPU buffer (is_cpu() is true). Otherwise, an assertion may be thrown or a null pointer may be returned.

To get the buffer’s data address regardless of its device, call address().

inline uint8_t *mutable_data()¶

Return a writable pointer to the buffer’s data.

The buffer has to be a mutable CPU buffer (is_cpu() and is_mutable() are true). Otherwise, an assertion may be thrown or a null pointer may be returned.

To get the buffer’s mutable data address regardless of its device, call mutable_address().

inline uintptr_t address() const¶: Return the device address of the buffer’s data.

inline uintptr_t mutable_address() const¶

Return a writable device address to the buffer’s data.

The buffer has to be a mutable buffer (is_mutable() is true). Otherwise, an assertion may be thrown or 0 may be returned.

inline int64_t size() const¶: Return the buffer’s size in bytes.

inline int64_t capacity() const¶: Return the buffer’s capacity (number of allocated bytes)

inline bool is_cpu() const¶

Whether the buffer is directly CPU-accessible.

If this function returns true, you can read directly from the buffer’s data() pointer. Otherwise, you’ll have to View() or Copy() it.

inline bool is_mutable() const¶

Whether the buffer is mutable.

If this function returns true, you are allowed to modify buffer contents using the pointer returned by mutable_data() or mutable_address().

Public Static Functions

static std::shared_ptr<Buffer> FromString(std::string data)¶

Construct an immutable buffer that takes ownership of the contents of an std::string (without copying it).

Parameters: [in] data – a string to own
Returns: a new Buffer instance

template<typename T, typename SizeType = int64_t> static inline std::shared_ptr<Buffer> Wrap(const T *data, SizeType length)¶

Create buffer referencing typed memory with some length without copying.

Parameters

[in] data – the typed memory as C array
[in] length – the number of values in the array

Returns

a new shared_ptr<Buffer>

template<typename T> static inline std::shared_ptr<Buffer> Wrap(const std::vector<T> &data)¶

Create buffer referencing std::vector with some length without copying.

Parameters: [in] data – the vector to be referenced. If this vector is changed, the buffer may become invalid
Returns: a new shared_ptr<Buffer>

static Result<std::shared_ptr<io::RandomAccessFile>> GetReader(std::shared_ptr<Buffer>)¶

Get a RandomAccessFile for reading a buffer.

The returned file object reads from this buffer’s underlying memory.

static Result<std::shared_ptr<io::OutputStream>> GetWriter(std::shared_ptr<Buffer>)¶

Get a OutputStream for writing to a buffer.

The buffer must be mutable. The returned stream object writes into the buffer’s underlying memory (but it won’t resize it).

static Result<std::shared_ptr<Buffer>> Copy(std::shared_ptr<Buffer> source, const std::shared_ptr<MemoryManager> &to)¶

Copy buffer.

The buffer contents will be copied into a new buffer allocated by the given MemoryManager. This function supports cross-device copies.

static Result<std::shared_ptr<Buffer>> View(std::shared_ptr<Buffer> source, const std::shared_ptr<MemoryManager> &to)¶

View buffer.

Return a Buffer that reflects this buffer, seen potentially from another device, without making an explicit copy of the contents. The underlying mechanism is typically implemented by the kernel or device driver, and may involve lazy caching of parts of the buffer contents on the destination device’s memory.

If a non-copy view is unsupported for the buffer on the given device, nullptr is returned. An error can be returned if some low-level operation fails (such as an out-of-memory condition).

static Result<std::shared_ptr<Buffer>> ViewOrCopy(std::shared_ptr<Buffer> source, const std::shared_ptr<MemoryManager> &to)¶

View or copy buffer.

Try to view buffer contents on the given MemoryManager’s device, but fall back to copying if a no-copy view isn’t supported.

class arrow::MutableBuffer : public arrow::Buffer ¶

A Buffer whose contents can be mutated. May or may not own its data.

Subclassed by arrow::cuda::CudaHostBuffer, arrow::ResizableBuffer

Public Static Functions

template<typename T, typename SizeType = int64_t> static inline std::shared_ptr<Buffer> Wrap(T *data, SizeType length)¶

Create buffer referencing typed memory with some length.

Parameters

[in] data – the typed memory as C array
[in] length – the number of values in the array

Returns

a new shared_ptr<Buffer>

class arrow::ResizableBuffer : public arrow::MutableBuffer ¶

A mutable buffer that can be resized.

Public Functions

virtual Status Resize(const int64_t new_size, bool shrink_to_fit = true) = 0¶

Change buffer reported size to indicated size, allocating memory if necessary.

This will ensure that the capacity of the buffer is a multiple of 64 bytes as defined in Layout.md. Consider using ZeroPadding afterwards, to conform to the Arrow layout specification.

Parameters

new_size – The new size for the buffer.
shrink_to_fit – Whether to shrink the capacity if new size < current size

virtual Status Reserve(const int64_t new_capacity) = 0¶

Ensure that buffer has enough memory allocated to fit the indicated capacity (and meets the 64 byte padding requirement in Layout.md).

It does not change buffer’s reported size and doesn’t zero the padding.

Memory Pools¶

MemoryPool *arrow::default_memory_pool()¶: Return the process-wide default memory pool.

Status arrow::jemalloc_memory_pool(MemoryPool **out)¶

Return a process-wide memory pool based on jemalloc.

May return NotImplemented if jemalloc is not available.

Status arrow::mimalloc_memory_pool(MemoryPool **out)¶

Return a process-wide memory pool based on mimalloc.

May return NotImplemented if mimalloc is not available.

MemoryPool *arrow::system_memory_pool()¶: Return a process-wide memory pool based on the system allocator.

class arrow::MemoryPool¶

Base class for memory allocation on the CPU.

Besides tracking the number of allocated bytes, the allocator also should take care of the required 64-byte alignment.

Subclassed by arrow::dataset::jni::ReservationListenableMemoryPool, arrow::LoggingMemoryPool, arrow::ProxyMemoryPool, arrow::stl::STLMemoryPool< Allocator >

Public Functions

virtual Status Allocate(int64_t size, uint8_t **out) = 0¶

Allocate a new memory region of at least size bytes.

The allocated region shall be 64-byte aligned.

virtual Status Reallocate(int64_t old_size, int64_t new_size, uint8_t **ptr) = 0¶

Resize an already allocated memory section.

As by default most default allocators on a platform don’t support aligned reallocation, this function can involve a copy of the underlying data.

virtual void Free(uint8_t *buffer, int64_t size) = 0¶

Free an allocated region.

Parameters

buffer – Pointer to the start of the allocated memory region
size – Allocated size located at buffer. An allocator implementation may use this for tracking the amount of allocated bytes as well as for faster deallocation if supported by its backend.

inline virtual void ReleaseUnused()¶

Return unused memory to the OS.

Only applies to allocators that hold onto unused memory. This will be best effort, a memory pool may not implement this feature or may be unable to fulfill the request due to fragmentation.

virtual int64_t bytes_allocated() const = 0¶: The number of bytes that were allocated and not yet free’d through this allocator.

virtual int64_t max_memory() const¶

Return peak memory allocation in this memory pool.

Returns: Maximum bytes allocated. If not known (or not implemented), returns -1

virtual std::string backend_name() const = 0¶: The name of the backend used by this MemoryPool (e.g. “system” or “jemalloc”).

Public Static Functions

static std::unique_ptr<MemoryPool> CreateDefault()¶: EXPERIMENTAL. Create a new instance of the default MemoryPool.

class arrow::LoggingMemoryPool : public arrow::MemoryPool ¶

Public Functions

virtual Status Allocate(int64_t size, uint8_t **out) override¶

Allocate a new memory region of at least size bytes.

The allocated region shall be 64-byte aligned.

virtual Status Reallocate(int64_t old_size, int64_t new_size, uint8_t **ptr) override¶

Resize an already allocated memory section.

As by default most default allocators on a platform don’t support aligned reallocation, this function can involve a copy of the underlying data.

virtual void Free(uint8_t *buffer, int64_t size) override¶

Free an allocated region.

Parameters

buffer – Pointer to the start of the allocated memory region
size – Allocated size located at buffer. An allocator implementation may use this for tracking the amount of allocated bytes as well as for faster deallocation if supported by its backend.

virtual int64_t bytes_allocated() const override¶: The number of bytes that were allocated and not yet free’d through this allocator.

virtual int64_t max_memory() const override¶

Return peak memory allocation in this memory pool.

Returns: Maximum bytes allocated. If not known (or not implemented), returns -1

virtual std::string backend_name() const override¶: The name of the backend used by this MemoryPool (e.g. “system” or “jemalloc”).

class arrow::ProxyMemoryPool : public arrow::MemoryPool ¶

Derived class for memory allocation.

Tracks the number of bytes and maximum memory allocated through its direct calls. Actual allocation is delegated to MemoryPool class.

Public Functions

virtual Status Allocate(int64_t size, uint8_t **out) override¶

Allocate a new memory region of at least size bytes.

The allocated region shall be 64-byte aligned.

virtual Status Reallocate(int64_t old_size, int64_t new_size, uint8_t **ptr) override¶

Resize an already allocated memory section.

As by default most default allocators on a platform don’t support aligned reallocation, this function can involve a copy of the underlying data.

virtual void Free(uint8_t *buffer, int64_t size) override¶

Free an allocated region.

Parameters

buffer – Pointer to the start of the allocated memory region
size – Allocated size located at buffer. An allocator implementation may use this for tracking the amount of allocated bytes as well as for faster deallocation if supported by its backend.

virtual int64_t bytes_allocated() const override¶: The number of bytes that were allocated and not yet free’d through this allocator.

virtual int64_t max_memory() const override¶

Return peak memory allocation in this memory pool.

Returns: Maximum bytes allocated. If not known (or not implemented), returns -1

virtual std::string backend_name() const override¶: The name of the backend used by this MemoryPool (e.g. “system” or “jemalloc”).

Allocation Functions¶

These functions allocate a buffer from a particular memory pool.

Result<std::unique_ptr<Buffer>> AllocateBuffer(const int64_t size, MemoryPool *pool = NULLPTR)¶

Allocate a fixed size mutable buffer from a memory pool, zero its padding.

Parameters

[in] size – size of buffer to allocate
[in] pool – a memory pool

Result<std::unique_ptr<ResizableBuffer>> AllocateResizableBuffer(const int64_t size, MemoryPool *pool = NULLPTR)¶

Allocate a resizeable buffer from a memory pool, zero its padding.

Parameters

[in] size – size of buffer to allocate
[in] pool – a memory pool

Result<std::shared_ptr<Buffer>> AllocateBitmap(int64_t length, MemoryPool *pool = NULLPTR)¶

Allocate a bitmap buffer from a memory pool no guarantee on values is provided.

Parameters

[in] length – size in bits of bitmap to allocate
[in] pool – memory pool to allocate memory from

Status AllocateBitmap(MemoryPool *pool, int64_t length, std::shared_ptr<Buffer> *out)¶

Result<std::shared_ptr<Buffer>> AllocateEmptyBitmap(int64_t length, MemoryPool *pool = NULLPTR)¶

Allocate a zero-initialized bitmap buffer from a memory pool.

Parameters

[in] length – size in bits of bitmap to allocate
[in] pool – memory pool to allocate memory from

Result<std::shared_ptr<Buffer>> ConcatenateBuffers(const BufferVector &buffers, MemoryPool *pool = NULLPTR)¶

Concatenate multiple buffers into a single buffer.

Parameters

[in] buffers – to be concatenated
[in] pool – memory pool to allocate the new buffer from

Status ConcatenateBuffers(const BufferVector &buffers, MemoryPool *pool, std::shared_ptr<Buffer> *out)¶

Slicing¶

static inline std::shared_ptr<Buffer> SliceBuffer(const std::shared_ptr<Buffer> &buffer, const int64_t offset, const int64_t length)¶

Construct a view on a buffer at the given offset and length.

This function cannot fail and does not check for errors (except in debug builds)

static inline std::shared_ptr<Buffer> SliceBuffer(const std::shared_ptr<Buffer> &buffer, const int64_t offset)¶

Construct a view on a buffer at the given offset, up to the buffer’s end.

This function cannot fail and does not check for errors (except in debug builds)

Result<std::shared_ptr<Buffer>> SliceBufferSafe(const std::shared_ptr<Buffer> &buffer, int64_t offset)¶

Input-checking version of SliceBuffer.

An Invalid Status is returned if the requested slice falls out of bounds.

Result<std::shared_ptr<Buffer>> SliceBufferSafe(const std::shared_ptr<Buffer> &buffer, int64_t offset, int64_t length)¶

Input-checking version of SliceBuffer.

An Invalid Status is returned if the requested slice falls out of bounds. Note that unlike SliceBuffer, length isn’t clamped to the available buffer size.

std::shared_ptr<Buffer> SliceMutableBuffer(const std::shared_ptr<Buffer> &buffer, const int64_t offset, const int64_t length)¶

Like SliceBuffer, but construct a mutable buffer slice.

If the parent buffer is not mutable, behavior is undefined (it may abort in debug builds).

static inline std::shared_ptr<Buffer> SliceMutableBuffer(const std::shared_ptr<Buffer> &buffer, const int64_t offset)¶

Like SliceBuffer, but construct a mutable buffer slice.

If the parent buffer is not mutable, behavior is undefined (it may abort in debug builds).

Result<std::shared_ptr<Buffer>> SliceMutableBufferSafe(const std::shared_ptr<Buffer> &buffer, int64_t offset)¶

Input-checking version of SliceMutableBuffer.

An Invalid Status is returned if the requested slice falls out of bounds.

Result<std::shared_ptr<Buffer>> SliceMutableBufferSafe(const std::shared_ptr<Buffer> &buffer, int64_t offset, int64_t length)¶

Input-checking version of SliceMutableBuffer.

An Invalid Status is returned if the requested slice falls out of bounds. Note that unlike SliceBuffer, length isn’t clamped to the available buffer size.

Buffer Builders¶

class arrow::BufferBuilder¶

A class for incrementally building a contiguous chunk of in-memory data.

Public Functions

inline explicit BufferBuilder(std::shared_ptr<ResizableBuffer> buffer, MemoryPool *pool = default_memory_pool())¶

Constructs new Builder that will start using the provided buffer until Finish/Reset are called.

The buffer is not resized.

inline Status Resize(const int64_t new_capacity, bool shrink_to_fit = true)¶

Resize the buffer to the nearest multiple of 64 bytes.

Parameters

new_capacity – the new capacity of the of the builder. Will be rounded up to a multiple of 64 bytes for padding
shrink_to_fit – if new capacity is smaller than the existing, reallocate internal buffer. Set to false to avoid reallocations when shrinking the builder.

Returns

Status

inline Status Reserve(const int64_t additional_bytes)¶

Ensure that builder can accommodate the additional number of bytes without the need to perform allocations.

Parameters: [in] additional_bytes – number of additional bytes to make space for
Returns: Status

inline Status Append(const void *data, const int64_t length)¶

Append the given data to the buffer.

The buffer is automatically expanded if necessary.

inline Status Append(const int64_t num_copies, uint8_t value)¶

Append copies of a value to the buffer.

The buffer is automatically expanded if necessary.

inline Status Finish(std::shared_ptr<Buffer> *out, bool shrink_to_fit = true)¶

Return result of builder as a Buffer object.

The builder is reset and can be reused afterwards.

Parameters

[out] out – the finalized Buffer object
shrink_to_fit – if the buffer size is smaller than its capacity, reallocate to fit more tightly in memory. Set to false to avoid a reallocation, at the expense of potentially more memory consumption.

Returns

Status

inline Result<std::shared_ptr<Buffer>> FinishWithLength(int64_t final_length, bool shrink_to_fit = true)¶

Like Finish, but override the final buffer size.

This is useful after writing data directly into the builder memory without calling the Append methods (basically, when using BufferBuilder mostly for memory allocation).

inline void Rewind(int64_t position)¶

Set size to a smaller value without modifying builder contents.

For reusable BufferBuilder classes

Parameters: [in] position – must be non-negative and less than or equal to the current length()

Public Static Functions

static inline int64_t GrowByFactor(int64_t current_capacity, int64_t new_capacity)¶: Return a capacity expanded by the desired growth factor.

template<typename T, typename Enable = void> class TypedBufferBuilder¶

STL Integration¶

template<class T> class arrow::stl::allocator¶

A STL allocator delegating allocations to a Arrow MemoryPool.

Public Functions

inline allocator() noexcept¶: Construct an allocator from the default MemoryPool.

inline explicit allocator(MemoryPool *pool) noexcept¶: Construct an allocator from the given MemoryPool.

template<class U> struct rebind¶

template<typename Allocator = std::allocator<uint8_t>> class arrow::stl::STLMemoryPool : public arrow::MemoryPool ¶

A MemoryPool implementation delegating allocations to a STL allocator.

Note that STL allocators don’t provide a resizing operation, and therefore any buffer resizes will do a full reallocation and copy.

Public Functions

inline explicit STLMemoryPool(const Allocator &alloc)¶: Construct a memory pool from the given allocator.

inline virtual Status Allocate(int64_t size, uint8_t **out) override¶

Allocate a new memory region of at least size bytes.

The allocated region shall be 64-byte aligned.

inline virtual Status Reallocate(int64_t old_size, int64_t new_size, uint8_t **ptr) override¶

Resize an already allocated memory section.

As by default most default allocators on a platform don’t support aligned reallocation, this function can involve a copy of the underlying data.

inline virtual void Free(uint8_t *buffer, int64_t size) override¶

Free an allocated region.

Parameters

buffer – Pointer to the start of the allocated memory region
size – Allocated size located at buffer. An allocator implementation may use this for tracking the amount of allocated bytes as well as for faster deallocation if supported by its backend.

inline virtual int64_t bytes_allocated() const override¶: The number of bytes that were allocated and not yet free’d through this allocator.

inline virtual int64_t max_memory() const override¶

Return peak memory allocation in this memory pool.

Returns: Maximum bytes allocated. If not known (or not implemented), returns -1

inline virtual std::string backend_name() const override¶: The name of the backend used by this MemoryPool (e.g. “system” or “jemalloc”).

Programming Support Data Types