Module page_store 

Pluggable storage for completed, serialized page blobs.

While a row group is being written the ArrowWriter must buffer every column’s encoded pages, because Parquet requires each column chunk to be contiguous in the file while record batches arrive with all columns interleaved. By default that buffer lives on the heap, so the writer’s peak memory grows with the row group size. A PageStore lets the buffer live somewhere else — a local temp file, object storage, etc. — bounding peak write memory independently of the row group size.

Structs

InMemoryPageStore

The default PageStore, holding blobs on the heap in a Vec<Bytes>.

InMemoryPageStoreFactory

Factory for InMemoryPageStore — the default used by ArrowWriter.

PageKey

An opaque, store-allocated handle to a blob held by a PageStore.

PageStoreArgs

Context for a single PageStoreFactory::create call.

Traits

PageStore

A pluggable store for completed, serialized page blobs.

PageStoreFactory

Creates a fresh PageStore for each column chunk.