pub struct DeltaBinaryPackedEncoderOptions {
block_size: usize,
mini_blocks_per_block: usize,
}Expand description
Controls the block layout emitted by [DeltaBitPackEncoder] and the integer sub-encoders used
by [DeltaLengthByteArrayEncoder] and [DeltaByteArrayEncoder].
In the Parquet DELTA_BINARY_PACKED encoding, each block records a minimum delta and each mini
block records its own bit width. The defaults use 128 values per block and 4 mini blocks for
INT32, and 256 values per block and 4 mini blocks for INT64.
Smaller blocks or mini blocks can reduce encoded size for bursty data, where the required bit width changes over short ranges, and for short pages that would otherwise need more padding. They also add more minimum-delta and bit-width metadata. Larger blocks or mini blocks can reduce that metadata and block-processing overhead for long inputs with stable delta distributions, improving compression or throughput, but an outlier then affects more values and the encoder uses a larger scratch buffer. Benchmark representative data before changing the defaults.
The format requires block sizes to be multiples of 128 and mini block sizes to be multiples of 32.
Fields§
§block_size: usize§mini_blocks_per_block: usizeImplementations§
Source§impl DeltaBinaryPackedEncoderOptions
impl DeltaBinaryPackedEncoderOptions
Sourcepub fn try_new(block_size: usize, mini_blocks_per_block: usize) -> Result<Self>
pub fn try_new(block_size: usize, mini_blocks_per_block: usize) -> Result<Self>
Creates validated delta binary packed encoder options.
Sourcepub fn block_size(&self) -> usize
pub fn block_size(&self) -> usize
Returns the number of values in each block.
Sourcepub fn mini_blocks_per_block(&self) -> usize
pub fn mini_blocks_per_block(&self) -> usize
Returns the number of mini blocks in each block.
Trait Implementations§
impl Copy for DeltaBinaryPackedEncoderOptions
impl Eq for DeltaBinaryPackedEncoderOptions
impl StructuralPartialEq for DeltaBinaryPackedEncoderOptions
Auto Trait Implementations§
impl Freeze for DeltaBinaryPackedEncoderOptions
impl RefUnwindSafe for DeltaBinaryPackedEncoderOptions
impl Send for DeltaBinaryPackedEncoderOptions
impl Sync for DeltaBinaryPackedEncoderOptions
impl Unpin for DeltaBinaryPackedEncoderOptions
impl UnsafeUnpin for DeltaBinaryPackedEncoderOptions
impl UnwindSafe for DeltaBinaryPackedEncoderOptions
Blanket Implementations§
impl<T> Allocation for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key and return true if they are equal.