Skip to main content

DeltaBinaryPackedEncoderOptions

Struct DeltaBinaryPackedEncoderOptions 

Source
pub struct DeltaBinaryPackedEncoderOptions {
    block_size: usize,
    mini_blocks_per_block: usize,
}
Expand description

Controls the block layout emitted by [DeltaBitPackEncoder] and the integer sub-encoders used by [DeltaLengthByteArrayEncoder] and [DeltaByteArrayEncoder].

In the Parquet DELTA_BINARY_PACKED encoding, each block records a minimum delta and each mini block records its own bit width. The defaults use 128 values per block and 4 mini blocks for INT32, and 256 values per block and 4 mini blocks for INT64.

Smaller blocks or mini blocks can reduce encoded size for bursty data, where the required bit width changes over short ranges, and for short pages that would otherwise need more padding. They also add more minimum-delta and bit-width metadata. Larger blocks or mini blocks can reduce that metadata and block-processing overhead for long inputs with stable delta distributions, improving compression or throughput, but an outlier then affects more values and the encoder uses a larger scratch buffer. Benchmark representative data before changing the defaults.

The format requires block sizes to be multiples of 128 and mini block sizes to be multiples of 32.

Fields§

§block_size: usize§mini_blocks_per_block: usize

Implementations§

Source§

impl DeltaBinaryPackedEncoderOptions

Source

pub fn try_new(block_size: usize, mini_blocks_per_block: usize) -> Result<Self>

Creates validated delta binary packed encoder options.

Source

pub fn block_size(&self) -> usize

Returns the number of values in each block.

Source

pub fn mini_blocks_per_block(&self) -> usize

Returns the number of mini blocks in each block.

Trait Implementations§

Source§

impl Clone for DeltaBinaryPackedEncoderOptions

Source§

fn clone(&self) -> Self

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Copy for DeltaBinaryPackedEncoderOptions

Source§

impl Debug for DeltaBinaryPackedEncoderOptions

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Eq for DeltaBinaryPackedEncoderOptions

Source§

impl PartialEq for DeltaBinaryPackedEncoderOptions

Source§

fn eq(&self, other: &Self) -> bool

Equality operator ==. Read more
1.0.0 (const: unstable) · Source§

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=. Read more
Source§

impl StructuralPartialEq for DeltaBinaryPackedEncoderOptions

Auto Trait Implementations§

Blanket Implementations§

§

impl<T> Allocation for T
where T: RefUnwindSafe + Send + Sync,

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

§

fn equivalent(&self, key: &K) -> bool

Checks if this value is equivalent to the given key. Read more
§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

§

fn equivalent(&self, key: &K) -> bool

Compare self to key and return true if they are equal.
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<T> Ungil for T
where T: Send,