Enum SubBatchStrategy 

pub(crate) enum SubBatchStrategy {
    Values(usize),
    Levels(usize),
}

How write_granular_chunk should cut mini-batch windows in one chunk.

Cutting on exact value counts is the precise option and the expensive one: it roughly doubles the number of mini-batches on a nullable column, because the level:value ratio no longer rounds a window up to cover a second value. It is used only where that precision buys something.

Variants

Values(usize)

Cut after exactly this many values, walking definition levels to find the boundary.

Used against the data page budget for encodings that compress a value against its predecessor, where the round-up below costs whole values of output: 16 MiB rather than 2 MiB for 128 values at one null in 16. See #10538 for the measurements.

Levels(usize)

Cut after this many levels: the value budget scaled by the chunk’s level:value ratio, rounded up.

Used everywhere else, because everywhere else the round-up changes no bytes — the dictionary page budget shrinks toward zero as the dictionary fills, so a one-value budget is routine there for ordinary values, and PLAIN and DELTA_LENGTH_BYTE_ARRAY values cost the same wherever they land — while value-exact windows cost throughput (#10554).

The round-up is bounded, which is what makes it an acceptable price. A window spans ceil(values * levels / values_in_chunk) levels, so where one value already fills the budget it covers at most two values, whatever the null density. The page bound #9972 added therefore still holds; it is two values per page rather than one.

Trait Implementations

impl Clone for SubBatchStrategy

fn clone(&self) -> SubBatchStrategy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for SubBatchStrategy

impl Debug for SubBatchStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for SubBatchStrategy

impl PartialEq for SubBatchStrategy

fn eq(&self, other: &SubBatchStrategy) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for SubBatchStrategy