Enum SubBatchStrategy 

pub(crate) enum SubBatchStrategy {
    Values(usize),
    Levels {
        levels: usize,
        max_values: usize,
    },
}

How write_granular_chunk should cut mini-batch windows in one chunk.

Cutting on exact value counts is the precise option and the expensive one: it roughly doubles the number of mini-batches on a nullable column, because the level:value ratio no longer rounds a window up to cover a second value. It is used only where that precision buys something.

Variants

Values(usize)

Cut after exactly this many values, walking definition levels to find the boundary.

Used against the data page budget for encodings that compress a value against its predecessor, where the round-up below costs whole values of output: 16 MiB rather than 2 MiB for 128 values at one null in 16. See #10538 for the measurements.

Levels

Cut after this many levels: the value budget scaled by the chunk’s level:value ratio, rounded up.

Used everywhere else, because everywhere else the round-up changes no bytes — the dictionary page budget shrinks toward zero as the dictionary fills, so a one-value budget is routine there for ordinary values, and PLAIN and DELTA_LENGTH_BYTE_ARRAY values cost the same wherever they land — while value-exact windows cost throughput (#10554).

The round-up must stay bounded for that price to be acceptable. The level count alone does not bound it: the ratio is a chunk average, so a window over a run of clustered values can hold far more than the budget (28 values of 76 MiB followed by 996 nulls all land in a 37-level window, #11994). Each window is therefore also capped at max_values values, twice the value budget, so the page bound #9972 added holds whatever the null layout: two values per page rather than one. The cap walks the window’s definition levels, but adds no mini-batches where nulls are evenly spread.

Fields

levels: usize

max_values: usize

Trait Implementations

impl Clone for SubBatchStrategy

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for SubBatchStrategy

impl Debug for SubBatchStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for SubBatchStrategy

impl PartialEq for SubBatchStrategy

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for SubBatchStrategy