Enum LengthTracker

pub(crate) enum LengthTracker {
    Fixed {
        length: usize,
        num_rows: usize,
    },
    Variable {
        fixed_length: usize,
        lengths: Vec<usize>,
    },
}

Stores the lengths of the rows. Lazily materializes lengths for columns with fixed-size types.

Variants

Fixed

Fixed state: All rows have length length

Fields

length: usize

num_rows: usize

Variable

Variable state: The length of row i is lengths[i] + fixed_length

Fields

fixed_length: usize

lengths: Vec<usize>

Implementations

impl LengthTracker

pub(crate) fn new(num_rows: usize) -> Self

pub(crate) fn push_fixed(&mut self, new_length: usize)

Adds a column of fixed-length elements, each of size new_length to the LengthTracker

pub(crate) fn push_variable( &mut self, new_lengths: impl ExactSizeIterator<Item = usize>, )

Adds a column of possibly variable-length elements, element i has length new_lengths.nth(i)

pub(crate) fn materialized(&mut self) -> &mut [usize]

Returns the tracked row lengths as a slice

pub(crate) fn extend_offsets( &self, initial_offset: usize, offsets: &mut Vec<usize>, ) -> usize

Initializes the offsets using the tracked lengths. Returns the sum of the lengths of the rows added.

We initialize the offsets shifted down by one row index.

As the rows are appended to the offsets will be incremented to match

For example, consider the case of 3 rows of length 3, 4, and 6 respectively. The offsets would be initialized to 0, 0, 3, 7

Writing the first row entirely would yield 0, 3, 3, 7 The second, 0, 3, 7, 7 The third, 0, 3, 7, 13 This would be the final offsets for reading In this way offsets tracks the position during writing whilst eventually serving