Skip to main content

ColumnWriterImpl

parquet::column::writer

Type Alias ColumnWriterImpl 

Source 
pub type ColumnWriterImpl<'a, T> = GenericColumnWriter<'a, ColumnValueEncoderImpl<T>>;
Expand description

Typed column writer for a primitive column.

Aliased Type§

pub struct ColumnWriterImpl<'a, T> {
Show 20 fields
    descr: Arc<ColumnDescriptor>,
    props: Arc<WriterProperties>,
    column_props: ResolvedColumnProperties,
    page_writer: Box<dyn PageWriter + 'a>,
    codec: Compression,
    compressor: Option<Box<dyn Codec>>,
    encoder: ColumnValueEncoderImpl<T>,
    page_metrics: PageMetrics,
    column_metrics: ColumnMetrics<<ColumnValueEncoderImpl<T> as ColumnValueEncoder>::T>,
    distinct_count_override: Option<u64>,
    encodings: BTreeSet<Encoding>,
    encoding_stats: Vec<PageEncodingStats>,
    def_levels_encoder: LevelEncoder,
    rep_levels_encoder: LevelEncoder,
    data_pages: VecDeque<CompressedPage>,
    column_index_builder: ColumnIndexBuilder,
    offset_index_builder: Option<OffsetIndexBuilder>,
    data_page_boundary_ascending: bool,
    data_page_boundary_descending: bool,
    last_non_null_data_page_min_max: Option<(<ColumnValueEncoderImpl<T> as ColumnValueEncoder>::T, <ColumnValueEncoderImpl<T> as ColumnValueEncoder>::T)>,

}

Fields§

§descr: Arc<ColumnDescriptor>§props: Arc<WriterProperties>§column_props: ResolvedColumnProperties

Per-column settings for Self::descr, resolved once here so that the per-batch and per-page write paths never search the per-column override map in props again.

§page_writer: Box<dyn PageWriter + 'a>§codec: Compression§compressor: Option<Box<dyn Codec>>§encoder: ColumnValueEncoderImpl<T>§page_metrics: PageMetrics§column_metrics: ColumnMetrics<<ColumnValueEncoderImpl<T> as ColumnValueEncoder>::T>§distinct_count_override: Option<u64>

Pre-computed distinct count to write into column chunk statistics. When set, takes precedence over column_metrics.column_distinct_count.

§encodings: BTreeSet<Encoding>

The order of encodings within the generated metadata does not impact its meaning, but we use a BTreeSet so that the output is deterministic

§encoding_stats: Vec<PageEncodingStats>§def_levels_encoder: LevelEncoder§rep_levels_encoder: LevelEncoder§data_pages: VecDeque<CompressedPage>§column_index_builder: ColumnIndexBuilder§offset_index_builder: Option<OffsetIndexBuilder>§data_page_boundary_ascending: bool§data_page_boundary_descending: bool§last_non_null_data_page_min_max: Option<(<ColumnValueEncoderImpl<T> as ColumnValueEncoder>::T, <ColumnValueEncoderImpl<T> as ColumnValueEncoder>::T)>

(min, max)