Module statistics 

StatisticsConverter to convert statistics in parquet format to arrow [ArrayRef].

Re-exports

use crate::arrow::buffer::bit_util::sign_extend_be; 🔒

use crate::arrow::parquet_column; 🔒

use crate::basic::ColumnOrder; 🔒

use crate::basic::Type as PhysicalType; 🔒

use crate::data_type::Int96; 🔒

use crate::errors::ParquetError; 🔒

use crate::errors::Result; 🔒

use crate::file::metadata::RowGroupMetaData; 🔒

use crate::file::metadata::page_index::PageIndexProvider; 🔒

use crate::file::page_index::column_index::ColumnIndexMetaData; 🔒

use crate::file::statistics::Statistics as ParquetStatistics; 🔒

use crate::schema::types::SchemaDescriptor; 🔒

use arrow_array::builder::BinaryBuilder; 🔒

use arrow_array::builder::BinaryViewBuilder; 🔒

use arrow_array::builder::BooleanBuilder; 🔒

use arrow_array::builder::Date32Builder; 🔒

use arrow_array::builder::Date64Builder; 🔒

use arrow_array::builder::Decimal32Builder; 🔒

use arrow_array::builder::Decimal64Builder; 🔒

use arrow_array::builder::FixedSizeBinaryBuilder; 🔒

use arrow_array::builder::Float16Builder; 🔒

use arrow_array::builder::Float32Builder; 🔒

use arrow_array::builder::Float64Builder; 🔒

use arrow_array::builder::Int8Builder; 🔒

use arrow_array::builder::Int16Builder; 🔒

use arrow_array::builder::Int32Builder; 🔒

use arrow_array::builder::Int64Builder; 🔒

use arrow_array::builder::LargeBinaryBuilder; 🔒

use arrow_array::builder::LargeStringBuilder; 🔒

use arrow_array::builder::StringBuilder; 🔒

use arrow_array::builder::StringViewBuilder; 🔒

use arrow_array::builder::Time32MillisecondBuilder; 🔒

use arrow_array::builder::Time32SecondBuilder; 🔒

use arrow_array::builder::Time64MicrosecondBuilder; 🔒

use arrow_array::builder::Time64NanosecondBuilder; 🔒

use arrow_array::builder::TimestampMicrosecondBuilder; 🔒

use arrow_array::builder::TimestampMillisecondBuilder; 🔒

use arrow_array::builder::TimestampNanosecondBuilder; 🔒

use arrow_array::builder::TimestampSecondBuilder; 🔒

use arrow_array::builder::UInt8Builder; 🔒

use arrow_array::builder::UInt16Builder; 🔒

use arrow_array::builder::UInt32Builder; 🔒

use arrow_array::builder::UInt64Builder; 🔒

use arrow_array::ArrayRef; 🔒

use arrow_array::BinaryArray; 🔒

use arrow_array::BooleanArray; 🔒

use arrow_array::Date32Array; 🔒

use arrow_array::Date64Array; 🔒

use arrow_array::Decimal32Array; 🔒

use arrow_array::Decimal64Array; 🔒

use arrow_array::Decimal128Array; 🔒

use arrow_array::Decimal256Array; 🔒

use arrow_array::Float16Array; 🔒

use arrow_array::Float32Array; 🔒

use arrow_array::Float64Array; 🔒

use arrow_array::Int8Array; 🔒

use arrow_array::Int16Array; 🔒

use arrow_array::Int32Array; 🔒

use arrow_array::Int64Array; 🔒

use arrow_array::LargeBinaryArray; 🔒

use arrow_array::Time32MillisecondArray; 🔒

use arrow_array::Time32SecondArray; 🔒

use arrow_array::Time64MicrosecondArray; 🔒

use arrow_array::Time64NanosecondArray; 🔒

use arrow_array::TimestampMicrosecondArray; 🔒

use arrow_array::TimestampMillisecondArray; 🔒

use arrow_array::TimestampNanosecondArray; 🔒

use arrow_array::TimestampSecondArray; 🔒

use arrow_array::UInt8Array; 🔒

use arrow_array::UInt16Array; 🔒

use arrow_array::UInt32Array; 🔒

use arrow_array::UInt64Array; 🔒

use arrow_array::new_null_array; 🔒

use arrow_buffer::NullBufferBuilder; 🔒

use arrow_buffer::i256; 🔒

use arrow_schema::DataType; 🔒

use arrow_schema::Field; 🔒

use arrow_schema::Schema; 🔒

use arrow_schema::TimeUnit; 🔒

use half::f16; 🔒

use std::sync::Arc; 🔒

Macros

get_data_page_statistics 🔒

get_statistics 🔒

Special macro to combine the statistics iterators for min and max. This is used to avoid repeating the same code for min and max statistics extractions

make_decimal_stats_iterator 🔒

Special iterator adapter for extracting i128 values from an iterator of ParquetStatistics

make_stats_iterator 🔒

Define an adapter iterator for extracting statistics from an iterator of ParquetStatistics

make_timestamp_stats_iterator 🔒

Extract timestamp bounds from INT64 or non-deprecated INT96 statistics.

Structs

MaxBooleanStatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MaxByteArrayStatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MaxDecimal32StatsIterator 🔒

MaxDecimal64StatsIterator 🔒

MaxDecimal128StatsIterator 🔒

MaxDecimal256StatsIterator 🔒

MaxDoubleStatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MaxFixedLenByteArrayStatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MaxFloatStatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MaxInt32StatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MaxInt64StatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MaxTimestampStatsIterator 🔒

MinBooleanStatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MinByteArrayStatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MinDecimal32StatsIterator 🔒

MinDecimal64StatsIterator 🔒

MinDecimal128StatsIterator 🔒

MinDecimal256StatsIterator 🔒

MinDoubleStatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MinFixedLenByteArrayStatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MinFloatStatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MinInt32StatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MinInt64StatsIterator 🔒

Maps an iterator of ParquetStatistics into an iterator of &$stat_value_type

MinTimestampStatsIterator 🔒

StatisticsConverter

Extracts Parquet statistics as Arrow arrays

Functions

from_bytes_to_f16 🔒

from_bytes_to_i32 🔒

from_bytes_to_i64 🔒

from_bytes_to_i128 🔒

from_bytes_to_i256 🔒

int96_statistics 🔒

Convert both bounds together: if either wraps in the reader’s timestamp unit, neither bound can safely describe the decoded values.

max_page_statistics 🔒

Extracts the max statistics from an iterator of parquet page ColumnIndexMetaData’s to an [ArrayRef]

max_statistics 🔒

Extracts the max statistics from an iterator of ParquetStatistics to an [ArrayRef]

min_page_statistics 🔒

Extracts the min statistics from an iterator of parquet page ColumnIndexMetaData’s to an [ArrayRef]

min_statistics 🔒

Extracts the min statistics from an iterator of ParquetStatistics to an [ArrayRef]

nan_counts_page_statistics 🔒

Extracts the NaN count statistics from an iterator of parquet page ColumnIndexMetaData’s to an [ArrayRef]

null_counts_page_statistics 🔒

Extracts the null count statistics from an iterator of parquet page ColumnIndexMetaData’s to an [ArrayRef]