Enum Statistics

pub enum Statistics {
    Boolean(ValueStatistics<bool>),
    Int32(ValueStatistics<i32>),
    Int64(ValueStatistics<i64>),
    Int96(ValueStatistics<Int96>),
    Float(ValueStatistics<f32>),
    Double(ValueStatistics<f64>),
    ByteArray(ValueStatistics<ByteArray>),
    FixedLenByteArray(ValueStatistics<FixedLenByteArray>),
}

Strongly typed statistics for a column chunk within a row group.

This structure is a natively typed, in memory representation of the Statistics structure in a parquet file footer. The statistics stored in this structure can be used by query engines to skip decoding pages while reading parquet data.

Page level statistics are stored separately, in NativeIndex.

Variants

Boolean(ValueStatistics<bool>)

Statistics for Boolean column

Int32(ValueStatistics<i32>)

Statistics for Int32 column

Int64(ValueStatistics<i64>)

Statistics for Int64 column

Int96(ValueStatistics<Int96>)

Statistics for Int96 column

Float(ValueStatistics<f32>)

Statistics for Float column

Double(ValueStatistics<f64>)

Statistics for Double column

ByteArray(ValueStatistics<ByteArray>)

Statistics for ByteArray column

FixedLenByteArray(ValueStatistics<FixedLenByteArray>)

Statistics for FixedLenByteArray column

Implementations

impl Statistics

pub fn new<T: ParquetValueType>( min: Option<T>, max: Option<T>, distinct_count: Option<u64>, null_count: Option<u64>, is_deprecated: bool, ) -> Self

Creates new statistics for a column type

pub fn boolean( min: Option<bool>, max: Option<bool>, distinct: Option<u64>, nulls: Option<u64>, is_deprecated: bool, ) -> Self

Creates new statistics for Boolean column type.

pub fn int32( min: Option<i32>, max: Option<i32>, distinct: Option<u64>, nulls: Option<u64>, is_deprecated: bool, ) -> Self

Creates new statistics for Int32 column type.

pub fn int64( min: Option<i64>, max: Option<i64>, distinct: Option<u64>, nulls: Option<u64>, is_deprecated: bool, ) -> Self

Creates new statistics for Int64 column type.

pub fn int96( min: Option<Int96>, max: Option<Int96>, distinct: Option<u64>, nulls: Option<u64>, is_deprecated: bool, ) -> Self

Creates new statistics for Int96 column type.

pub fn float( min: Option<f32>, max: Option<f32>, distinct: Option<u64>, nulls: Option<u64>, is_deprecated: bool, ) -> Self

Creates new statistics for Float column type.

pub fn double( min: Option<f64>, max: Option<f64>, distinct: Option<u64>, nulls: Option<u64>, is_deprecated: bool, ) -> Self

Creates new statistics for Double column type.

pub fn byte_array( min: Option<ByteArray>, max: Option<ByteArray>, distinct: Option<u64>, nulls: Option<u64>, is_deprecated: bool, ) -> Self

Creates new statistics for ByteArray column type.

pub fn fixed_len_byte_array( min: Option<FixedLenByteArray>, max: Option<FixedLenByteArray>, distinct: Option<u64>, nulls: Option<u64>, is_deprecated: bool, ) -> Self

Creates new statistics for FixedLenByteArray column type.

pub fn is_min_max_deprecated(&self) -> bool

Returns true if statistics have old min and max fields set. This means that the column order is likely to be undefined, which, for old files could mean a signed sort order of values.

Refer to ColumnOrder and SortOrder for more information.

pub fn is_min_max_backwards_compatible(&self) -> bool

Old versions of parquet stored statistics in min and max fields, ordered using signed comparison. This resulted in an undefined ordering for unsigned quantities, such as booleans and unsigned integers.

These fields were therefore deprecated in favour of min_value and max_value, which have a type-defined sort order.

However, not all readers have been updated. For backwards compatibility, this method returns true if the statistics within this have a signed sort order, that is compatible with being stored in the deprecated min and max fields

pub fn distinct_count(&self) -> Option<u64>

👎Deprecated since 53.0.0: Use distinct_count_opt method instead

Returns optional value of number of distinct values occurring. When it is None, the value should be ignored.

pub fn distinct_count_opt(&self) -> Option<u64>

Returns optional value of number of distinct values occurring. When it is None, the value should be ignored.

pub fn null_count(&self) -> u64

👎Deprecated since 53.0.0: Use null_count_opt method instead

Returns number of null values for the column. Note that this includes all nulls when column is part of the complex type.

Note this API returns 0 if the null count is not available.

pub fn has_nulls(&self) -> bool

👎Deprecated since 53.0.0: Use null_count_opt method instead

Returns true if statistics collected any null values, false otherwise.

pub fn null_count_opt(&self) -> Option<u64>

Returns number of null values for the column, if known. Note that this includes all nulls when column is part of the complex type.

Note this API returns Some(0) even if the null count was not present in the statistics. See https://github.com/apache/arrow-rs/pull/6216/files

pub fn has_min_max_set(&self) -> bool

👎Deprecated since 53.0.0: Use min_bytes_opt and max_bytes_opt methods instead

Whether or not min and max values are set. Normally both min/max values will be set to Some(value) or None.

pub fn min_is_exact(&self) -> bool

Returns true if the min value is set, and is an exact min value.

pub fn max_is_exact(&self) -> bool

Returns true if the max value is set, and is an exact max value.

pub fn min_bytes_opt(&self) -> Option<&[u8]>

Returns slice of bytes that represent min value, if min value is known.

pub fn min_bytes(&self) -> &[u8]

👎Deprecated since 53.0.0: Use max_bytes_opt instead

Returns slice of bytes that represent min value. Panics if min value is not set.

pub fn max_bytes_opt(&self) -> Option<&[u8]>

Returns slice of bytes that represent max value, if max value is known.

pub fn max_bytes(&self) -> &[u8]

👎Deprecated since 53.0.0: Use max_bytes_opt instead

Returns slice of bytes that represent max value. Panics if max value is not set.

pub fn physical_type(&self) -> Type

Returns physical type associated with statistics.

Trait Implementations

impl Clone for Statistics

fn clone(&self) -> Statistics

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Statistics

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Statistics

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T: ParquetValueType> From<ValueStatistics<T>> for Statistics

fn from(t: ValueStatistics<T>) -> Self

Converts to this type from the input type.

impl HeapSize for Statistics

fn heap_size(&self) -> usize

Return the size of any bytes allocated on the heap by this object, including heap memory in those structures

impl PartialEq for Statistics

fn eq(&self, other: &Statistics) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Statistics