Module thrift 

This module is the bridge between a Parquet file’s thrift encoded metadata and this crate’s Parquet metadata API. It contains objects and functions used to serialize/deserialize metadata objects into/from the Thrift compact protocol format as defined by the Parquet specification.

Re-exports

use std::io::Write; 🔒

use std::sync::Arc; 🔒

use crate::file::column_crypto_metadata::ColumnCryptoMetaData; 🔒

use crate::file::metadata::thrift::encryption::EncryptionAlgorithm; 🔒

use crate::basic::ColumnOrder; 🔒

use crate::basic::CompressionCodec; 🔒

use crate::basic::ConvertedType; 🔒

use crate::basic::Encoding; 🔒

use crate::basic::EncodingMask; 🔒

use crate::basic::LogicalType; 🔒

use crate::basic::PageType; 🔒

use crate::basic::Repetition; 🔒

use crate::basic::Type; 🔒

use crate::data_type::ByteArray; 🔒

use crate::data_type::FixedLenByteArray; 🔒

use crate::data_type::Int96; 🔒

use crate::errors::ParquetError; 🔒

use crate::errors::Result; 🔒

use crate::file::metadata::ColumnChunkMetaData; 🔒

use crate::file::metadata::ColumnChunkMetaDataBuilder; 🔒

use crate::file::metadata::FileMetaData; 🔒

use crate::file::metadata::KeyValue; 🔒

use crate::file::metadata::LevelHistogram; 🔒

use crate::file::metadata::PageEncodingStats; 🔒

use crate::file::metadata::ParquetMetaData; 🔒

use crate::file::metadata::ParquetMetaDataOptions; 🔒

use crate::file::metadata::ParquetPageEncodingStats; 🔒

use crate::file::metadata::RowGroupMetaData; 🔒

use crate::file::metadata::RowGroupMetaDataBuilder; 🔒

use crate::file::metadata::SortingColumn; 🔒

use crate::file::statistics::ValueStatistics; 🔒

use crate::parquet_thrift::ElementType; 🔒

use crate::parquet_thrift::FieldType; 🔒

use crate::parquet_thrift::ReadThrift; 🔒

use crate::parquet_thrift::ThriftCompactInputProtocol; 🔒

use crate::parquet_thrift::ThriftCompactOutputProtocol; 🔒

use crate::parquet_thrift::ThriftSliceInputProtocol; 🔒

use crate::parquet_thrift::WriteThrift; 🔒

use crate::parquet_thrift::WriteThriftField; 🔒

use crate::parquet_thrift::check_list_size; 🔒

use crate::parquet_thrift::read_thrift_vec; 🔒

use crate::parquet_thrift::validate_list_type; 🔒

use crate::schema::types::ColumnDescriptor; 🔒

use crate::schema::types::SchemaDescriptor; 🔒

use crate::schema::types::TypePtr; 🔒

use crate::schema::types::num_nodes; 🔒

use crate::schema::types::parquet_schema_from_array; 🔒

use crate::util::bit_util::FromBytes; 🔒

use crate::geospatial::statistics::GeospatialStatistics as RustGeospatialStatistics; 🔒

use crate::geospatial::bounding_box::BoundingBox as RustBoundingBox; 🔒

Modules

encryption 🔒

Encryption support for Thrift serialization

Structs

BoundingBox 🔒

DataPageHeader 🔒

DataPageHeaderV2 🔒

DictionaryPageHeader 🔒

FileMeta 🔒

GeospatialStatistics 🔒

IndexPageHeader 🔒

PageHeader 🔒

PageStatistics 🔒

Statistics for the page header.

SchemaElement 🔒

SizeStatistics 🔒

Statistics 🔒

Constants

COL_META_ALL_REQUIRED 🔒

COL_META_CODEC 🔒

COL_META_DATA_PAGE_OFFSET 🔒

COL_META_ENCODINGS 🔒

COL_META_NUM_VALUES 🔒

COL_META_TOTAL_COMP_SZ 🔒

COL_META_TOTAL_UNCOMP_SZ 🔒

COL_META_TYPE 🔒

Functions

convert_bounding_box 🔒

convert_geo_stats 🔒

convert_stats 🔒

Create a crate::file::statistics::Statistics from a thrift Statistics object.

ensure_row_group_ordinals 🔒

Ensure RowGroupMetaData::ordinal is usable after decode without rejecting spec-valid files (RowGroup.ordinal is optional in the parquet-format Thrift definition, with no uniformity requirement):

file_metadata_and_row_groups_from_bytes 🔒

parquet_metadata_from_bytes 🔒

Create ParquetMetaData from thrift input. Note that this only decodes the file metadata in the Parquet footer. Page indexes will need to be added later.

parquet_schema_from_bytes 🔒

Create a SchemaDescriptor from thrift input. The input buffer must contain a complete Parquet footer.

read_column_chunk 🔒

read_column_metadata 🔒

read_encoding_stats_as_mask 🔒

read_row_group 🔒

serialize_column_meta_data 🔒

should_write_column_stats 🔒

validate_column_metadata 🔒

write_schema 🔒

write_schema_helper 🔒