Module writer 

SerializedFileWriter: Low level Parquet writer API

Re-exports

use crate::bloom_filter::Sbbf; 🔒

use crate::file::metadata::page_index::PageIndex; 🔒

use crate::file::metadata::thrift::PageHeader; 🔒

use crate::file::page_index::column_index::ColumnIndexMetaData; 🔒

use crate::file::page_index::offset_index::OffsetIndexMetaData; 🔒

use crate::parquet_thrift::ThriftCompactOutputProtocol; 🔒

use crate::parquet_thrift::WriteThrift; 🔒

use bytes::Bytes; 🔒

use std::fmt::Debug; 🔒

use std::fmt::Debug; 🔒

use std::io::BufWriter; 🔒

use std::io::IoSlice; 🔒

use std::io::Read; 🔒

use std::io::Write; 🔒

use std::sync::Arc; 🔒

use crate::column::page_encryption::PageEncryptor; 🔒

use crate::column::writer::ColumnCloseResult; 🔒

use crate::column::writer::ColumnWriterImpl; 🔒

use crate::column::writer::get_typed_column_writer_mut; 🔒

use crate::column::page::CompressedPage; 🔒

use crate::column::page::PageWriteSpec; 🔒

use crate::column::page::PageWriter; 🔒

use crate::column::writer::ColumnWriter; 🔒

use crate::column::writer::get_column_writer; 🔒

use crate::data_type::DataType; 🔒

use crate::encryption::encrypt::FileEncryptionProperties; 🔒

use crate::encryption::encrypt::FileEncryptor; 🔒

use crate::encryption::encrypt::get_column_crypto_metadata; 🔒

use crate::errors::ParquetError; 🔒

use crate::errors::Result; 🔒

use crate::file::PARQUET_MAGIC_ENCR_FOOTER; 🔒

use crate::file::properties::BloomFilterPosition; 🔒

use crate::file::properties::WriterPropertiesPtr; 🔒

use crate::file::reader::ChunkReader; 🔒

use crate::file::PARQUET_MAGIC; 🔒

use crate::schema::types::ColumnDescPtr; 🔒

use crate::schema::types::SchemaDescPtr; 🔒

use crate::schema::types::SchemaDescriptor; 🔒

use crate::schema::types::TypePtr; 🔒

use crate::file::metadata::*; 🔒

Structs

PageEncryptorContext 🔒

Context required to create a PageEncryptor for a column

SerializedColumnWriter

A wrapper around a ColumnWriter that invokes a callback on Self::close

SerializedFileWriter

Parquet file writer API.

SerializedPageWriter

A serialized implementation for Parquet PageWriter. Writes and serializes pages and metadata into output stream.

SerializedRowGroupWriter

Parquet row group writer API.

TrackedWrite

A wrapper around a Write that keeps track of the number of bytes that have been written. The given Write is wrapped with a BufWriter to optimize writing performance.

Functions

get_file_magic 🔒

Get the magic bytes at the start and end of the file that identify this as a Parquet file.

write_bloom_filters 🔒

Serialize all the bloom filters of the given row group to the given buffer, and returns the updated row group metadata.

Type Aliases

OnCloseColumnChunk

Callback invoked on closing a column chunk

OnCloseRowGroup

Callback invoked on closing a row group, arguments are: