Crate parquet_variant 

Implementation of Variant Binary Encoding from Apache Parquet.

Main APIs

• Variant: Represents a variant value, which can be an object, list, or primitive.
• VariantBuilder: For building Variant values.

🚧 Work In Progress

This crate is under active development and is not yet ready for production use. If you are interested in helping, you can find more information on the GitHub Variant issue

Modules

builder 🔒

decoder 🔒

path 🔒

utils 🔒

variant 🔒

Structs

ListBuilder

A builder for creating Variant::List values.

ObjectBuilder

A builder for creating Variant::Object values.

ReadOnlyMetadataBuilder

A metadata builder that cannot register new field names, and merely returns the field id associated with a known field name. This is useful for variant unshredding operations, where the metadata column is fixed and – per variant shredding spec – already contains all field names from the typed_value column. It is also useful when projecting a subset of fields from a variant object value, since the bytes can be copied across directly without re-encoding their field ids.

ShortString

A Variant ShortString

ValueBuilder

Wrapper around a Vec<u8> that provides methods for appending primitive values, variant types, and metadata.

VariantBuilder

Top level builder for Variant values

VariantDecimal4

Represents a 4-byte decimal value in the Variant format.

VariantDecimal8

Represents an 8-byte decimal value in the Variant format.

VariantDecimal16

Represents an 16-byte decimal value in the Variant format.

VariantList

Variant Array.

VariantMetadata

Variant Metadata

VariantObject

A Variant Object (struct with named fields).

VariantPath

Represents a qualified path to a potential subfield or index of a variant value.

WritableMetadataBuilder

Builder for constructing metadata for Variant values.

Enums

ParentState

Tracks information needed to correctly finalize a nested builder, for each parent builder type.

Variant

Represents a Parquet Variant

VariantPathElement

Element of a VariantPath that can be a field name or an index.

Traits

MetadataBuilder

A trait for building variant metadata dictionaries, to be used in conjunction with a ValueBuilder. The trait provides methods for managing field names and their IDs, as well as rolling back a failed builder operation that might have created new field ids.

VariantBuilderExt

Extends VariantBuilder to help building nested Variants