Crate parquet_variant 

Implementation of Variant Binary Encoding from Apache Parquet.

Main APIs

• Variant: Represents a variant value, which can be an object, list, or primitive.
• VariantBuilder: For building Variant values.

🚧 Work In Progress

This crate is under active development and is not yet ready for production use. If you are interested in helping, you can find more information on the GitHub Variant issue

Modules

builder 🔒

decoder 🔒

path 🔒

utils 🔒

variant 🔒

Structs

ListBuilder

A builder for creating Variant::List values.

ListState

Internal state for list building

ObjectBuilder

A builder for creating Variant::Object values.

ObjectFieldBuilder

A VariantBuilderExt that inserts a new field into a variant object.

ObjectState

Internal state for object building

ParentState

Tracks information needed to correctly finalize a nested builder.

ReadOnlyMetadataBuilder

A metadata builder that cannot register new field names, and merely returns the field id associated with a known field name. This is useful for variant unshredding operations, where the metadata column is fixed and – per variant shredding spec – already contains all field names from the typed_value column. It is also useful when projecting a subset of fields from a variant object value, since the bytes can be copied across directly without re-encoding their field ids.

ShortString

A Variant ShortString

Uuid

A Universally Unique Identifier (UUID).

ValueBuilder

Wrapper around a Vec<u8> that provides methods for appending primitive values, variant types, and metadata.

VariantBuilder

Top level builder for Variant values

VariantDecimal4

Represents a 4-byte decimal value in the Variant format.

VariantDecimal8

Represents an 8-byte decimal value in the Variant format.

VariantDecimal16

Represents an 16-byte decimal value in the Variant format.

VariantList

Variant Array.

VariantMetadata

Variant Metadata

VariantObject

A Variant Object (struct with named fields).

VariantPath

Represents a qualified path to a potential subfield or index of a variant value.

WritableMetadataBuilder

Builder for constructing metadata for Variant values.

f16

A 16-bit floating point type implementing the IEEE 754-2008 standard binary16 a.k.a “half” format.

Enums

Variant

Represents a Parquet Variant

VariantPathElement

Element of a VariantPath that can be a field name or an index.

Constants

EMPTY_VARIANT_METADATA

The empty metadata dictionary.

EMPTY_VARIANT_METADATA_BYTES

The canonical byte slice corresponding to an empty metadata dictionary.

Traits

BuilderSpecificState

A trait for managing state specific to different builder types.

MetadataBuilder

A trait for building variant metadata dictionaries, to be used in conjunction with a ValueBuilder. The trait provides methods for managing field names and their IDs, as well as rolling back a failed builder operation that might have created new field ids.

VariantBuilderExt

Extends VariantBuilder to help building nested Variants