Expand description
Implementation of Variant Binary Encoding from Apache Parquet.
Β§Main APIs
Variant: Represents a variant value, which can be an object, list, or primitive.VariantBuilder: For buildingVariantvalues.
Β§π§ Work In Progress
This crate is under active development and is not yet ready for production use. If you are interested in helping, you can find more information on the GitHub Variant issue
Β§Platform Support
Only little-endian platforms are officially supported and tested in CI. Big-endian platforms are not tested in CI and may not work correctly. Fixes for big-endian platforms are welcome and handled on a best-effort basis, but compatibility is not guaranteed.
Re-exportsΒ§
pub use path::VariantPath;pub use path::VariantPathElement;pub use self::decimal::VariantDecimal4;pub use self::decimal::VariantDecimal8;pub use self::decimal::VariantDecimal16;pub use self::decimal::VariantDecimalType;pub use self::list::VariantList;pub use self::metadata::EMPTY_VARIANT_METADATA;pub use self::metadata::EMPTY_VARIANT_METADATA_BYTES;pub use self::metadata::VARIANT_SPEC_VERSION;pub use self::metadata::VariantMetadata;pub use self::object::VariantObject;
ModulesΒ§
MacrosΒ§
- variant_
append_ value - Macro to generate the match statement for each append_variant, try_append_variant, and append_variant_bytes β they each have slightly different handling for object and list handling.
StructsΒ§
- HexString
- Invalid
Variant - List
Builder - A builder for creating
Variant::Listvalues. - List
State - Internal state for list building
- Object
Builder - A builder for creating
Variant::Objectvalues. - Object
Field Builder - A
VariantBuilderExtthat inserts a new field into a variant object. - Object
Header Writer - Object
State - Internal state for object building
- Parent
State - Tracks information needed to correctly finalize a nested builder.
- Read
Only Metadata Builder - A metadata builder that cannot register new field names, and merely returns the field id associated with a known field name. This is useful for variant unshredding operations, where the metadata column is fixed and β per variant shredding spec β already contains all field names from the typed_value column. It is also useful when projecting a subset of fields from a variant object value, since the bytes can be copied across directly without re-encoding their field ids.
- Short
String - A Variant
ShortString - Uuid
- A Universally Unique Identifier (UUID).
- Value
Builder - Wrapper around a
Vec<u8>that provides methods for appending primitive values, variant types, and metadata. - Variant
Builder - Top level builder for
Variantvalues - Variant
Type - Arrow Variant [
ExtensionType]. - Writable
Metadata Builder - Builder for constructing metadata for
Variantvalues. - f16
- A 16-bit floating point type implementing the IEEE 754-2008 standard
binary16a.k.a βhalfβ format.
EnumsΒ§
- Variant
- Represents a Parquet Variant
ConstantsΒ§
- BASIC_
TYPE_ BITS - EMPTY_
BUILDER_ MSG - MAX_
NESTING_ DEPTH - The maximum number of nested objects and arrays a
Variantmay contain. - MAX_
SHORT_ STRING_ BYTES - ONE_
TOP_ LEVEL_ VALUE_ MSG - UNIX_
EPOCH_ DATE
TraitsΒ§
- Builder
Specific State - A trait for managing state specific to different builder types.
- Metadata
Builder - A trait for building variant metadata dictionaries, to be used in conjunction with a
ValueBuilder. The trait provides methods for managing field names and their IDs, as well as rolling back a failed builder operation that might have created new field ids. - Variant
Builder Ext - Extends
VariantBuilderto help building nestedVariants
FunctionsΒ§
- append_
packed_ u32 - Append
value_sizebytes of givenvalueintodest. - append_
packed_ u32 - array_
header - empty_
builder_ error - int_
size - object_
header - primitive_
header - short_
string_ header - top_
level_ value_ error - top_
level_ value_ panic - write_
offset - Write little-endian integer to buffer