Namespace Apache.Arrow.Operations.Shredding

Classes

ShredArrayResult

The typed_value result for a shredded array. Contains one ShredResult per element in the source array.

ShredObjectResult

The typed_value result for a shredded object. Contains one ShredResult per field defined in the object's ShredSchema.

ShredOptions

Options controlling how ShredSchemaInferer infers a shredding schema.

ShredResult

The result of shredding a single variant value: a (value, typed_value) pair.

Follows the Parquet variant shredding spec encoding matrix:

• Both null → missing (only valid for object sub-fields)
• value non-null, typed_value null → unshredded (value in binary)
• value null, typed_value non-null → fully shredded into typed column
• Both non-null → partially shredded object (typed_value has shredded fields, value has residual)

ShredSchema

Describes the shredding schema for a variant column — which fields to extract into typed Parquet columns and at what types.

ShredSchemaInferer

Analyzes a batch of VariantValues and infers an optimal ShredSchema for shredding them.

ShreddedVariantArrayBuilder

Assembles a shredded VariantArray from pre-shredded rows. Produces an Arrow struct with shared metadata, residual value, and the typed_value tree whose Arrow shape matches the ShredSchema.

VariantArrayShreddingExtensions

Shredding-aware extensions on VariantArray. Provides both transparent materialization (GetLogicalVariantValue(VariantArray, int)) and a reader-style API (GetShreddedVariant(VariantArray, int)) that exposes typed columns and residual bytes side-by-side.

VariantShredder

Decomposes VariantValues into shredded (value, typed_value) pairs according to a ShredSchema.

Per the Parquet variant shredding spec, the variant metadata dictionary is shared across an entire column. The Shred(IEnumerable<VariantValue>, ShredSchema) batch entrypoint builds that shared metadata and emits per-row value bytes that reference it — ready to drop into a Parquet value column.

VariantUnshredder

Reconstructs VariantValues from shredded (value, typed_value) pairs according to a ShredSchema. The residual Value bytes are interpreted against the column-level variant metadata supplied to Reconstruct(ShredResult, ShredSchema, ReadOnlySpan<byte>).

Structs

ShreddedArray

Reader for a single row of a shredded-array slot. The underlying storage is a list of element groups (each a {value, typed_value} struct).

ShreddedObject

Reader for a single row of a shredded-object slot. Provides field-wise access to both typed sub-columns and residual unshredded fields.

ShreddedVariant

Zero-copy reader for a single row of a (possibly shredded) variant column. Composes with the ShredSchema for this position to expose the typed columns and residual bytes side-by-side, or to materialize the logical value on demand.

A ShreddedVariant does not own any Arrow buffers; it is only valid while the underlying Arrow arrays are alive.

Enums

ShredType

Describes the type that a shredded typed_value column expects. Maps variant primitive types to the logical Parquet types used for shredding.