Class VariantShredder

Namespace

Apache.Arrow.Operations.Shredding

Assembly

Apache.Arrow.Operations.dll

Decomposes VariantValues into shredded (value, typed_value) pairs according to a ShredSchema.

Per the Parquet variant shredding spec, the variant metadata dictionary is shared across an entire column. The Shred(IEnumerable<VariantValue>, ShredSchema) batch entrypoint builds that shared metadata and emits per-row value bytes that reference it — ready to drop into a Parquet value column.

public static class VariantShredder

Inheritance

object

VariantShredder

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Methods

Shred(VariantValue, ShredSchema, VariantMetadataBuilder, int[])

Shreds a single variant value against a caller-managed metadata dictionary. Use this when combining shredded columns with external metadata, or when streaming rows one at a time. The caller is responsible for ensuring metadata already contains every field name the residual may reference.

public static ShredResult Shred(VariantValue value, ShredSchema schema, VariantMetadataBuilder metadata, int[] idRemap)

Parameters

value VariantValue

schema ShredSchema

metadata VariantMetadataBuilder

idRemap int[]

Returns

ShredResult

Shred(IEnumerable<VariantValue>, ShredSchema)

Shreds a column of variant values into a shared metadata dictionary and per-row ShredResults. The residual Value bytes for each row reference the returned metadata.

public static (byte[] Metadata, IReadOnlyList<ShredResult> Rows) Shred(IEnumerable<VariantValue> values, ShredSchema schema)

Parameters

values IEnumerable<VariantValue>

schema ShredSchema

Returns

(byte[] Metadata, IReadOnlyList<ShredResult> Rows)

Shred(IEnumerable<VariantValue?>, ShredSchema)

Shreds a column of nullable variant values. A null entry is a SQL-NULL row, as opposed to Null, which is a present variant null. SQL-NULL rows produce a null entry in the returned rows, which Build(ShredSchema, byte[], IReadOnlyList<ShredResult>, MemoryAllocator) turns into a null element of the resulting array.

public static (byte[] Metadata, IReadOnlyList<ShredResult> Rows) Shred(IEnumerable<VariantValue?> values, ShredSchema schema)

Parameters

values IEnumerable<VariantValue?>

schema ShredSchema

Returns

(byte[] Metadata, IReadOnlyList<ShredResult> Rows)