Class VariantArrayShreddingExtensions

Namespace

Apache.Arrow.Operations.Shredding

Assembly

Apache.Arrow.Operations.dll

Shredding-aware extensions on VariantArray. Provides both transparent materialization (GetLogicalVariantValue(VariantArray, int)) and a reader-style API (GetShreddedVariant(VariantArray, int)) that exposes typed columns and residual bytes side-by-side.

public static class VariantArrayShreddingExtensions

Inheritance

object

VariantArrayShreddingExtensions

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Methods

GetLogicalVariantValue(VariantArray, int)

Materializes the element at index into a logical VariantValue, transparently merging shredded columns and residual bytes. Works for both shredded and unshredded columns.

public static VariantValue GetLogicalVariantValue(this VariantArray array, int index)

Parameters

array VariantArray

index int

Returns

VariantValue

GetShredSchema(VariantArray)

Gets the ShredSchema for a variant array, derived from its Arrow storage type. Returns Unshredded() for unshredded columns.

public static ShredSchema GetShredSchema(this VariantArray array)

Parameters

array VariantArray

Returns

ShredSchema

GetShreddedVariant(VariantArray, int)

Gets a ShreddedVariant reader for the element at the given index. Exposes typed-column access and residual bytes without materializing the full logical variant. Works for both shredded and unshredded columns.

public static ShreddedVariant GetShreddedVariant(this VariantArray array, int index)

Parameters

array VariantArray

index int

Returns

ShreddedVariant

Exceptions

InvalidOperationException

If the element is null.

InferShredSchema(VariantArray, ShredOptions)

Infers a ShredSchema from the logical values of a variant array. Null elements are ignored. Works for both shredded and unshredded columns.

public static ShredSchema InferShredSchema(this VariantArray array, ShredOptions options = null)

Parameters

array VariantArray

options ShredOptions

Returns

ShredSchema

Remarks

When writing a column in batches (e.g. Parquet row groups), infer once over a representative batch and pass the result to Shred(VariantArray, ShredSchema, MemoryAllocator) for every batch, so that all batches share one layout.

Reassemble(VariantArray, MemoryAllocator)

Converts a shredded variant array into its unshredded equivalent, in which every element is stored as self-contained metadata and value bytes. Null elements remain null. An unshredded input is returned unchanged.

public static VariantArray Reassemble(this VariantArray array, MemoryAllocator allocator = null)

Parameters

array VariantArray

The variant array to reassemble.

allocator MemoryAllocator

Arrow memory allocator, or default if null.

Returns

VariantArray

Shred(VariantArray, ShredSchema, MemoryAllocator)

Shreds a variant array into the layout described by schema. The input may itself be shredded (under any schema); its logical values are re-shredded. Null elements remain null.

public static VariantArray Shred(this VariantArray array, ShredSchema schema, MemoryAllocator allocator = null)

Parameters

array VariantArray

The variant array to shred.

schema ShredSchema

The target shredding schema.

allocator MemoryAllocator

Arrow memory allocator, or default if null.

Returns

VariantArray

TryShred(VariantArray, ShredOptions, out VariantArray, MemoryAllocator)

Infers a shredding schema from array and, if it produces a shredded layout, shreds the array into it.

public static bool TryShred(this VariantArray array, ShredOptions options, out VariantArray shredded, MemoryAllocator allocator = null)

Parameters

array VariantArray

The variant array to shred.

options ShredOptions

Inference options, or Default if null.

shredded VariantArray

The shredded array, or null when this method returns false.

allocator MemoryAllocator

Arrow memory allocator, or default if null.

Returns

bool

True if a shredded layout was inferred; false if the values have no layout worth shredding (the inferred schema is Unshredded()).