Class ShredSchema

Namespace

Apache.Arrow.Operations.Shredding

Assembly

Apache.Arrow.Operations.dll

Describes the shredding schema for a variant column — which fields to extract into typed Parquet columns and at what types.

public sealed class ShredSchema

Inheritance

object

ShredSchema

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Properties

ArrayElement

For Array: the shredding schema applied to each element. Null for non-array types.

public ShredSchema ArrayElement { get; }

Property Value

ShredSchema

ObjectFields

For Object: the shredding schemas for each named sub-field. Null for non-object types.

public IReadOnlyDictionary<string, ShredSchema> ObjectFields { get; }

Property Value

IReadOnlyDictionary<string, ShredSchema>

TypedValueType

The type of the typed_value column. For primitives, this is the expected scalar type. For objects, use Object and populate ObjectFields. For arrays, use Array and populate ArrayElement. None means no typed_value — everything goes to binary value.

public ShredType TypedValueType { get; }

Property Value

ShredType

Methods

ForArray(ShredSchema)

Creates a schema that shreds array values by applying the element schema to each element.

public static ShredSchema ForArray(ShredSchema elementSchema)

Parameters

elementSchema ShredSchema

Returns

ShredSchema

ForObject(IDictionary<string, ShredSchema>)

Creates a schema that shreds object values by extracting named fields into typed sub-columns.

public static ShredSchema ForObject(IDictionary<string, ShredSchema> fields)

Parameters

fields IDictionary<string, ShredSchema>

Returns

ShredSchema

FromArrowType(IArrowType)

Derives a ShredSchema from the Arrow type of a typed_value column.

public static ShredSchema FromArrowType(IArrowType typedValueType)

Parameters

typedValueType IArrowType

The typed_value Arrow type, or null for a fully unshredded column.

Returns

ShredSchema

A ShredSchema describing the shredding.

Exceptions

ArgumentException

Thrown when typedValueType is not a valid shredded type per the Parquet variant shredding spec (for example, an unsigned integer or a fixed-size binary that isn't UUID).

Primitive(ShredType)

Creates a schema that shreds values into a typed primitive column. Values not matching this type fall back to the binary value column.

public static ShredSchema Primitive(ShredType type)

Parameters

type ShredType

Returns

ShredSchema

ShredTypeFromPrimitive(VariantPrimitiveType)

Maps a VariantPrimitiveType to the corresponding ShredType.

public static ShredType ShredTypeFromPrimitive(VariantPrimitiveType primitiveType)

Parameters

primitiveType VariantPrimitiveType

Returns

ShredType

Unshredded()

Creates a schema that does no shredding (all values go to binary).

public static ShredSchema Unshredded()

Returns

ShredSchema