Expand description
Encodes RecordBatch into the Avro binary format.
Avro Encoder for Arrow types.
Re-exportsΒ§
use crate::codec::AvroDataType;πuse crate::codec::AvroField;πuse crate::codec::Codec;πuse crate::errors::AvroError;πuse crate::schema::Fingerprint;πuse crate::schema::Nullability;πuse crate::schema::Prefix;πuse arrow_array::Float16Array;πuse arrow_array::cast::AsArray;πuse arrow_array::types::ArrowPrimitiveType;πuse arrow_array::types::Date32Type;πuse arrow_array::types::Date64Type;πuse arrow_array::types::DurationMicrosecondType;πuse arrow_array::types::DurationMillisecondType;πuse arrow_array::types::DurationNanosecondType;πuse arrow_array::types::DurationSecondType;πuse arrow_array::types::Float16Type;πuse arrow_array::types::Float32Type;πuse arrow_array::types::Float64Type;πuse arrow_array::types::Int8Type;πuse arrow_array::types::Int16Type;πuse arrow_array::types::Int32Type;πuse arrow_array::types::Int64Type;πuse arrow_array::types::IntervalDayTimeType;πuse arrow_array::types::IntervalMonthDayNanoType;πuse arrow_array::types::IntervalYearMonthType;πuse arrow_array::types::RunEndIndexType;πuse arrow_array::types::Time32MillisecondType;πuse arrow_array::types::Time32SecondType;πuse arrow_array::types::Time64MicrosecondType;πuse arrow_array::types::Time64NanosecondType;πuse arrow_array::types::TimestampMicrosecondType;πuse arrow_array::types::TimestampMillisecondType;πuse arrow_array::types::TimestampNanosecondType;πuse arrow_array::types::TimestampSecondType;πuse arrow_array::types::UInt8Type;πuse arrow_array::types::UInt16Type;πuse arrow_array::types::UInt32Type;πuse arrow_array::types::UInt64Type;πuse arrow_array::Array;πuse arrow_array::BinaryViewArray;πuse arrow_array::Decimal128Array;πuse arrow_array::Decimal256Array;πuse arrow_array::DictionaryArray;πuse arrow_array::FixedSizeBinaryArray;πuse arrow_array::FixedSizeListArray;πuse arrow_array::GenericBinaryArray;πuse arrow_array::GenericListArray;πuse arrow_array::GenericListViewArray;πuse arrow_array::GenericStringArray;πuse arrow_array::LargeListArray;πuse arrow_array::LargeListViewArray;πuse arrow_array::ListArray;πuse arrow_array::ListViewArray;πuse arrow_array::MapArray;πuse arrow_array::OffsetSizeTrait;πuse arrow_array::PrimitiveArray;πuse arrow_array::RecordBatch;πuse arrow_array::RunArray;πuse arrow_array::StringArray;πuse arrow_array::StringViewArray;πuse arrow_array::StructArray;πuse arrow_array::UnionArray;πuse arrow_array::Decimal32Array;πuse arrow_array::Decimal64Array;πuse arrow_buffer::ArrowNativeType;πuse arrow_buffer::NullBuffer;πuse arrow_schema::DataType;πuse arrow_schema::Field;πuse arrow_schema::IntervalUnit;πuse arrow_schema::Schema as ArrowSchema;πuse arrow_schema::TimeUnit;πuse arrow_schema::UnionMode;πuse bytes::BufMut;πuse bytes::BytesMut;πuse std::io::Write;πuse std::sync::Arc;πuse uuid::Uuid;π
MacrosΒ§
- for_
rows_ πwith_ prefix
StructsΒ§
- Binary
Encoder π - Unified binary encoder generic over offset size (i32/i64).
- Binary
View πEncoder - BinaryView (byte view) encoder.
- Boolean
Encoder π - Decimal
Encoder π - Generic Avro decimal encoder over Arrow decimal arrays.
- Duration
Encoder π - Single generic encoder used for all three interval units.
Writes Avro
fixed(12)as three little-endian u32 values in one call. - Duration
Parts π - Enum
Encoder π - Avro
enumencoder for ArrowDictionaryArray<Int32, Utf8>. - F32Encoder π
- F64Encoder π
- Field
Binding π - Field
Encoder π - Arrow to Avro FieldEncoder:
- Fixed
Encoder π - Avro
fixedencoder for ArrowFixedSizeBinaryArray. Spec: a fixed is encoded as exactlysizebytes, with no length prefix. - Fixed
Size πList Encoder - FixedSizeList encoder.
- Float16
Fixed πEncoder - Float16 to Avro fixed(2) encoder (IEEE-754 little-endian bits)
- Int8
Encoder π - Int8 to Avro int encoder (converts i8 to i32)
- Int16
Encoder π - Int16 to Avro int encoder (converts i16 to i32)
- IntEncoder π
- Generic Avro
intencoder for primitive arrays withi32native values. - Interval
DayTime πFixed Encoder - Interval(DayTime) to Avro fixed(8) encoder (little-endian i32 days + i32 milliseconds)
- Interval
Month πDayNano Fixed Encoder - Interval(MonthDayNano) encoder for the Arrow-specific logical type
arrow.interval-month-day-nano. - Interval
Year πMonth Fixed Encoder - Interval(YearMonth) to Avro fixed(4) encoder (little-endian i32 months)
- List
Encoder π - List
View πEncoder - ListView encoder using
(offset, size)buffers. - Long
Encoder π - Generic Avro
longencoder for primitive arrays withi64native values. - MapEncoder π
- Record
Encoder π - A pre-computed plan for encoding a
RecordBatchto Avro. - Record
Encoder πBuilder - Builder for
RecordEncoderwrite plan - RunEncoded
Encoder π - Generic encoder for Arrow
RunArray<R>-based sites (run-end encoded). Follows the pattern used by other generic encoders (i.e.,ListEncoder<O>), avoiding runtime branching on run-end width. - Struct
Encoder π - Time32
Seconds πToMillis Encoder - Time32(Second) to Avro time-millis (int), via safe scaling by 1000
- UInt8
Encoder π - UInt8 to Avro int encoder (converts u8 to i32)
- UInt16
Encoder π - UInt16 to Avro int encoder (converts u16 to i32)
- UInt32
Encoder π - UInt32 to Avro long encoder (converts u32 to i64)
- UInt64
Fixed πEncoder - UInt64 to Avro fixed(8) encoder (little-endian bytes)
- Union
Encoder π - Utf8
Generic πEncoder - Utf8
View πEncoder - StringView encoder.
- Uuid
Encoder π - Avro UUID logical type encoder: Arrow FixedSizeBinary(16) to Avro string (UUID). Spec: uuid is a logical type over string (RFCβ4122). We output hyphenated form.
EnumsΒ§
- Encoder π
- Field
Plan π - Perβsite encoder plan for a field. This mirrors the Avro structure, so nested optional branch order can be honored exactly as declared by the schema.
- KeyKind π
- Internal key array kind used by Map encoder.
- Null
State π
TraitsΒ§
- Decimal
BeBytes π - Minimal trait to obtain a big-endian fixed-size byte array for a decimalβs
unscaled integer value at
idx. - Interval
ToDuration πParts - Trait mapping an Arrow interval native value to Avro duration
(months, days, millis).
FunctionsΒ§
- encode_
blocked_ πrange - Encode a blocked range of items with Avro array block framing.
- find_
map_ πvalue_ field_ index - find_
struct_ πchild_ index - minimal_
twos_ πcomplement - Minimal twoβs-complement big-endian representation helper for Avro decimal (bytes).
- union_
value_ πbranch_ byte - write_
bool π - write_
int π - write_
len_ πprefixed - write_
long π - Encode a single Avro-
longusing ZigZag + variable length, buffered. - write_
optional_ πindex - Write the union branch index for an optional field.
- write_
sign_ πextended - Sign-extend (or validate/truncate) big-endian integer bytes to exactly
nbytes.
Type AliasesΒ§
- Decimal32
Encoder π - Decimal64
Encoder π - Decimal128
Encoder π - Decimal256
Encoder π - List
Encoder32 π - List
Encoder64 π - List
View πEncoder32 - List
View πEncoder64 - RunEncoded
Encoder16 π - RunEncoded
Encoder32 π - RunEncoded
Encoder64 π - Utf8
Encoder π - Utf8
Large πEncoder