Expand description
Avro schema parsing and representation
Provides types for parsing and representing Avro schema definitions. Avro Schema representations for Arrow.
Structs§
- Array 🔒
- An array
- Attributes 🔒
- Additional attributes within a
Schema - Avro
Schema - A wrapper for an Avro schema in its JSON string representation.
- Avro
Schema 🔒Options - Enum 🔒
- An enumeration
- Field 🔒
- A field within a
Record - Fixed 🔒
- A fixed length binary array
- Map 🔒
- A map
- Name
Generator 🔒 - Prefix 🔒
- A stack-allocated, fixed-size buffer for the prefix.
- Record 🔒
- A record
- Schema
Store - An in-memory cache of Avro schemas, indexed by their fingerprint.
- Type 🔒
- A type definition that is not a variant of
ComplexType
Enums§
- Complex
Type 🔒 - A complex type
- Fingerprint
- A schema fingerprint in one of the supported formats.
- Fingerprint
Algorithm - Supported fingerprint algorithms for Avro schema identification. For use with Confluent Schema Registry IDs, set to None.
- Fingerprint
Strategy - Defines the strategy for generating the per-record prefix for an Avro binary stream.
- Nullability 🔒
- Avro types are not nullable, with nullability instead encoded as a union where one of the variants is the null type.
- Primitive
Type 🔒 - A primitive type
- Schema 🔒
- An Avro schema
- Type
Name 🔒 - Either a
PrimitiveTypeor a reference to a previously defined named type
Constants§
- AVRO_
DOC_ METADATA_ KEY - Metadata key used to store the documentation for a type in an Avro schema.
- AVRO_
ENUM_ SYMBOLS_ METADATA_ KEY - Metadata key used to represent Avro enum symbols in an Arrow schema.
- AVRO_
FIELD_ DEFAULT_ METADATA_ KEY - Metadata key used to store the default value of a field in an Avro schema.
- AVRO_
NAMESPACE_ METADATA_ KEY - Metadata key used to store the name of a type in an Avro schema.
- AVRO_
NAME_ METADATA_ KEY - Metadata key used to store the name of a type in an Avro schema.
- AVRO_
ROOT_ RECORD_ DEFAULT_ NAME - Default name for the root record in an Avro schema.
- CONFLUENT_
MAGIC - The Confluent “magic” byte (
0x00) - EMPTY 🔒
- 64‑bit Rabin fingerprint as described in the Avro spec.
- MAX_
PREFIX_ LEN - The maximum possible length of a prefix. SHA256 (32) + single-object magic (2)
- SCHEMA_
METADATA_ KEY - The metadata key used for storing the JSON encoded
Schema - SINGLE_
OBJECT_ MAGIC - The Avro single‑object encoding “magic” bytes (
0xC3 0x01)
Statics§
- FINGERPRINT_
TABLE 🔒 - The pre‑computed table.
Functions§
- arrow_
field_ 🔒to_ avro - build_
canonical 🔒 - build_
table 🔒 - Build the full 256‑entry table at compile time.
- compute_
fingerprint_ 🔒md5 - Compute the 128‑bit MD5 fingerprint of the canonical form.
- compute_
fingerprint_ 🔒rabin - Computes the 64-bit Rabin fingerprint for a given canonical schema string. This implementation is based on the Avro specification for schema fingerprinting.
- compute_
fingerprint_ 🔒sha256 - Compute the 256‑bit SHA‑256 fingerprint of the canonical form.
- datatype_
to_ 🔒avro - deserialize_
default 🔒 - extend_
with_ 🔒passthrough_ metadata - Copies Arrow schema metadata entries to the provided JSON map,
skipping keys that are Avro-reserved, internal Arrow keys, or
nested under the
avro.schema.namespace. Values that parse as JSON are inserted as JSON; otherwise the raw string is preserved. - is_
avro_ 🔒json_ null - is_
internal_ 🔒arrow_ key - make_
full_ 🔒name - merge_
extras 🔒 - min_
fixed_ 🔒bytes_ for_ precision - one_
entry 🔒 - Build one entry of the polynomial‑division table.
- process_
datatype 🔒 - quote 🔒
- sanitise_
avro_ 🔒name - union_
branch_ 🔒signature - wrap_
nullable 🔒 - write_
prefix 🔒