Expand description
Avro schema parsing and representation
Provides types for parsing and representing Avro schema definitions. Avro Schema representations for Arrow.
StructsΒ§
- Array π
- An array
- Attributes π
- Additional attributes within a
Schema
- Avro
Schema - A wrapper for an Avro schema in its JSON string representation.
- Avro
Schema πOptions - Enum π
- An enumeration
- Field π
- A field within a
Record
- Fixed π
- A fixed length binary array
- Map π
- A map
- Name
Generator π - Prefix π
- A stack-allocated, fixed-size buffer for the prefix.
- Record π
- A record
- Schema
Store - An in-memory cache of Avro schemas, indexed by their fingerprint.
- Type π
- A type definition that is not a variant of
ComplexType
EnumsΒ§
- Complex
Type π - A complex type
- Fingerprint
- A schema fingerprint in one of the supported formats.
- Fingerprint
Algorithm - Supported fingerprint algorithms for Avro schema identification. For use with Confluent Schema Registry IDs, set to None.
- Fingerprint
Strategy - Defines the strategy for generating the per-record prefix for an Avro binary stream.
- Nullability π
- Avro types are not nullable, with nullability instead encoded as a union where one of the variants is the null type.
- Primitive
Type π - A primitive type
- Schema π
- An Avro schema
- Type
Name π - Either a
PrimitiveType
or a reference to a previously defined named type
ConstantsΒ§
- AVRO_
DOC_ METADATA_ KEY - Metadata key used to store the documentation for a type in an Avro schema.
- AVRO_
ENUM_ SYMBOLS_ METADATA_ KEY - Metadata key used to represent Avro enum symbols in an Arrow schema.
- AVRO_
FIELD_ DEFAULT_ METADATA_ KEY - Metadata key used to store the default value of a field in an Avro schema.
- AVRO_
NAMESPACE_ METADATA_ KEY - Metadata key used to store the name of a type in an Avro schema.
- AVRO_
NAME_ METADATA_ KEY - Metadata key used to store the name of a type in an Avro schema.
- AVRO_
ROOT_ RECORD_ DEFAULT_ NAME - Default name for the root record in an Avro schema.
- CONFLUENT_
MAGIC - The Confluent βmagicβ byte (
0x00
) - EMPTY π
- 64βbit Rabin fingerprint as described in the Avro spec.
- MAX_
PREFIX_ LEN - The maximum possible length of a prefix. SHA256 (32) + single-object magic (2)
- SCHEMA_
METADATA_ KEY - The metadata key used for storing the JSON encoded
Schema
- SINGLE_
OBJECT_ MAGIC - The Avro singleβobject encoding βmagicβ bytes (
0xC3 0x01
)
StaticsΒ§
- FINGERPRINT_
TABLE π - The preβcomputed table.
FunctionsΒ§
- arrow_
field_ πto_ avro - build_
canonical π - build_
table π - Build the full 256βentry table at compile time.
- compute_
fingerprint_ πmd5 - Compute the 128βbit MD5 fingerprint of the canonical form.
- compute_
fingerprint_ πrabin - Computes the 64-bit Rabin fingerprint for a given canonical schema string. This implementation is based on the Avro specification for schema fingerprinting.
- compute_
fingerprint_ πsha256 - Compute the 256βbit SHAβ256 fingerprint of the canonical form.
- datatype_
to_ πavro - deserialize_
default π - extend_
with_ πpassthrough_ metadata - Copies Arrow schema metadata entries to the provided JSON map,
skipping keys that are Avro-reserved, internal Arrow keys, or
nested under the
avro.schema.
namespace. Values that parse as JSON are inserted as JSON; otherwise the raw string is preserved. - is_
avro_ πjson_ null - is_
internal_ πarrow_ key - make_
full_ πname - merge_
extras π - min_
fixed_ πbytes_ for_ precision - one_
entry π - Build one entry of the polynomialβdivision table.
- process_
datatype π - quote π
- sanitise_
avro_ πname - union_
branch_ πsignature - wrap_
nullable π - write_
prefix π