Crate arrow_integration_test 

Partial support for the Apache Arrow JSON test data format

These utilities define structs that read the integration JSON format for integration testing purposes.

This is not a canonical format, but provides a human-readable way of verifying language implementations

This crate is only intended for integration testing the Arrow project. It is not intended for usage outside of this context.

Platform Support

Only little-endian platforms are officially supported and tested in CI. Big-endian platforms are not tested in CI and may not work correctly. Fixes for big-endian platforms are welcome and handled on a best-effort basis, but compatibility is not guaranteed.

Re-exports

pub(crate) use IntervalDayTime; 🔒

pub(crate) use IntervalMonthDayNano; 🔒

pub(crate) use ScalarBuffer; 🔒

pub(crate) use hex::decode; 🔒

pub(crate) use num_bigint::BigInt; 🔒

pub(crate) use num_traits::Signed; 🔒

pub(crate) use Deserialize; 🔒

pub(crate) use Deserialize; 🔒

pub(crate) use Serialize; 🔒

pub(crate) use Serialize; 🔒

pub(crate) use Map as SJMap; 🔒

pub(crate) use Value; 🔒

pub(crate) use std::collections::HashMap; 🔒

pub(crate) use std::sync::Arc; 🔒

pub(crate) use Buffer; 🔒

pub(crate) use MutableBuffer; 🔒

pub(crate) use ArrowError; 🔒

pub(crate) use Result; 🔒

pub(crate) use arrow::util::bit_util; 🔒

pub(crate) use arrow::array::*; 🔒

pub(crate) use arrow::datatypes::*; 🔒

Modules

datatype 🔒

field 🔒

schema 🔒

Structs

ArrowJson

A struct that represents an Arrow file with a schema and record batches

ArrowJsonBatch

A struct that partially reads the Arrow JSON record batch

ArrowJsonColumn

A struct that partially reads the Arrow JSON column/array

ArrowJsonDictionaryBatch

A struct that partially reads the Arrow JSON dictionary batch

ArrowJsonField

Fields are left as JSON Value as they vary by DataType

ArrowJsonFieldDictionary

Represents a dictionary-encoded field in the Arrow JSON format

ArrowJsonSchema

A struct that partially reads the Arrow JSON schema.

DictionaryIndexType

Type of an index for a dictionary-encoded field in the Arrow JSON format

MetadataKeyValue

Functions

array_from_json

Construct an Arrow array from a partially typed JSON column

create_null_buf 🔒

A helper to create a null buffer from a Vec<bool>

data_type_from_json

Parse a data type from a JSON representation.

data_type_to_json

Generate a JSON representation of the data type.

dictionary_array_from_json

Construct a [DictionaryArray] from a partially typed JSON column

field_from_json

Parse a Field definition from a JSON representation.

field_to_json

Generate a JSON representation of the Field.

from_metadata

Parse a metadata definition from a JSON representation. The JSON can either be an Object or an Array of Objects.

json_int

Read a JSON number as an integer of type T.

record_batch_from_json

Generates a [RecordBatch] from an Arrow JSON batch, given a schema

schema_from_json

Parse a Schema definition from a JSON representation.

schema_to_json

Generate a JSON representation of the Schema.