Module writer

JSON Writer

This JSON writer converts Arrow [RecordBatch]es into arrays of JSON objects or JSON formatted byte streams.

Writing JSON formatted byte streams

To serialize [RecordBatch]es into line-delimited JSON bytes, use LineDelimitedWriter:

let schema = Schema::new(vec![Field::new("a", DataType::Int32, false)]);
let a = Int32Array::from(vec![1, 2, 3]);
let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(a)]).unwrap();

// Write the record batch out as JSON
let buf = Vec::new();
let mut writer = arrow_json::LineDelimitedWriter::new(buf);
writer.write_batches(&vec![&batch]).unwrap();
writer.finish().unwrap();

// Get the underlying buffer back,
let buf = writer.into_inner();
assert_eq!(r#"{"a":1}
{"a":2}
{"a":3}
"#, String::from_utf8(buf).unwrap())

To serialize [RecordBatch]es into a well formed JSON array, use ArrayWriter:

use arrow_schema::{DataType, Field, Schema};

let schema = Schema::new(vec![Field::new("a", DataType::Int32, false)]);
let a = Int32Array::from(vec![1, 2, 3]);
let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(a)]).unwrap();

// Write the record batch out as a JSON array
let buf = Vec::new();
let mut writer = arrow_json::ArrayWriter::new(buf);
writer.write_batches(&vec![&batch]).unwrap();
writer.finish().unwrap();

// Get the underlying buffer back,
let buf = writer.into_inner();
assert_eq!(r#"[{"a":1},{"a":2},{"a":3}]"#, String::from_utf8(buf).unwrap())

LineDelimitedWriter and ArrayWriter will omit writing keys with null values. In order to explicitly write null values for keys, configure a custom Writer by using a WriterBuilder to construct a Writer.

Writing to serde_json JSON Objects

To serialize [RecordBatch]es into an array of JSON objects you can reparse the resulting JSON string. Note that this is less efficient than using the Writer API.

let schema = Schema::new(vec![Field::new("a", DataType::Int32, false)]);
let a = Int32Array::from(vec![1, 2, 3]);
let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(a)]).unwrap();

// Write the record batch out as json bytes (string)
let buf = Vec::new();
let mut writer = arrow_json::ArrayWriter::new(buf);
writer.write_batches(&vec![&batch]).unwrap();
writer.finish().unwrap();
let json_data = writer.into_inner();

// Parse the string using serde_json
use serde_json::{Map, Value};
let json_rows: Vec<Map<String, Value>> = serde_json::from_reader(json_data.as_slice()).unwrap();
assert_eq!(
    serde_json::Value::Object(json_rows[1].clone()),
    serde_json::json!({"a": 2}),
);

Modules

• encoder 🔒

Structs

• JsonArray
  Produces JSON output as a single JSON array.
• LineDelimited
  Produces JSON output with one record per line.
• Writer
  A JSON writer which serializes [RecordBatch]es to a stream of u8 encoded JSON objects.
• WriterBuilder
  JSON writer builder.

Traits

• JsonFormat
  This trait defines how to format a sequence of JSON objects to a byte stream.

Type Aliases

• ArrayWriter
  A JSON writer which serializes [RecordBatch]es to JSON arrays.
• LineDelimitedWriter
  A JSON writer which serializes [RecordBatch]es to newline delimited JSON objects.