pub struct Writer<W: Write, F: AvroFormat> {
writer: W,
schema: Arc<Schema>,
format: F,
compression: Option<CompressionCodec>,
capacity: usize,
encoder: RecordEncoder,
}
Expand description
Generic Avro writer.
This type is generic over the output Write sink (W
) and the Avro format (F
).
You’ll usually use the concrete aliases:
AvroWriter
for OCF (self‑describing container file)AvroStreamWriter
for raw Avro binary streams
Fields§
§writer: W
§schema: Arc<Schema>
§format: F
§compression: Option<CompressionCodec>
§capacity: usize
§encoder: RecordEncoder
Implementations§
Source§impl<W: Write> Writer<W, AvroOcfFormat>
impl<W: Write> Writer<W, AvroOcfFormat>
Sourcepub fn new(writer: W, schema: Schema) -> Result<Self, ArrowError>
pub fn new(writer: W, schema: Schema) -> Result<Self, ArrowError>
Convenience constructor – same as WriterBuilder::build
with AvroOcfFormat
.
§Example
use std::sync::Arc;
use arrow_array::{ArrayRef, Int32Array, RecordBatch};
use arrow_schema::{DataType, Field, Schema};
use arrow_avro::writer::AvroWriter;
let schema = Schema::new(vec![Field::new("id", DataType::Int32, false)]);
let batch = RecordBatch::try_new(
Arc::new(schema.clone()),
vec![Arc::new(Int32Array::from(vec![1, 2, 3])) as ArrayRef],
)?;
let buf: Vec<u8> = Vec::new();
let mut w = AvroWriter::new(buf, schema)?;
w.write(&batch)?;
w.finish()?;
let bytes = w.into_inner();
assert!(!bytes.is_empty());
Sourcepub fn sync_marker(&self) -> Option<&[u8; 16]>
pub fn sync_marker(&self) -> Option<&[u8; 16]>
Return a reference to the 16‑byte sync marker generated for this file.
Source§impl<W: Write> Writer<W, AvroBinaryFormat>
impl<W: Write> Writer<W, AvroBinaryFormat>
Sourcepub fn new(writer: W, schema: Schema) -> Result<Self, ArrowError>
pub fn new(writer: W, schema: Schema) -> Result<Self, ArrowError>
Convenience constructor to create a new AvroStreamWriter
.
The resulting stream contains just Avro binary bodies (no OCF header/sync and no single‑object or Confluent framing). If you need those frames, add them externally.
§Example
use std::sync::Arc;
use arrow_array::{ArrayRef, Int64Array, RecordBatch};
use arrow_schema::{DataType, Field, Schema};
use arrow_avro::writer::AvroStreamWriter;
let schema = Schema::new(vec![Field::new("x", DataType::Int64, false)]);
let batch = RecordBatch::try_new(
Arc::new(schema.clone()),
vec![Arc::new(Int64Array::from(vec![10, 20])) as ArrayRef],
)?;
let sink: Vec<u8> = Vec::new();
let mut w = AvroStreamWriter::new(sink, schema)?;
w.write(&batch)?;
w.finish()?;
let bytes = w.into_inner();
assert!(!bytes.is_empty());
Source§impl<W: Write, F: AvroFormat> Writer<W, F>
impl<W: Write, F: AvroFormat> Writer<W, F>
Sourcepub fn write(&mut self, batch: &RecordBatch) -> Result<(), ArrowError>
pub fn write(&mut self, batch: &RecordBatch) -> Result<(), ArrowError>
Serialize one [RecordBatch
] to the output.
Sourcepub fn write_batches(
&mut self,
batches: &[&RecordBatch],
) -> Result<(), ArrowError>
pub fn write_batches( &mut self, batches: &[&RecordBatch], ) -> Result<(), ArrowError>
A convenience method to write a slice of [RecordBatch
].
This is equivalent to calling write
for each batch in the slice.
Sourcepub fn finish(&mut self) -> Result<(), ArrowError>
pub fn finish(&mut self) -> Result<(), ArrowError>
Flush remaining buffered data and (for OCF) ensure the header is present.
Sourcepub fn into_inner(self) -> W
pub fn into_inner(self) -> W
Consume the writer, returning the underlying output object.