pub struct Writer<W: Write, F: AvroFormat> {
writer: W,
schema: Arc<Schema>,
format: F,
compression: Option<CompressionCodec>,
capacity: usize,
encoder: RecordEncoder,
}
Expand description
Generic Avro writer.
This type is generic over the output Write sink (W
) and the Avro format (F
).
You’ll usually use the concrete aliases:
AvroWriter
for OCF (self‑describing container file)AvroStreamWriter
for SOE Avro streams
Fields§
§writer: W
§schema: Arc<Schema>
§format: F
§compression: Option<CompressionCodec>
§capacity: usize
§encoder: RecordEncoder
Implementations§
Source§impl<W: Write> Writer<W, AvroOcfFormat>
impl<W: Write> Writer<W, AvroOcfFormat>
Sourcepub fn new(writer: W, schema: Schema) -> Result<Self, ArrowError>
pub fn new(writer: W, schema: Schema) -> Result<Self, ArrowError>
Convenience constructor – same as WriterBuilder::build
with AvroOcfFormat
.
§Example
use std::sync::Arc;
use arrow_array::{ArrayRef, Int32Array, RecordBatch};
use arrow_schema::{DataType, Field, Schema};
use arrow_avro::writer::AvroWriter;
let schema = Schema::new(vec![Field::new("id", DataType::Int32, false)]);
let batch = RecordBatch::try_new(
Arc::new(schema.clone()),
vec![Arc::new(Int32Array::from(vec![1, 2, 3])) as ArrayRef],
)?;
let buf: Vec<u8> = Vec::new();
let mut w = AvroWriter::new(buf, schema)?;
w.write(&batch)?;
w.finish()?;
let bytes = w.into_inner();
assert!(!bytes.is_empty());
Sourcepub fn sync_marker(&self) -> Option<&[u8; 16]>
pub fn sync_marker(&self) -> Option<&[u8; 16]>
Return a reference to the 16‑byte sync marker generated for this file.
Source§impl<W: Write> Writer<W, AvroSoeFormat>
impl<W: Write> Writer<W, AvroSoeFormat>
Sourcepub fn new(writer: W, schema: Schema) -> Result<Self, ArrowError>
pub fn new(writer: W, schema: Schema) -> Result<Self, ArrowError>
Convenience constructor to create a new AvroStreamWriter
.
The resulting stream contains Single Object Encodings (no OCF header/sync).
§Example
use std::sync::Arc;
use arrow_array::{ArrayRef, Int64Array, RecordBatch};
use arrow_schema::{DataType, Field, Schema};
use arrow_avro::writer::AvroStreamWriter;
let schema = Schema::new(vec![Field::new("x", DataType::Int64, false)]);
let batch = RecordBatch::try_new(
Arc::new(schema.clone()),
vec![Arc::new(Int64Array::from(vec![10, 20])) as ArrayRef],
)?;
let sink: Vec<u8> = Vec::new();
let mut w = AvroStreamWriter::new(sink, schema)?;
w.write(&batch)?;
w.finish()?;
let bytes = w.into_inner();
assert!(!bytes.is_empty());
Source§impl<W: Write, F: AvroFormat> Writer<W, F>
impl<W: Write, F: AvroFormat> Writer<W, F>
Sourcepub fn write(&mut self, batch: &RecordBatch) -> Result<(), ArrowError>
pub fn write(&mut self, batch: &RecordBatch) -> Result<(), ArrowError>
Serialize one [RecordBatch
] to the output.
Sourcepub fn write_batches(
&mut self,
batches: &[&RecordBatch],
) -> Result<(), ArrowError>
pub fn write_batches( &mut self, batches: &[&RecordBatch], ) -> Result<(), ArrowError>
A convenience method to write a slice of [RecordBatch
].
This is equivalent to calling write
for each batch in the slice.
Sourcepub fn finish(&mut self) -> Result<(), ArrowError>
pub fn finish(&mut self) -> Result<(), ArrowError>
Flush remaining buffered data and (for OCF) ensure the header is present.
Sourcepub fn into_inner(self) -> W
pub fn into_inner(self) -> W
Consume the writer, returning the underlying output object.
fn write_ocf_block( &mut self, batch: &RecordBatch, sync: &[u8; 16], ) -> Result<(), ArrowError>
fn write_stream(&mut self, batch: &RecordBatch) -> Result<(), ArrowError>
Trait Implementations§
Auto Trait Implementations§
impl<W, F> Freeze for Writer<W, F>
impl<W, F> RefUnwindSafe for Writer<W, F>where
W: RefUnwindSafe,
F: RefUnwindSafe,
impl<W, F> Send for Writer<W, F>
impl<W, F> Sync for Writer<W, F>
impl<W, F> Unpin for Writer<W, F>
impl<W, F> UnwindSafe for Writer<W, F>where
W: UnwindSafe,
F: UnwindSafe,
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more