Struct FileWriter

pub struct FileWriter<W> {
    writer: W,
    write_options: IpcWriteOptions,
    schema: SchemaRef,
    block_offsets: usize,
    dictionary_blocks: Vec<Block>,
    record_blocks: Vec<Block>,
    finished: bool,
    dictionary_tracker: DictionaryTracker,
    custom_metadata: HashMap<String, String>,
    data_gen: IpcDataGenerator,
}

Writer for an IPC file

Fields

writer: W

The object to write to

write_options: IpcWriteOptions

IPC write options

schema: SchemaRef

A reference to the schema, used in validating record batches

block_offsets: usize

The number of bytes between each block of bytes, as an offset for random access

dictionary_blocks: Vec<Block>

Dictionary blocks that will be written as part of the IPC footer

record_blocks: Vec<Block>

Record blocks that will be written as part of the IPC footer

finished: bool

Whether the writer footer has been written, and the writer is finished

dictionary_tracker: DictionaryTracker

Keeps track of dictionaries that have been written

custom_metadata: HashMap<String, String>

User level customized metadata

data_gen: IpcDataGenerator

Implementations

impl<W: Write> FileWriter<BufWriter<W>>

pub fn try_new_buffered(writer: W, schema: &Schema) -> Result<Self, ArrowError>

Try to create a new file writer with the writer wrapped in a BufWriter.

See FileWriter::try_new for an unbuffered version.

impl<W: Write> FileWriter<W>

pub fn try_new(writer: W, schema: &Schema) -> Result<Self, ArrowError>

Try to create a new writer, with the schema written as part of the header

Note the created writer is not buffered. See FileWriter::try_new_buffered for details.

Errors

An ‘Err’ may be returned if writing the header to the writer fails.

pub fn try_new_with_options( writer: W, schema: &Schema, write_options: IpcWriteOptions, ) -> Result<Self, ArrowError>

Try to create a new writer with IpcWriteOptions

Note the created writer is not buffered. See FileWriter::try_new_buffered for details.

Errors

An ‘Err’ may be returned if writing the header to the writer fails.

pub fn write_metadata( &mut self, key: impl Into<String>, value: impl Into<String>, )

Adds a key-value pair to the FileWriter’s custom metadata

pub fn write(&mut self, batch: &RecordBatch) -> Result<(), ArrowError>

Write a record batch to the file

pub fn finish(&mut self) -> Result<(), ArrowError>

Write footer and closing tag, then mark the writer as done

pub fn schema(&self) -> &SchemaRef

Returns the arrow [SchemaRef] for this arrow file.

pub fn get_ref(&self) -> &W

Gets a reference to the underlying writer.

pub fn get_mut(&mut self) -> &mut W

Gets a mutable reference to the underlying writer.

It is inadvisable to directly write to the underlying writer.

pub fn flush(&mut self) -> Result<(), ArrowError>

Flush the underlying writer.

Both the BufWriter and the underlying writer are flushed.

pub fn into_inner(self) -> Result<W, ArrowError>

Unwraps the the underlying writer.

The writer is flushed and the FileWriter is finished before returning.

Errors

An ‘Err’ may be returned if an error occurs while finishing the StreamWriter or while flushing the writer.

Trait Implementations

impl<W: Write> RecordBatchWriter for FileWriter<W>

fn write(&mut self, batch: &RecordBatch) -> Result<(), ArrowError>

Write a single batch to the writer.

fn close(self) -> Result<(), ArrowError>

Write footer or termination data, then mark the writer as done.