Module org.apache.arrow.vector

Package org.apache.arrow.vector.ipc

Class ArrowWriter

java.lang.Object

org.apache.arrow.vector.ipc.ArrowWriter

All Implemented Interfaces:

AutoCloseable

Direct Known Subclasses:

ArrowFileWriter, ArrowStreamWriter

public abstract class ArrowWriter
extends Object
implements AutoCloseable

Abstract base class for implementing Arrow writers for IPC over a WriteChannel.

Field Summary

Fields

Modifier and Type	Field	Description
protected static final org.slf4j.Logger	LOGGER
protected IpcOption	option
protected final WriteChannel	out
protected final Schema	schema

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	ArrowWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out)
protected	ArrowWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, IpcOption option)
protected	ArrowWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, IpcOption option, CompressionCodec.Factory compressionFactory, CompressionUtil.CodecType codecType, Optional<Integer> compressionLevel)	Note: fields are not closed when the writer is closed.

Method Summary

Modifier and Type	Method	Description
long	bytesWritten()
void	close()
void	end()
protected void	endInternal(WriteChannel out)
protected abstract void	ensureDictionariesWritten(DictionaryProvider provider, Set<Long> dictionaryIdsUsed)	Write dictionaries after schema and before recordBatches, dictionaries won't be written if empty stream (only has schema data in IPC).
void	start()
protected void	startInternal(WriteChannel out)
void	writeBatch()	Writes the record batch currently loaded in this instance's VectorSchemaRoot.
protected void	writeDictionaryBatch(Dictionary dictionary)
protected ArrowBlock	writeDictionaryBatch(ArrowDictionaryBatch batch)
protected ArrowBlock	writeRecordBatch(ArrowRecordBatch batch)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOGGER

protected static final org.slf4j.Logger LOGGER

schema

protected final Schema schema

out

protected final WriteChannel out

option

protected IpcOption option

Constructor Details

ArrowWriter

protected ArrowWriter(VectorSchemaRoot root,
 DictionaryProvider provider,
 WritableByteChannel out)

ArrowWriter

protected ArrowWriter(VectorSchemaRoot root,
 DictionaryProvider provider,
 WritableByteChannel out,
 IpcOption option)

ArrowWriter

protected ArrowWriter(VectorSchemaRoot root,
 DictionaryProvider provider,
 WritableByteChannel out,
 IpcOption option,
 CompressionCodec.Factory compressionFactory,
 CompressionUtil.CodecType codecType,
 Optional<Integer> compressionLevel)

Note: fields are not closed when the writer is closed.

Parameters:

root - the vectors to write to the output

provider - where to find the dictionaries

out - the output where to write

option - IPC write options

compressionFactory - Compression codec factory

codecType - Compression codec

compressionLevel - Compression level

Method Details

start

public void start()
           throws IOException

Throws:

IOException

writeBatch

public void writeBatch()
                throws IOException

Writes the record batch currently loaded in this instance's VectorSchemaRoot.

Throws:

IOException

writeDictionaryBatch

protected void writeDictionaryBatch(Dictionary dictionary)
                             throws IOException

Throws:

IOException

writeDictionaryBatch

protected ArrowBlock writeDictionaryBatch(ArrowDictionaryBatch batch)
                                   throws IOException

Throws:

IOException

writeRecordBatch

protected ArrowBlock writeRecordBatch(ArrowRecordBatch batch)
                               throws IOException

Throws:

IOException

end

public void end()
         throws IOException

Throws:

IOException

bytesWritten

public long bytesWritten()

ensureDictionariesWritten

protected abstract void ensureDictionariesWritten(DictionaryProvider provider,
 Set<Long> dictionaryIdsUsed)
                                           throws IOException

Write dictionaries after schema and before recordBatches, dictionaries won't be written if empty stream (only has schema data in IPC).

Throws:

IOException

startInternal

protected void startInternal(WriteChannel out)
                      throws IOException

Throws:

IOException

endInternal

protected void endInternal(WriteChannel out)
                    throws IOException

Throws:

IOException

close

public void close()

Specified by:

close in interface AutoCloseable