Module writer

Source
Expand description

Arrow IPC File and Stream Writers

Β§Notes

FileWriter and StreamWriter have similar interfaces, however the FileWriter expects a reader that supports Seeking

Structs§

DictionaryTracker
Keeps track of dictionaries that have been written, to avoid emitting the same dictionary multiple times.
EncodedData
Stores the encoded data, which is an crate::Message, and optional Arrow data
FileWriter
Arrow File Writer
IpcDataGenerator
Handles low level details of encoding [Array] and [Schema] into the Arrow IPC Format.
IpcWriteOptions
IPC write options used to control the behaviour of the IpcDataGenerator
StreamWriter
Arrow Stream Writer

Constants§

PADDING πŸ”’

Functions§

append_variadic_buffer_counts πŸ”’
buffer_need_truncate πŸ”’
Whether to truncate the buffer
get_buffer_element_width πŸ”’
Returns byte width for a buffer spec. Only for BufferSpec::FixedWidth.
get_byte_array_buffers πŸ”’
Returns the values and offsets [Buffer] for a ByteArray with offset type O
get_list_array_buffers πŸ”’
Similar logic as get_byte_array_buffers() but slices the child array instead of a values buffer.
has_validity_bitmap πŸ”’
In V4, null types have no validity bitmap In V5 and later, null and union types have no validity bitmap Run end encoded type has no validity bitmap.
into_zero_offset_run_array πŸ”’
pad_to_alignment πŸ”’
Calculate an alignment boundary and return the number of bytes needed to pad to the alignment boundary
reencode_offsets πŸ”’
Common functionality for re-encoding offsets. Returns the new offsets as well as original start offset and length for use in slicing child data.
unslice_run_array πŸ”’
write_array_data πŸ”’
Write array data to a vector of bytes
write_body_buffers πŸ”’
write_buffer πŸ”’
Write a buffer into arrow_data, a vector of bytes, and adds its crate::Buffer to buffers. Returns the new offset in arrow_data
write_continuation πŸ”’
Write a record batch to the writer, writing the message size before the message if the record batch is being written to a stream
write_message
Write a message’s IPC data and buffers, returning metadata and buffer data lengths written