Expand description
Arrow IPC File and Stream Writers
Β§Notes
FileWriter
and StreamWriter
have similar interfaces,
however the FileWriter
expects a reader that supports Seek
ing
Structs§
- Dictionary
Tracker - Keeps track of dictionaries that have been written, to avoid emitting the same dictionary multiple times.
- Encoded
Data - Stores the encoded data, which is an crate::Message, and optional Arrow data
- File
Writer - Arrow File Writer
- IpcData
Generator - Handles low level details of encoding [
Array
] and [Schema
] into the Arrow IPC Format. - IpcWrite
Options - IPC write options used to control the behaviour of the
IpcDataGenerator
- Stream
Writer - Arrow Stream Writer
Constants§
- PADDING π
Functions§
- append_
variadic_ πbuffer_ counts - buffer_
need_ πtruncate - Whether to truncate the buffer
- get_
buffer_ πelement_ width - Returns byte width for a buffer spec. Only for
BufferSpec::FixedWidth
. - get_
byte_ πarray_ buffers - Returns the values and offsets [
Buffer
] for a ByteArray with offset typeO
- get_
list_ πarray_ buffers - Similar logic as
get_byte_array_buffers()
but slices the child array instead of a values buffer. - has_
validity_ πbitmap - In V4, null types have no validity bitmap In V5 and later, null and union types have no validity bitmap Run end encoded type has no validity bitmap.
- into_
zero_ πoffset_ run_ array - pad_
to_ πalignment - Calculate an alignment boundary and return the number of bytes needed to pad to the alignment boundary
- reencode_
offsets π - Common functionality for re-encoding offsets. Returns the new offsets as well as original start offset and length for use in slicing child data.
- unslice_
run_ πarray - write_
array_ πdata - Write array data to a vector of bytes
- write_
body_ πbuffers - write_
buffer π - Write a buffer into
arrow_data
, a vector of bytes, and adds itscrate::Buffer
tobuffers
. Returns the new offset inarrow_data
- write_
continuation π - Write a record batch to the writer, writing the message size before the message if the record batch is being written to a stream
- write_
message - Write a messageβs IPC data and buffers, returning metadata and buffer data lengths written