pub struct IpcWriteOptions {
alignment: u8,
write_legacy_ipc_format: bool,
metadata_version: MetadataVersion,
batch_compression_type: Option<CompressionType>,
batch_compression_level: Option<i32>,
batch_lz4_block_size: Option<Lz4BlockSize>,
dictionary_handling: DictionaryHandling,
}Expand description
IPC write options used to control the behaviour of the IpcDataGenerator
Fields§
§alignment: u8Write padding after memory buffers to this multiple of bytes. Must be 8, 16, 32, or 64 - defaults to 64.
write_legacy_ipc_format: boolThe legacy format is for releases before 0.15.0, and uses metadata V4
metadata_version: MetadataVersionThe metadata version to write. The Rust IPC writer supports V4+
Default versions per crate
When creating the default IpcWriteOptions, the following metadata versions are used:
version 2.0.0: V4, with legacy format enabled version 4.0.0: V5
batch_compression_type: Option<CompressionType>Compression, if desired. Will result in a runtime error if the corresponding feature is not enabled
batch_compression_level: Option<i32>§batch_lz4_block_size: Option<Lz4BlockSize>The lz4 frame block size; None selects 256 KiB.
dictionary_handling: DictionaryHandlingHow to handle updating dictionaries in IPC messages
Implementations§
Source§impl IpcWriteOptions
impl IpcWriteOptions
Sourcepub fn try_with_compression(
self,
batch_compression_type: Option<CompressionType>,
) -> Result<Self, ArrowError>
pub fn try_with_compression( self, batch_compression_type: Option<CompressionType>, ) -> Result<Self, ArrowError>
Configures compression when writing IPC files.
Will result in a runtime error if the corresponding feature is not enabled
Sourcepub fn try_with_compression_level(
self,
batch_compression_level: Option<i32>,
) -> Result<Self, ArrowError>
pub fn try_with_compression_level( self, batch_compression_level: Option<i32>, ) -> Result<Self, ArrowError>
Configures the compression level used when writing compressed IPC batches.
Compression levels require metadata V5 or newer and are currently only supported for ZSTD compression.
Sourcepub fn try_with_lz4_block_size(
self,
block_size: Option<usize>,
) -> Result<Self, ArrowError>
pub fn try_with_lz4_block_size( self, block_size: Option<usize>, ) -> Result<Self, ArrowError>
Configures the lz4 frame block size used when compressing batches with
crate::CompressionType::LZ4_FRAME.
None restores the default of 256 KiB, which balances compression throughput against the
buffers retained by the reusable encoder. The LZ4 frame header can only describe the
block sizes 64 KiB, 256 KiB, 1 MiB and 4 MiB; any other value returns an
[ArrowError::InvalidArgumentError]. Like the other compression settings this requires
metadata version V5 or newer, and it has no effect when another compression codec is
selected.
Sourcepub(crate) fn lz4_block_size(&self) -> Lz4BlockSize
pub(crate) fn lz4_block_size(&self) -> Lz4BlockSize
The lz4 frame block size to compress with, in bytes.
fn check_zstd_level(self, level: i32) -> Result<Self, ArrowError>
Sourcepub fn try_new(
alignment: usize,
write_legacy_ipc_format: bool,
metadata_version: MetadataVersion,
) -> Result<Self, ArrowError>
pub fn try_new( alignment: usize, write_legacy_ipc_format: bool, metadata_version: MetadataVersion, ) -> Result<Self, ArrowError>
Try to create IpcWriteOptions, checking for incompatible settings
Sourcepub fn with_dictionary_handling(
self,
dictionary_handling: DictionaryHandling,
) -> Self
pub fn with_dictionary_handling( self, dictionary_handling: DictionaryHandling, ) -> Self
Configure how dictionaries are handled in IPC messages