Module ffi_stream 

Contains declarations to bind to the C Stream Interface.

This module has two main interfaces: One interface maps C ABI to native Rust types, i.e. convert c-pointers, c_char, to native rust. This is handled by FFI_ArrowArrayStream.

The second interface is used to import FFI_ArrowArrayStream as a Rust reader. A stream whose top-level type is Struct can be read as RecordBatch by RecordBatchStreamReader; a stream of any type can be read as ArrayRef by ArrayStreamReader.

// create an record batch reader natively
let file = File::open("arrow_file").unwrap();
let reader = Box::new(FileReader::try_new(file).unwrap());

// export it
let mut stream = FFI_ArrowArrayStream::empty();
unsafe { export_reader_into_raw(reader, &mut stream) };

// consumed and used by something else...

// import it
let stream_reader = unsafe { RecordBatchStreamReader::from_raw(&mut stream).unwrap() };
let imported_schema = stream_reader.schema();

let mut produced_batches = vec![];
for batch in stream_reader {
     produced_batches.push(batch.unwrap());
}
Ok(())
}

Re-exports

use arrow_schema::DataType; 🔒

use std::ffi::CStr; 🔒

use std::ptr::addr_of; 🔒

use std::ffi::CString; 🔒

use std::os::raw::c_char; 🔒

use std::os::raw::c_int; 🔒

use std::os::raw::c_void; 🔒

use std::sync::Arc; 🔒

use arrow_data::ffi::FFI_ArrowArray; 🔒

use arrow_schema::ArrowError; 🔒

use arrow_schema::Field; 🔒

use arrow_schema::FieldRef; 🔒

use arrow_schema::Schema; 🔒

use arrow_schema::SchemaRef; 🔒

use arrow_schema::ffi::FFI_ArrowSchema; 🔒

use crate::RecordBatchOptions; 🔒

use crate::array::Array; 🔒

use crate::array::StructArray; 🔒

use crate::array::ArrayRef; 🔒

use crate::array::make_array; 🔒

use crate::array_reader::ArrayReader; 🔒

use crate::ffi::from_ffi_and_data_type; 🔒

use crate::record_batch::RecordBatch; 🔒

use crate::record_batch::RecordBatchReader; 🔒

use libc::EINVAL; 🔒

use libc::EIO; 🔒

use libc::ENOMEM; 🔒

use libc::ENOSYS; 🔒

Structs

ArrayStreamReader

An ArrayReader which imports arrays from an FFI_ArrowArrayStream.

ExportedArrayStream 🔒

FFI_ArrowArrayStream

ABI-compatible struct for ArrayStream from C Stream Interface See https://arrow.apache.org/docs/format/CStreamInterface.html#structure-definitions This was created by bindgen

RecordBatchStreamReader

A RecordBatchReader which imports Arrays from FFI_ArrowArrayStream.

StreamPrivateData 🔒

Functions

get_error_code 🔒

get_last_error 🔒 ^⚠

get_next 🔒 ^⚠

get_schema 🔒 ^⚠

get_stream_field 🔒

Gets the field from a raw pointer of FFI_ArrowArrayStream. This is used when constructing ArrayStreamReader to cache the field.

get_stream_schema 🔒

Gets schema from a raw pointer of FFI_ArrowArrayStream. This is used when constructing RecordBatchStreamReader to cache schema.

producer_error 🔒 ^⚠

Returns the producer’s message for the last failed call on a FFI_ArrowArrayStream.

release_stream 🔒 ^⚠

Type Aliases

ArrowArrayStreamReaderDeprecated

Renamed to RecordBatchStreamReader, to distinguish it from ArrayStreamReader.

Result 🔒