Struct FlightDataEncoder

pub struct FlightDataEncoder {
    inner: BoxStream<'static, Result<RecordBatch>>,
    schema: Option<SchemaRef>,
    max_flight_data_size: usize,
    encoder: FlightIpcEncoder,
    app_metadata: Option<Bytes>,
    queue: VecDeque<FlightData>,
    done: bool,
    descriptor: Option<FlightDescriptor>,
    dictionary_handling: DictionaryHandling,
}

Stream that encodes a stream of record batches to flight data.

See FlightDataEncoderBuilder for details and example.

Fields

inner: BoxStream<'static, Result<RecordBatch>>

Input stream

schema: Option<SchemaRef>

schema, set after the first batch

max_flight_data_size: usize

Target maximum size of flight data (see details on FlightDataEncoderBuilder::with_max_flight_data_size).

encoder: FlightIpcEncoder

do the encoding / tracking of dictionaries

app_metadata: Option<Bytes>

optional metadata to add to schema FlightData

queue: VecDeque<FlightData>

data queued up to send but not yet sent

done: bool

Is this stream done (inner is empty or errored)

descriptor: Option<FlightDescriptor>

cleared after the first FlightData message is sent

dictionary_handling: DictionaryHandling

Deterimines how DictionaryArrays are encoded for transport. See DictionaryHandling for more information.

Implementations

impl FlightDataEncoder

fn new( inner: BoxStream<'static, Result<RecordBatch>>, schema: Option<SchemaRef>, max_flight_data_size: usize, options: IpcWriteOptions, app_metadata: Bytes, descriptor: Option<FlightDescriptor>, dictionary_handling: DictionaryHandling, ) -> Self

pub fn known_schema(&self) -> Option<SchemaRef>

Report the schema of the encoded data when known. A schema is known when provided via the FlightDataEncoderBuilder::with_schema method.

fn queue_message(&mut self, data: FlightData)

Place the FlightData in the queue to send

fn queue_messages(&mut self, datas: impl IntoIterator<Item = FlightData>)

Place the FlightData in the queue to send

fn encode_schema(&mut self, schema: &SchemaRef) -> SchemaRef

Encodes schema as a FlightData in self.queue. Updates self.schema and returns the new schema

fn encode_batch(&mut self, batch: RecordBatch) -> Result<()>

Encodes batch into one or more FlightData messages in self.queue

Trait Implementations

impl Stream for FlightDataEncoder

type Item = Result<FlightData, FlightError>

Values yielded by the stream.

fn poll_next( self: Pin<&mut Self>, cx: &mut Context<'_>, ) -> Poll<Option<Self::Item>>

Attempt to pull out the next value of this stream, registering the current task for wakeup if the value is not yet available, and returning None if the stream is exhausted.

fn size_hint(&self) -> (usize, Option<usize>)

Returns the bounds on the remaining length of the stream.