pyarrow.ipc.RecordBatchStreamReader

class pyarrow.ipc.RecordBatchStreamReader(source)

Bases: pyarrow.lib._RecordBatchStreamReader

Reader for the Arrow streaming binary format.

Parameters

sourcebytes/buffer-like, pyarrow.NativeFile, or file-like Python object

Either an in-memory buffer, or a readable file object.

__init__(source)

Methods

__init__(source)
from_batches(schema, batches)	Create RecordBatchReader from an iterable of batches.
get_next_batch(self)
read_all(self)	Read all record batches as a pyarrow.Table.
read_next_batch(self)	Read next RecordBatch from the stream.
read_pandas(self, **options)	Read contents of stream to a pandas.DataFrame.

Attributes

schema	Shared schema of the record batches in the stream.
stats	Current IPC read statistics.

static from_batches(schema, batches)

Create RecordBatchReader from an iterable of batches.

Parameters

schemaSchema

The shared schema of the record batches

batchesIterable[RecordBatch]

The batches that this reader will return.

Returns

readerRecordBatchReader

get_next_batch(self)

read_all(self)

Read all record batches as a pyarrow.Table.

read_next_batch(self)

Read next RecordBatch from the stream.

Raises

StopIteration:

At end of stream.

read_pandas(self, **options)

Read contents of stream to a pandas.DataFrame.

Read all record batches as a pyarrow.Table then convert it to a pandas.DataFrame using Table.to_pandas.

Parameters

**options

Arguments to forward to Table.to_pandas.

Returns

dfpandas.DataFrame

schema

Shared schema of the record batches in the stream.

stats

Current IPC read statistics.

previous

pyarrow.ipc.RecordBatchFileWriter

next

pyarrow.ipc.RecordBatchStreamWriter