pyarrow.input_stream

pyarrow.input_stream(source, compression='detect', buffer_size=None)

Create an Arrow input stream.

Parameters:

sourcestr, Path, buffer, or file-like object

The source to open for reading.

compressionstr optional, default ‘detect’

The compression algorithm to use for on-the-fly decompression. If “detect” and source is a file path, then compression will be chosen based on the file extension. If None, no compression will be applied. Otherwise, a well-known algorithm name must be supplied (e.g. “gzip”).

buffer_sizeint, default None

If None or 0, no buffering will happen. Otherwise the size of the temporary read buffer.

Examples

Create a readable BufferReader (NativeFile) from a Buffer or a memoryview object:

>>> import pyarrow as pa
>>> buf = memoryview(b"some data")
>>> with pa.input_stream(buf) as stream:
...     stream.read(4)
...
b'some'

Create a readable OSFile (NativeFile) from a string or file path:

>>> import gzip
>>> with gzip.open('example.gz', 'wb') as f:
...     f.write(b'some data')
...
9
>>> with pa.input_stream('example.gz') as stream:
...     stream.read()
...
b'some data'

Create a readable PythonFile (NativeFile) from a a Python file object:

>>> with open('example.txt', mode='w') as f:
...     f.write('some text')
...
9
>>> with pa.input_stream('example.txt') as stream:
...     stream.read(6)
...
b'some t'

previous

Streams and File Access

next

pyarrow.output_stream