pyarrow.output_stream#
- pyarrow.output_stream(source, compression='detect', buffer_size=None)#
Create an Arrow output stream.
- Parameters:
- source
str
,Path
, buffer, file-like object The source to open for writing.
- compression
str
optional, default ‘detect’ The compression algorithm to use for on-the-fly compression. If “detect” and source is a file path, then compression will be chosen based on the file extension. If None, no compression will be applied. Otherwise, a well-known algorithm name must be supplied (e.g. “gzip”).
- buffer_size
int
, defaultNone
If None or 0, no buffering will happen. Otherwise the size of the temporary write buffer.
- source
Examples
Create a writable NativeFile from a pyarrow Buffer:
>>> import pyarrow as pa >>> data = b"buffer data" >>> empty_obj = bytearray(11) >>> buf = pa.py_buffer(empty_obj) >>> with pa.output_stream(buf) as stream: ... stream.write(data) ... 11 >>> with pa.input_stream(buf) as stream: ... stream.read(6) ... b'buffer'
or from a memoryview object:
>>> buf = memoryview(empty_obj) >>> with pa.output_stream(buf) as stream: ... stream.write(data) ... 11 >>> with pa.input_stream(buf) as stream: ... stream.read() ... b'buffer data'
Create a writable NativeFile from a string or file path:
>>> with pa.output_stream('example_second.txt') as stream: ... stream.write(b'Write some data') ... 15 >>> with pa.input_stream('example_second.txt') as stream: ... stream.read() ... b'Write some data'