pyarrow.NativeFile#

class pyarrow.NativeFile#

Bases: pyarrow.lib._Weakrefable

The base class for all Arrow streams.

Streams are either readable, writable, or both. They optionally support seeking.

While this class exposes methods to read or write data from Python, the primary intent of using a Arrow stream is to pass it to other Arrow facilities that will make use of it, such as Arrow IPC routines.

Be aware that there are subtle differences with regular Python files, e.g. destroying a writable Arrow stream without closing it explicitly will not flush any pending data.

__init__(*args, **kwargs)#

Methods

`__init__`(args, *kwargs)
`close`(self)
`download`(self, stream_or_path[, buffer_size])	Read this file completely to a local path or destination stream.
`fileno`(self)	NOT IMPLEMENTED
`flush`(self)	Flush the stream, if applicable.
`isatty`(self)
`metadata`(self)	Return file metadata
`read`(self[, nbytes])	Read indicated number of bytes from file, or read all remaining bytes if no argument passed
`read1`(self[, nbytes])	Read and return up to n bytes.
`read_at`(self, nbytes, offset)	Read indicated number of bytes at offset from the file
`read_buffer`(self[, nbytes])
`readable`(self)
`readall`(self)
`readinto`(self, b)	Read into the supplied buffer
`readline`(self[, size])	NOT IMPLEMENTED.
`readlines`(self[, hint])	NOT IMPLEMENTED.
`seek`(self, int64_t position, int whence=0)	Change current file stream position
`seekable`(self)
`size`(self)	Return file size
`tell`(self)	Return current stream position
`truncate`(self)	NOT IMPLEMENTED
`upload`(self, stream[, buffer_size])	Write from a source stream to this file.
`writable`(self)
`write`(self, data)	Write byte from any object implementing buffer protocol (bytes, bytearray, ndarray, pyarrow.Buffer)
`writelines`(self, lines)

Attributes

`closed`
`mode`	The file mode.

close(self)#

closed#

download(self, stream_or_path, buffer_size=None)#

Read this file completely to a local path or destination stream.

This method first seeks to the beginning of the file.

Parameters

stream_or_pathstr or file-like object: If a string, a local file path to write to; otherwise, should be a writable stream.
buffer_sizeint, optional: The buffer size to use for data transfers.

fileno(self)#: NOT IMPLEMENTED

flush(self)#

Flush the stream, if applicable.

An error is raised if stream is not writable.

isatty(self)#

metadata(self)#: Return file metadata

mode#

The file mode. Currently instances of NativeFile may support:

rb: binary read
wb: binary write
rb+: binary read and write

read(self, nbytes=None)#

Read indicated number of bytes from file, or read all remaining bytes if no argument passed

Parameters

nbytesint, default None

Returns

databytes

read1(self, nbytes=None)#

Read and return up to n bytes.

Alias for read, needed to match the BufferedIOBase interface.

Parameters

nbytesint: The maximum number of bytes to read.

read_at(self, nbytes, offset)#

Read indicated number of bytes at offset from the file

Parameters

nbytesint
offsetint

Returns

databytes

read_buffer(self, nbytes=None)#

readable(self)#

readall(self)#

readinto(self, b)#

Read into the supplied buffer

Parameters

bbuffer-like object: A writable buffer object (such as a bytearray).

Returns

writtenint: number of bytes written

readline(self, size=None)#

NOT IMPLEMENTED. Read and return a line of bytes from the file.

If size is specified, read at most size bytes.

Line terminator is always b”n”.

Parameters

sizeint: maximum number of bytes read

readlines(self, hint=None)#

NOT IMPLEMENTED. Read lines of the file

Parameters

hintint: maximum number of bytes read until we stop

seek(self, int64_t position, int whence=0)#

Change current file stream position

Parameters

positionint: Byte offset, interpreted relative to value of whence argument
whenceint, default 0: Point of reference for seek offset

Returns

int: The new absolute stream position.

Notes

Values of whence: * 0 – start of stream (the default); offset should be zero or positive * 1 – current stream position; offset may be negative * 2 – end of stream; offset is usually negative

seekable(self)#

size(self)#: Return file size

tell(self)#: Return current stream position

truncate(self)#: NOT IMPLEMENTED

upload(self, stream, buffer_size=None)#

Write from a source stream to this file.

Parameters

streamfile-like object: Source stream to pipe to this file.
buffer_sizeint, optional: The buffer size to use for data transfers.

writable(self)#

write(self, data)#

Write byte from any object implementing buffer protocol (bytes, bytearray, ndarray, pyarrow.Buffer)

Parameters

databytes-like object or exporter of buffer protocol

Returns

int: nbytes: number of bytes written

writelines(self, lines)#

pyarrow.create_memory_map

pyarrow.OSFile