pyarrow.HdfsFile¶
-
class
pyarrow.HdfsFile¶ Bases:
pyarrow.lib.NativeFile-
__init__()¶ Initialize self. See help(type(self)) for accurate signature.
Methods
__init__()Initialize self.
close(self)download(self, stream_or_path[, buffer_size])Read file completely to local path (rather than reading completely into memory).
fileno(self)NOT IMPLEMENTED
flush(self)Flush the stream, if applicable.
isatty(self)metadata(self)Return file metadata
read(self[, nbytes])Read indicated number of bytes from file, or read all remaining bytes if no argument passed
read1(self[, nbytes])Read and return up to n bytes.
read_at(self, nbytes, offset)Read indicated number of bytes at offset from the file
read_buffer(self[, nbytes])readable(self)readall(self)readinto(self, b)Read into the supplied buffer
readline(self[, size])NOT IMPLEMENTED.
readlines(self[, hint])NOT IMPLEMENTED.
seek(self, int64_t position, int whence=0)Change current file stream position
seekable(self)size(self)Return file size
tell(self)Return current stream position
truncate(self)NOT IMPLEMENTED
upload(self, stream[, buffer_size])Pipe file-like object to file
writable(self)write(self, data)Write byte from any object implementing buffer protocol (bytes, bytearray, ndarray, pyarrow.Buffer)
writelines(self, lines)Attributes
-
buffer_size¶
-
close(self)¶
-
closed¶
-
download(self, stream_or_path, buffer_size=None)¶ Read file completely to local path (rather than reading completely into memory). First seeks to the beginning of the file.
-
fileno(self)¶ NOT IMPLEMENTED
-
flush(self)¶ Flush the stream, if applicable.
An error is raised if stream is not writable.
-
isatty(self)¶
-
metadata(self)¶ Return file metadata
-
mode¶
-
parent¶
-
read(self, nbytes=None)¶ Read indicated number of bytes from file, or read all remaining bytes if no argument passed
- Parameters
nbytes (int, default None) –
- Returns
data (bytes)
-
read1(self, nbytes=None)¶ Read and return up to n bytes.
Alias for read, needed to match the IOBase interface.
-
read_at(self, nbytes, offset)¶ Read indicated number of bytes at offset from the file
- Parameters
nbytes (int) –
offset (int) –
- Returns
data (bytes)
-
read_buffer(self, nbytes=None)¶
-
readable(self)¶
-
readall(self)¶
-
readinto(self, b)¶ Read into the supplied buffer
- Parameters
b (any python object supporting buffer interface) –
- Returns
number of bytes written
-
readline(self, size=None)¶ NOT IMPLEMENTED. Read and return a line of bytes from the file.
If size is specified, read at most size bytes.
Line terminator is always b”n”.
-
readlines(self, hint=None)¶ NOT IMPLEMENTED. Read lines of the file
- Parameters
hint (int maximum number of bytes read until we stop) –
-
seek(self, int64_t position, int whence=0)¶ Change current file stream position
- Parameters
position (int) – Byte offset, interpreted relative to value of whence argument
whence (int, default 0) – Point of reference for seek offset
Notes
Values of whence: * 0 – start of stream (the default); offset should be zero or positive * 1 – current stream position; offset may be negative * 2 – end of stream; offset is usually negative
- Returns
new_position (the new absolute stream position)
-
seekable(self)¶
-
size(self)¶ Return file size
-
tell(self)¶ Return current stream position
-
truncate(self)¶ NOT IMPLEMENTED
-
upload(self, stream, buffer_size=None)¶ Pipe file-like object to file
-
writable(self)¶
-
write(self, data)¶ Write byte from any object implementing buffer protocol (bytes, bytearray, ndarray, pyarrow.Buffer)
- Parameters
data (bytes-like object or exporter of buffer protocol) –
- Returns
nbytes (number of bytes written)
-
writelines(self, lines)¶
-