pyarrow.LocalFileSystem

class pyarrow.LocalFileSystem[source]

Bases: pyarrow.filesystem.FileSystem

__init__()

Initialize self. See help(type(self)) for accurate signature.

Methods

cat(path) Return contents of file as a bytes object
delete(path[, recursive]) Delete the indicated file or directory
disk_usage(path) Compute bytes used by all contents under indicated path in file tree
exists(path)
get_instance()
isdir(path) Return True if path is a directory
isfile(path) Return True if path is a file
ls(path) Return list of file paths
mkdir(path[, create_parents])
mv(path, new_path) Alias for FileSystem.rename
open(path[, mode]) Open file for reading or writing
read_parquet(path[, columns, metadata, …]) Read Parquet data from path in file system.
rename(path, new_path) Rename file, like UNIX mv command
rm(path[, recursive]) Alias for FileSystem.delete
stat(path)
returns:stat (dict)
walk(top_dir) Directory tree generator, see os.walk
cat(path)

Return contents of file as a bytes object

Returns:contents (bytes)
delete(path, recursive=False)

Delete the indicated file or directory

Parameters:
  • path (string) –
  • recursive (boolean, default False) – If True, also delete child paths for directories
disk_usage(path)

Compute bytes used by all contents under indicated path in file tree

Parameters:path (string) – Can be a file path or directory
Returns:usage (int)
exists(path)[source]
classmethod get_instance()[source]
isdir(path)[source]

Return True if path is a directory

isfile(path)[source]

Return True if path is a file

ls(path)[source]

Return list of file paths

mkdir(path, create_parents=True)[source]
mv(path, new_path)

Alias for FileSystem.rename

open(path, mode='rb')[source]

Open file for reading or writing

pathsep
read_parquet(path, columns=None, metadata=None, schema=None, nthreads=1, use_pandas_metadata=False)

Read Parquet data from path in file system. Can read from a single file or a directory of files

Parameters:
  • path (str) – Single file path or directory
  • columns (List[str], optional) – Subset of columns to read
  • metadata (pyarrow.parquet.FileMetaData) – Known metadata to validate files against
  • schema (pyarrow.parquet.Schema) – Known schema to validate files against. Alternative to metadata argument
  • nthreads (int, default 1) – Number of columns to read in parallel. If > 1, requires that the underlying file source is threadsafe
  • use_pandas_metadata (boolean, default False) – If True and file has custom pandas schema metadata, ensure that index columns are also loaded
Returns:

table (pyarrow.Table)

rename(path, new_path)

Rename file, like UNIX mv command

Parameters:
  • path (string) – Path to alter
  • new_path (string) – Path to move to
rm(path, recursive=False)

Alias for FileSystem.delete

stat(path)
Returns:stat (dict)
walk(top_dir)[source]

Directory tree generator, see os.walk