pyarrow.fs.FileSelector¶

class pyarrow.fs.FileSelector(base_dir, bool allow_not_found=False, bool recursive=False)¶

Bases: _Weakrefable

File and directory selector.

It contains a set of options that describes how to search for files and directories.

Parameters:

base_dirstr: The directory in which to select files. Relative paths also work, use ‘.’ for the current directory and ‘..’ for the parent.
allow_not_foundbool, default False: The behavior if base_dir doesn’t exist in the filesystem. If false, an error is returned. If true, an empty selection is returned.
recursivebool, default False: Whether to recurse into subdirectories.

Examples

List the contents of a directory and subdirectories:

>>> selector_1 = fs.FileSelector(local_path, recursive=True)
>>> local.get_file_info(selector_1) 
[<FileInfo for 'tmp/alphabet/example.dat': type=FileType.File, size=4>,
<FileInfo for 'tmp/alphabet/subdir': type=FileType.Directory>,
<FileInfo for 'tmp/alphabet/subdir/example_copy.dat': type=FileType.File, size=4>]

List only the contents of the base directory:

>>> selector_2 = fs.FileSelector(local_path)
>>> local.get_file_info(selector_2) 
[<FileInfo for 'tmp/alphabet/example.dat': type=FileType.File, size=4>,
<FileInfo for 'tmp/alphabet/subdir': type=FileType.Directory>]

Return empty selection if the directory doesn’t exist:

>>> selector_not_found = fs.FileSelector(local_path + '/missing',
...                                      recursive=True,
...                                      allow_not_found=True)
>>> local.get_file_info(selector_not_found)
[]

__init__(*args, **kwargs)¶

Methods

__init__(*args, **kwargs)

Attributes

`allow_not_found`
`base_dir`
`recursive`

allow_not_found¶

base_dir¶

recursive¶

pyarrow.fs.FileInfo

pyarrow.fs.FileSystem