Filesystems

Interface

FileInfo(path, FileType type=FileType.Unknown)	FileSystem entry info.
FileSelector(base_dir, ...)	File and directory selector.
FileSystem()	Abstract file system API.

Filesystem Implementations

LocalFileSystem([use_mmap])	A FileSystem implementation accessing files on the local machine.
S3FileSystem([access_key, secret_key, ...])	S3-backed FileSystem implementation
GcsFileSystem(bool anonymous=False, *[, ...])	Google Cloud Storage (GCS) backed FileSystem implementation
HadoopFileSystem(unicode host, ...[, ...])	HDFS backed FileSystem implementation
SubTreeFileSystem(base_path, FileSystem base_fs)	Delegates to another implementation after prepending a fixed base path.

To define filesystems with behavior implemented in Python:

PyFileSystem(handler)	A FileSystem with behavior implemented in Python.
FileSystemHandler()	An abstract class exposing methods to implement PyFileSystem's behavior.
FSSpecHandler(fs)	Handler for fsspec-based Python filesystems.

Utilities

copy_files(source, destination[, ...])	Copy files between FileSystems.
initialize_s3(...)	Initialize S3 support
finalize_s3()
resolve_s3_region(bucket)	Resolve the S3 region of a bucket.
S3LogLevel(value[, names, module, qualname, ...])