FileSystem is an abstract file system API,
LocalFileSystem is an implementation accessing files
on the local machine.
SubTreeFileSystem is an implementation that delegates
to another implementation after prepending a fixed base path
LocalFileSystem$create() returns the object and takes no arguments.
SubTreeFileSystem$create() takes the following arguments:
base_path, a string path
S3FileSystem$create() optionally takes arguments:
anonymous: logical, default
FALSE. If true, will not attempt to look up credentials using standard AWS configuration methods.
secret_key: authentication credentials. If one is provided, the other must be as well. If both are provided, they will override any AWS configuration set at the environment level.
session_token: optional string for authentication along with
role_arn: string AWS ARN of an AccessRole. If provided instead of
secret_key, temporary credentials will be fetched by assuming this role.
session_name: optional string identifier for the assumed role session.
external_id: optional unique string identifier that might be required when you assume a role in another account.
load_frequency: integer, frequency (in seconds) with which temporary credentials from an assumed role session will be refreshed. Default is 900 (i.e. 15 minutes)
region: AWS region to connect to. If omitted, the AWS library will provide a sensible default based on client configuration, falling back to "us-east-1" if no other alternatives are found.
endpoint_override: If non-empty, override region with a connect string such as "localhost:9000". This is useful for connecting to file systems that emulate S3.
scheme: S3 connection transport (default "https")
background_writes: logical, whether
OutputStreamwrites will be issued in the background, without blocking (default
allow_bucket_creation: logical, if TRUE, the filesystem will create buckets if
$CreateDir()is called on the bucket level (default
allow_bucket_deletion: logical, if TRUE, the filesystem will delete buckets if
$DeleteDir()is called on the bucket level (default
request_timeout: Socket read time on Windows and MacOS in seconds. If negative, the AWS SDK default (typically 3 seconds).
connect_timeout: Socket connection timeout in seconds. If negative, AWS SDK default is used (typically 1 second).
GcsFileSystem$create() optionally takes arguments:
anonymous: logical, default
FALSE. If true, will not attempt to look up credentials using standard GCS configuration methods.
access_token: optional string for authentication. Should be provided along with
expiration: optional date representing point at which
json_credentials: optional string for authentication. Point to a JSON credentials file downloaded from GCS.
endpoint_override: if non-empty, will connect to provided host name / port, such as "localhost:9001", instead of default GCS ones. This is primarily useful for testing purposes.
scheme: connection transport (default "https")
default_bucket_location: the default location (or "region") to create new buckets in.
retry_limit_seconds: the maximum amount of time to spend retrying if the filesystem encounters errors. Default is 15 seconds.
default_metadata: default metadata to write in new objects.
xmay be a FileSelector or a character vector of paths. Returns a list of FileInfo
$CreateDir(path, recursive = TRUE): Create a directory and subdirectories.
$DeleteDir(path): Delete a directory and its contents, recursively.
$DeleteDirContents(path): Delete a directory's contents, recursively. Like
$DeleteDir(), but doesn't delete the directory itself. Passing an empty path (
"") will wipe the entire filesystem tree.
$DeleteFile(path): Delete a file.
$DeleteFiles(paths): Delete many files. The default implementation issues individual delete operations in sequence.
$Move(src, dest): Move / rename a file or directory. If the destination exists: if it is a non-empty directory, an error is returned otherwise, if it has the same type as the source, it is replaced otherwise, behavior is unspecified (implementation-dependent).
$CopyFile(src, dest): Copy a file. If the destination exists and is a directory, an error is returned. Otherwise, it is replaced.
$OpenInputStream(path): Open an input stream for sequential reading.
$OpenInputFile(path): Open an input file for random access reading.
$OpenOutputStream(path): Open an output stream for sequential writing.
$OpenAppendStream(path): Open an output stream for appending.
$type_name: string filesystem type name, such as "local", "s3", etc.
$region: string AWS region, for
SubTreeFileSystem, the path in
$base_fswhich is considered root in this
$CreateDir() on a top-level directory creates a new bucket.
When S3FileSystem creates new buckets (assuming allow_bucket_creation is TRUE),
it does not pass any non-default settings. In AWS S3, the bucket and all
objects will be not publicly visible, and will have no bucket policies
and no resource tags. To have more control over how buckets are created,
use a different API to create them.