File system classes: Apache Arrow GLib Reference Manual

File system classes

Functions

GArrowFileInfo *	garrow_file_info_new ()
gboolean	garrow_file_info_equal ()
gboolean	garrow_file_info_is_file ()
gboolean	garrow_file_info_is_dir ()
gchar *	garrow_file_info_to_string ()
GArrowFileSystem *	garrow_file_system_create ()
gchar *	garrow_file_system_get_type_name ()
GArrowFileInfo *	garrow_file_system_get_file_info ()
GList *	garrow_file_system_get_file_infos_paths ()
GList *	garrow_file_system_get_file_infos_selector ()
gboolean	garrow_file_system_create_dir ()
gboolean	garrow_file_system_delete_dir ()
gboolean	garrow_file_system_delete_dir_contents ()
gboolean	garrow_file_system_delete_file ()
gboolean	garrow_file_system_delete_files ()
gboolean	garrow_file_system_move ()
gboolean	garrow_file_system_copy_file ()
GArrowInputStream *	garrow_file_system_open_input_stream ()
GArrowSeekableInputStream *	garrow_file_system_open_input_file ()
GArrowOutputStream *	garrow_file_system_open_output_stream ()
GArrowOutputStream *	garrow_file_system_open_append_stream ()
GArrowSubTreeFileSystem *	garrow_sub_tree_file_system_new ()
GArrowSlowFileSystem *	garrow_slow_file_system_new_average_latency ()
GArrowSlowFileSystem *	garrow_slow_file_system_new_average_latency_and_seed ()

Properties

gchar *	base-name	Read
gchar *	dir-name	Read
gchar *	extension	Read
gint64	mtime	Read / Write
gchar *	path	Read / Write
gint64	size	Read / Write
GArrowFileType	type	Read / Write
gboolean	allow-not-found	Read / Write
gchar *	base-dir	Read / Write
gint	max-recursion	Read / Write
gboolean	recursive	Read / Write
gpointer	file-system	Write / Construct Only
GArrowFileSystem *	base-file-system	Read / Write / Construct Only
GArrowFileSystem *	base-file-system	Read / Write / Construct Only

Types and Values

typedef	GArrowTimePoint
enum	GArrowFileType
#define	GARROW_TYPE_FILE_INFO
struct	GArrowFileInfoClass
#define	GARROW_TYPE_FILE_SELECTOR
struct	GArrowFileSelectorClass
#define	GARROW_TYPE_FILE_SYSTEM
struct	GArrowFileSystemClass
#define	GARROW_TYPE_SUB_TREE_FILE_SYSTEM
struct	GArrowSubTreeFileSystemClass
#define	GARROW_TYPE_SLOW_FILE_SYSTEM
struct	GArrowSlowFileSystemClass
#define	GARROW_TYPE_MOCK_FILE_SYSTEM
struct	GArrowMockFileSystemClass
#define	GARROW_TYPE_HDFS_FILE_SYSTEM
struct	GArrowHDFSFileSystemClass
#define	GARROW_TYPE_S3_FILE_SYSTEM
struct	GArrowS3FileSystemClass
	GArrowFileInfo
	GArrowFileSelector
	GArrowFileSystem
	GArrowHDFSFileSystem
	GArrowMockFileSystem
	GArrowS3FileSystem
	GArrowSlowFileSystem
	GArrowSubTreeFileSystem

Object Hierarchy

    GEnum
    ╰── GArrowFileType
    GObject
    ├── GArrowFileInfo
    ├── GArrowFileSelector
    ╰── GArrowFileSystem
        ├── GArrowHDFSFileSystem
        ├── GArrowLocalFileSystem
        ├── GArrowMockFileSystem
        ├── GArrowS3FileSystem
        ├── GArrowSlowFileSystem
        ╰── GArrowSubTreeFileSystem

Includes

#include <arrow-glib/arrow-glib.h>

Description

GArrowFileInfo is a class for information for a file system entry.

GArrowFileSelector is a class for a selector for file system APIs.

GArrowFileSystem is an interface for file system.

GArrowSubTreeFileSystem is a delegator to another file system that is a logical view of a subtree of a file system, such as a directory in a local file system.

GArrowSlowFileSystem is a delegator to another file system. This inserts latencies at various points.

GArrowMockFileSystem is a class for mock file system that holds its contents in memory.

GArrowHDFSFileSystem is a class for HDFS-backed file system.

GArrowS3FileSystem is a class for S3-backed file system.

Functions

garrow_file_info_new ()

GArrowFileInfo *
garrow_file_info_new (void);

Returns

A newly created GArrowFileInfo.

Since: 0.17.0

garrow_file_info_equal ()

gboolean
garrow_file_info_equal (GArrowFileInfo *file_info,
                        GArrowFileInfo *other_file_info);

Parameters

file_info	A GArrowFileInfo.
other_file_info	A GArrowFileInfo to be compared.

Returns

TRUE if both of them have the same data, FALSE otherwise.

Since: 0.17.0

garrow_file_info_is_file ()

gboolean
garrow_file_info_is_file (GArrowFileInfo *file_info);

Parameters

file_info

A GArrowFileInfo.

Returns

TRUE if the entry is a file, FALSE otherwise.

Since: 0.17.0

garrow_file_info_is_dir ()

gboolean
garrow_file_info_is_dir (GArrowFileInfo *file_info);

Parameters

file_info

A GArrowFileInfo.

Returns

TRUE if the entry is a directory, FALSE otherwise.

Since: 0.17.0

garrow_file_info_to_string ()

gchar *
garrow_file_info_to_string (GArrowFileInfo *file_info);

Parameters

file_info

A GArrowFileInfo.

Returns

The string representation of the file statistics.

It should be freed with g_free() when no longer needed.

Since: 0.17.0

garrow_file_system_create ()

GArrowFileSystem *
garrow_file_system_create (const gchar *uri,
                           GError **error);

This is a factory function to create a specific GArrowFileSystem object.

Parameters

uri	An URI to specify file system with options. If you only have an absolute path, `g_filename_to_uri()` will help you.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The newly created file system that is an object of a subclass of GArrowFileSystem.

[nullable][transfer full]

Since: 3.0.0

garrow_file_system_get_type_name ()

gchar *
garrow_file_system_get_type_name (GArrowFileSystem *file_system);

Parameters

file_system

A GArrowFileSystem.

Returns

The name of file system type.

It should be freed with g_free() when no longer needed.

Since: 0.17.0

garrow_file_system_get_file_info ()

GArrowFileInfo *
garrow_file_system_get_file_info (GArrowFileSystem *file_system,
                                  const gchar *path,
                                  GError **error);

Get information for the given target.

Any symlink is automatically dereferenced, recursively. A non-existing or unreachable file returns an OK status and has a GArrowFileType of value GARROW_FILE_TYPE_NOT_FOUND. An error status indicates a truly exceptional condition (low-level I/O error, etc.).

Parameters

file_system	A GArrowFileSystem.
path	The path of the target.
error	Return location for a GError or `NULL`.	[nullable]

Returns

A GArrowFileInfo.

[nullable][transfer full]

Since: 0.17.0

garrow_file_system_get_file_infos_paths ()

GList *
garrow_file_system_get_file_infos_paths
                               (GArrowFileSystem *file_system,
                                const gchar **paths,
                                gsize n_paths,
                                GError **error);

Get information same as garrow_file_system_get_file_info() for the given many targets at once.

Parameters

file_system	A GArrowFileSystem.
paths	The paths of the targets.	[array length=n_paths]
n_paths	The number of items in `paths` .
error	Return location for a GError or `NULL`.	[nullable]

Returns

A list of GArrowFileInfo.

[element-type GArrowFileInfo][transfer full]

Since: 0.17.0

garrow_file_system_get_file_infos_selector ()

GList *
garrow_file_system_get_file_infos_selector
                               (GArrowFileSystem *file_system,
                                GArrowFileSelector *file_selector,
                                GError **error);

Get information same as garrow_file_system_get_file_info() according to a selector.

The selector's base directory will not be part of the results, even if it exists.

Parameters

file_system	A GArrowFileSystem.
file_selector	A GArrowFileSelector.
error	Return location for a GError or `NULL`.	[nullable]

Returns

A list of GArrowFileInfo.

[element-type GArrowFileInfo][transfer full]

Since: 0.17.0

garrow_file_system_create_dir ()

gboolean
garrow_file_system_create_dir (GArrowFileSystem *file_system,
                               const gchar *path,
                               gboolean recursive,
                               GError **error);

Create a directory and subdirectories. This function succeeds if the directory already exists.

Parameters

file_system	A GArrowFileSystem.
path	The paths of the directory.
recursive	Whether creating directory recursively or not.
error	Return location for a GError or `NULL`.	[nullable]

Returns

TRUE on success, FALSE if there was an error.

Since: 0.17.0

garrow_file_system_delete_dir ()

gboolean
garrow_file_system_delete_dir (GArrowFileSystem *file_system,
                               const gchar *path,
                               GError **error);

Delete a directory and its contents, recursively.

Parameters

file_system	A GArrowFileSystem.
path	The paths of the directory.
error	Return location for a GError or `NULL`.	[nullable]

Returns

TRUE on success, FALSE if there was an error.

Since: 0.17.0

garrow_file_system_delete_dir_contents ()

gboolean
garrow_file_system_delete_dir_contents
                               (GArrowFileSystem *file_system,
                                const gchar *path,
                                GError **error);

Delete a directory's contents, recursively. Like garrow_file_system_delete_dir(), but doesn't delete the directory itself. Passing an empty path ("") will wipe the entire file system tree.

Parameters

file_system	A GArrowFileSystem.
path	The paths of the directory.
error	Return location for a GError or `NULL`.	[nullable]

Returns

TRUE on success, FALSE if there was an error.

Since: 0.17.0

garrow_file_system_delete_file ()

gboolean
garrow_file_system_delete_file (GArrowFileSystem *file_system,
                                const gchar *path,
                                GError **error);

Delete a file.

Parameters

file_system	A GArrowFileSystem.
path	The paths of the file to be delete.
error	Return location for a GError or `NULL`.	[nullable]

Returns

TRUE on success, FALSE if there was an error.

Since: 0.17.0

garrow_file_system_delete_files ()

gboolean
garrow_file_system_delete_files (GArrowFileSystem *file_system,
                                 const gchar **paths,
                                 gsize n_paths,
                                 GError **error);

Delete many files.

Parameters

file_system	A GArrowFileSystem.
paths	The paths of the files to be delete.	[array length=n_paths]
n_paths	The number of items in `paths` .
error	Return location for a GError or `NULL`.	[nullable]

Returns

TRUE on success, FALSE if there was an error.

Since: 0.17.0

garrow_file_system_move ()

gboolean
garrow_file_system_move (GArrowFileSystem *file_system,
                         const gchar *src,
                         const gchar *dest,
                         GError **error);

Move / rename a file or a directory. If the destination exists:

if it is a non-empty directory, an error is returned
otherwise, if it has the same type as the source, it is replaced
otherwise, behavior is unspecified (implementation-dependent).

Parameters

file_system	A GArrowFileSystem.
src	The path of the source file.
dest	The path of the destination.
error	Return location for a GError or `NULL`.	[nullable]

Returns

TRUE on success, FALSE if there was an error.

Since: 0.17.0

garrow_file_system_copy_file ()

gboolean
garrow_file_system_copy_file (GArrowFileSystem *file_system,
                              const gchar *src,
                              const gchar *dest,
                              GError **error);

Copy a file. If the destination exists and is a directory, an error is returned. Otherwise, it is replaced.

Parameters

file_system	A GArrowFileSystem.
src	The path of the source file.
dest	The path of the destination.
error	Return location for a GError or `NULL`.	[nullable]

Returns

TRUE on success, FALSE if there was an error.

Since: 0.17.0

garrow_file_system_open_input_stream ()

GArrowInputStream *
garrow_file_system_open_input_stream (GArrowFileSystem *file_system,
                                      const gchar *path,
                                      GError **error);

Open an input stream for sequential reading.

Parameters

file_system	A GArrowFileSystem.
path	The path of the input stream.
error	Return location for a GError or `NULL`.	[nullable]

Returns

A newly created GArrowInputStream.

[nullable][transfer full]

Since: 0.17.0

garrow_file_system_open_input_file ()

GArrowSeekableInputStream *
garrow_file_system_open_input_file (GArrowFileSystem *file_system,
                                    const gchar *path,
                                    GError **error);

Open an input file for random access reading.

Parameters

file_system	A GArrowFileSystem.
path	The path of the input file.
error	Return location for a GError or `NULL`.	[nullable]

Returns

A newly created GArrowSeekableInputStream.

[nullable][transfer full]

Since: 0.17.0

garrow_file_system_open_output_stream ()

GArrowOutputStream *
garrow_file_system_open_output_stream (GArrowFileSystem *file_system,
                                       const gchar *path,
                                       GError **error);

Open an output stream for sequential writing. If the target already exists, the existing data is truncated.

Parameters

file_system	A GArrowFileSystem.
path	The path of the output stream.
error	Return location for a GError or `NULL`.	[nullable]

Returns

A newly created GArrowOutputStream.

[nullable][transfer full]

Since: 0.17.0

garrow_file_system_open_append_stream ()

GArrowOutputStream *
garrow_file_system_open_append_stream (GArrowFileSystem *file_system,
                                       const gchar *path,
                                       GError **error);

Open an output stream for appending. If the target doesn't exist, a new empty file is created.

Parameters

file_system	A GArrowFileSystem.
path	The path of the output stream.
error	Return location for a GError or `NULL`.	[nullable]

Returns

A newly created GArrowOutputStream for appending.

[nullable][transfer full]

Since: 0.17.0

garrow_sub_tree_file_system_new ()

GArrowSubTreeFileSystem *
garrow_sub_tree_file_system_new (const gchar *base_path,
                                 GArrowFileSystem *base_file_system);

Parameters

base_path	A base path of the sub tree file system.
base_file_system	A GArrowFileSystem as the base file system.

Returns

A newly created GArrowSubTreeFileSystem.

[transfer full]

Since: 0.17.0

garrow_slow_file_system_new_average_latency ()

GArrowSlowFileSystem *
garrow_slow_file_system_new_average_latency
                               (GArrowFileSystem *base_file_system,
                                gdouble average_latency);

The latency is normally distributed with a standard deviation of average_latency * 0.1.

The random seed is given by the default random device.

Parameters

base_file_system	A GArrowFileSystem as the base file system.
average_latency	The average value of the latency.

Returns

A newly created GArrowSlowFileSystem.

[transfer full]

Since: 0.17.0

garrow_slow_file_system_new_average_latency_and_seed ()

GArrowSlowFileSystem *
garrow_slow_file_system_new_average_latency_and_seed
                               (GArrowFileSystem *base_file_system,
                                gdouble average_latency,
                                gint32 seed);

The latency is normally distributed with a standard deviation of average_latency * 0.1.

Parameters

base_file_system	A GArrowFileSystem as the base file system.
average_latency	The average value of the latency.
seed	A random seed.

Returns

A newly created GArrowSlowFileSystem.

[transfer full]

Since: 0.17.0

Types and Values

GArrowTimePoint

typedef gint64 GArrowTimePoint;

enum GArrowFileType

They are corresponding to arrow::fs::FileType values.

Members

GARROW_FILE_TYPE_NOT_FOUND	Entry is not found
GARROW_FILE_TYPE_UNKNOWN	Entry exists but its type is unknown
GARROW_FILE_TYPE_FILE	Entry is a regular file
GARROW_FILE_TYPE_DIR	Entry is a directory

Since: 1.0.0

GARROW_TYPE_FILE_INFO

#define GARROW_TYPE_FILE_INFO (garrow_file_info_get_type())

struct GArrowFileInfoClass

struct GArrowFileInfoClass {
  GObjectClass parent_class;
};

GARROW_TYPE_FILE_SELECTOR

#define GARROW_TYPE_FILE_SELECTOR (garrow_file_selector_get_type())

struct GArrowFileSelectorClass

struct GArrowFileSelectorClass {
  GObjectClass parent_class;
};

GARROW_TYPE_FILE_SYSTEM

#define GARROW_TYPE_FILE_SYSTEM (garrow_file_system_get_type())

struct GArrowFileSystemClass

struct GArrowFileSystemClass {
  GObjectClass parent_class;
};

GARROW_TYPE_SUB_TREE_FILE_SYSTEM

#define GARROW_TYPE_SUB_TREE_FILE_SYSTEM (garrow_sub_tree_file_system_get_type())

struct GArrowSubTreeFileSystemClass

struct GArrowSubTreeFileSystemClass {
  GArrowFileSystemClass parent_class;
};

GARROW_TYPE_SLOW_FILE_SYSTEM

#define GARROW_TYPE_SLOW_FILE_SYSTEM (garrow_slow_file_system_get_type())

struct GArrowSlowFileSystemClass

struct GArrowSlowFileSystemClass {
  GArrowFileSystemClass parent_class;
};

GARROW_TYPE_MOCK_FILE_SYSTEM

#define GARROW_TYPE_MOCK_FILE_SYSTEM (garrow_mock_file_system_get_type())

struct GArrowMockFileSystemClass

struct GArrowMockFileSystemClass {
  GArrowFileSystemClass parent_class;
};

GARROW_TYPE_HDFS_FILE_SYSTEM

#define GARROW_TYPE_HDFS_FILE_SYSTEM (garrow_hdfs_file_system_get_type())

struct GArrowHDFSFileSystemClass

struct GArrowHDFSFileSystemClass {
  GArrowFileSystemClass parent_class;
};

GARROW_TYPE_S3_FILE_SYSTEM

#define GARROW_TYPE_S3_FILE_SYSTEM (garrow_s3_file_system_get_type())

struct GArrowS3FileSystemClass

struct GArrowS3FileSystemClass {
  GArrowFileSystemClass parent_class;
};

GArrowFileInfo

typedef struct _GArrowFileInfo GArrowFileInfo;

GArrowFileSelector

typedef struct _GArrowFileSelector GArrowFileSelector;

GArrowFileSystem

typedef struct _GArrowFileSystem GArrowFileSystem;

GArrowHDFSFileSystem

typedef struct _GArrowHDFSFileSystem GArrowHDFSFileSystem;

GArrowMockFileSystem

typedef struct _GArrowMockFileSystem GArrowMockFileSystem;

GArrowS3FileSystem

typedef struct _GArrowS3FileSystem GArrowS3FileSystem;

GArrowSlowFileSystem

typedef struct _GArrowSlowFileSystem GArrowSlowFileSystem;

GArrowSubTreeFileSystem

typedef struct _GArrowSubTreeFileSystem GArrowSubTreeFileSystem;

Property Details

The `“base-name”` property

  “base-name”                gchar *

The file base name (component after the last directory separator).

Owner: GArrowFileInfo

Flags: Read

Default value: ""

Since: 0.17.0

The `“dir-name”` property

  “dir-name”                 gchar *

The directory base name (component before the file base name).

Owner: GArrowFileInfo

Flags: Read

Default value: ""

Since: 0.17.0

The `“extension”` property

  “extension”                gchar *

The file extension (excluding the dot).

Owner: GArrowFileInfo

Flags: Read

Default value: ""

Since: 0.17.0

The `“mtime”` property

  “mtime”                    gint64

The time of last modification, if available.

Owner: GArrowFileInfo

Flags: Read / Write

Allowed values: >= -1

Default value: -1

Since: 0.17.0

The `“path”` property

  “path”                     gchar *

The full file path in the file system.

Owner: GArrowFileInfo

Flags: Read / Write

Default value: ""

Since: 0.17.0

The `“size”` property

  “size”                     gint64

The size in bytes, if available Only regular files are guaranteed to have a size.

Owner: GArrowFileInfo

Flags: Read / Write

Allowed values: >= -1

Default value: -1

Since: 0.17.0

The `“type”` property

  “type”                     GArrowFileType

The type of the entry.

Owner: GArrowFileInfo

Flags: Read / Write

Default value: GARROW_FILE_TYPE_UNKNOWN

Since: 0.17.0

The `“allow-not-found”` property

  “allow-not-found”          gboolean

The behavior if base_dir isn't found in the file system. If false, an error is returned. If true, an empty selection is returned.

Owner: GArrowFileSelector

Flags: Read / Write

Default value: FALSE

Since: 0.17.0

The `“base-dir”` property

  “base-dir”                 gchar *

The directory in which to select files. If the path exists but doesn't point to a directory, this should be an error.

Owner: GArrowFileSelector

Flags: Read / Write

Default value: ""

Since: 0.17.0

The `“max-recursion”` property

  “max-recursion”            gint

The maximum number of subdirectories to recurse into.

Owner: GArrowFileSelector

Flags: Read / Write

Allowed values: >= 0

Default value: 2147483647

Since: 0.17.0

The `“recursive”` property

  “recursive”                gboolean

Whether to recurse into subdirectories.

Owner: GArrowFileSelector

Flags: Read / Write

Default value: FALSE

Since: 0.17.0

The `“file-system”` property

  “file-system”              gpointer

The raw std::shared<arrow::fs::FileSystem> *.

Owner: GArrowFileSystem

Flags: Write / Construct Only

The `“base-file-system”` property

  “base-file-system”         GArrowFileSystem *

The base GArrowFileSystem.

Owner: GArrowSlowFileSystem

Flags: Read / Write / Construct Only

The `“base-file-system”` property

  “base-file-system”         GArrowFileSystem *

The base GArrowFileSystem.

Owner: GArrowSubTreeFileSystem

Flags: Read / Write / Construct Only