pyarrow.dataset.CsvFileFormat#

class pyarrow.dataset.CsvFileFormat(ParseOptions parse_options=None, default_fragment_scan_options=None, ConvertOptions convert_options=None, ReadOptions read_options=None)#

Bases: FileFormat

FileFormat for CSV files.

Parameters:

parse_optionspyarrow.csv.ParseOptions: Options regarding CSV parsing.
default_fragment_scan_optionsCsvFragmentScanOptions: Default options for fragments scan.
convert_optionspyarrow.csv.ConvertOptions: Options regarding value conversion.
read_optionspyarrow.csv.ReadOptions: General read options.

__init__(*args, **kwargs)#

Methods

`__init__`(args, *kwargs)
`equals`(self, CsvFileFormat other)	Parameters:
`inspect`(self, file[, filesystem])	Infer the schema of a file.
`make_fragment`(self, file[, filesystem, ...])	Make a FileFragment from a given file.
`make_write_options`(self, **kwargs)	Parameters:

Attributes

`default_extname`
`default_fragment_scan_options`
`parse_options`

default_extname#

default_fragment_scan_options#

equals(self, CsvFileFormat other)#

Parameters:

otherpyarrow.dataset.CsvFileFormat

Returns:

bool

inspect(self, file, filesystem=None)#

Infer the schema of a file.

Parameters:

filefile-like object, path-like or str: The file or file path to infer a schema from.
filesystemFilesystem, optional: If filesystem is given, file must be a string and specifies the path of the file to read from the filesystem.

Returns:

schemaSchema: The schema inferred from the file

make_fragment(self, file, filesystem=None, Expression partition_expression=None, *, file_size=None)#

Make a FileFragment from a given file.

Parameters:

filefile-like object, path-like or str: The file or file path to make a fragment from.
filesystemFilesystem, optional: If filesystem is given, file must be a string and specifies the path of the file to read from the filesystem.
partition_expressionExpression, optional: An expression that is guaranteed true for all rows in the fragment. Allows fragment to be potentially skipped while scanning with a filter.
file_sizeint, optional: The size of the file in bytes. Can improve performance with high-latency filesystems when file size needs to be known before reading.

Returns:

fragmentFragment: The file fragment

make_write_options(self, **kwargs)#

Parameters:

**kwargsdict

Returns:

pyarrow.csv.WriteOptions

parse_options#