pyarrow.dataset.JsonFileFormat#
- class pyarrow.dataset.JsonFileFormat(default_fragment_scan_options=None, ParseOptions parse_options=None, ReadOptions read_options=None)#
Bases:
FileFormat
FileFormat for JSON files.
- Parameters:
- default_fragment_scan_options
JsonFragmentScanOptions
Default options for fragments scan.
- parse_options
pyarrow.json.ParseOptions
Options regarding json parsing.
- read_options
pyarrow.json.ReadOptions
General read options.
- default_fragment_scan_options
- __init__(*args, **kwargs)#
Methods
__init__
(*args, **kwargs)equals
(self, JsonFileFormat other)- Parameters:
inspect
(self, file[, filesystem])Infer the schema of a file.
make_fragment
(self, file[, filesystem, ...])Make a FileFragment from a given file.
make_write_options
(self)Attributes
- default_extname#
- default_fragment_scan_options#
- equals(self, JsonFileFormat other)#
- Parameters:
- Returns:
- inspect(self, file, filesystem=None)#
Infer the schema of a file.
- make_fragment(self, file, filesystem=None, Expression partition_expression=None, *, file_size=None)#
Make a FileFragment from a given file.
- Parameters:
- filefile-like object, path-like or
str
The file or file path to make a fragment from.
- filesystem
Filesystem
, optional If filesystem is given, file must be a string and specifies the path of the file to read from the filesystem.
- partition_expression
Expression
, optional An expression that is guaranteed true for all rows in the fragment. Allows fragment to be potentially skipped while scanning with a filter.
- file_size
int
, optional The size of the file in bytes. Can improve performance with high-latency filesystems when file size needs to be known before reading.
- filefile-like object, path-like or
- Returns:
- fragment
Fragment
The file fragment
- fragment
- make_write_options(self)#