pyarrow.dataset.Partitioning

class pyarrow.dataset.Partitioning

Bases: _Weakrefable

__init__(*args, **kwargs)

Methods

__init__(*args, **kwargs)
format(self, expr)	Convert a filter expression into a tuple of (directory, filename) using the current partitioning scheme
parse(self, path)	Parse a path into a partition expression.

Attributes

schema

The arrow Schema attached to the partitioning.

format(self, expr)

Convert a filter expression into a tuple of (directory, filename) using the current partitioning scheme

Parameters:

exprpyarrow.dataset.Expression

Returns:

tuple[str, str]

Examples

Specify the Schema for paths like “/2009/June”:

>>> import pyarrow as pa
>>> import pyarrow.dataset as ds
>>> import pyarrow.compute as pc
>>> part = ds.partitioning(pa.schema([("year", pa.int16()),
...                                   ("month", pa.string())]))
>>> part.format(
...     (pc.field("year") == 1862) & (pc.field("month") == "Jan")
... )
('1862/Jan', '')

parse(self, path)

Parse a path into a partition expression.

Parameters:

pathstr

Returns:

pyarrow.dataset.Expression

schema

The arrow Schema attached to the partitioning.