pyarrow.acero.ScanNodeOptions¶
- class pyarrow.acero.ScanNodeOptions(dataset, **kwargs)¶
Bases:
_ScanNodeOptionsA Source node which yields batches from a Dataset scan.
This is the option class for the “scan” node factory.
This node is capable of applying pushdown projections or filters to the file readers which reduce the amount of data that needs to be read (if supported by the file format). But note that this does not construct associated filter or project nodes to perform the final filtering or projection. Rather, you may supply the same filter expression or projection to the scan node that you also supply to the filter or project node.
Yielded batches will be augmented with fragment/batch indices to enable stable ordering for simple ExecPlans.
- Parameters:
- dataset
pyarrow.dataset.Dataset The table which acts as the data source.
- **kwargs
dict, optional Scan options. See Scanner.from_dataset for possible arguments.
- dataset
- __init__(self, Dataset dataset, **kwargs)¶
Methods
__init__(self, Dataset dataset, **kwargs)