Module org.apache.arrow.dataset

Package org.apache.arrow.dataset.scanner.csv

Class CsvFragmentScanOptions

java.lang.Object

org.apache.arrow.dataset.scanner.csv.CsvFragmentScanOptions

All Implemented Interfaces:

FragmentScanOptions

public class CsvFragmentScanOptions
extends Object
implements FragmentScanOptions

Constructor Summary

Constructors

Constructor	Description
CsvFragmentScanOptions(CsvConvertOptions convertOptions, Map<String,String> readOptions, Map<String,String> parseOptions)	CSV scan options, map to CPP struct CsvFragmentScanOptions.

Method Summary

Modifier and Type	Method	Description
FileFormat	fileFormat()	File format.
CsvConvertOptions	getConvertOptions()
Map<String,String>	getParseOptions()
Map<String,String>	getReadOptions()
String[]	serialize()	This is an internal function to invoke by serializer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

CsvFragmentScanOptions

public CsvFragmentScanOptions(CsvConvertOptions convertOptions,
 Map<String,String> readOptions,
 Map<String,String> parseOptions)

CSV scan options, map to CPP struct CsvFragmentScanOptions. The key in config map is the field name of mapping cpp struct

Currently, multi-valued options (which are std::vector values in C++) only support having a single value set. For example, for the null_values option, only one string can be set as the null value.

Parameters:

convertOptions - similar to CsvFragmentScanOptions#convert_options in CPP, the ArrowSchema represents column_types, convert data option such as null value recognition.

readOptions - similar to CsvFragmentScanOptions#read_options in CPP, specify how to read the file such as block_size

parseOptions - similar to CsvFragmentScanOptions#parse_options in CPP, parse file option such as delimiter

Method Details

fileFormat

public FileFormat fileFormat()

File format.

Specified by:

fileFormat in interface FragmentScanOptions

Returns:

file format.

serialize

public String[] serialize()

This is an internal function to invoke by serializer. Serialize this class to string array and then called by JNI call.

Specified by:

serialize in interface FragmentScanOptions

Returns:

string array as Map JNI bridge format.

getConvertOptions

public CsvConvertOptions getConvertOptions()

getReadOptions

public Map<String,String> getReadOptions()

getParseOptions

public Map<String,String> getParseOptions()