org.apache.arrow.adapter.jdbc

Class JdbcToArrowUtils

java.lang.Object

org.apache.arrow.adapter.jdbc.JdbcToArrowUtils

public class JdbcToArrowUtils
extends Object

Class that does most of the work to convert JDBC ResultSet data into Arrow columnar format Vector objects.

Since:

0.10.0

Constructor Summary

Constructors

Constructor and Description
JdbcToArrowUtils()

Method Summary

Modifier and Type	Method and Description
static ArrowType	getArrowTypeFromJdbcType(JdbcFieldInfo fieldInfo, Calendar calendar) Converts the provided JDBC type to its respective ArrowType counterpart.
static JdbcConsumer	getConsumer(ArrowType arrowType, int columnIndex, boolean nullable, FieldVector vector, JdbcToArrowConfig config) Default function used for JdbcConsumerFactory.
static Calendar	getUtcCalendar() Returns the instance of a {java.util.Calendar} with the UTC time zone and root locale.
static Schema	jdbcToArrowSchema(ParameterMetaData parameterMetaData, Calendar calendar) Create Arrow Schema object for the given JDBC ResultSetMetaData.
static Schema	jdbcToArrowSchema(ResultSetMetaData rsmd, Calendar calendar) Create Arrow Schema object for the given JDBC ResultSetMetaData.
static Schema	jdbcToArrowSchema(ResultSetMetaData rsmd, JdbcToArrowConfig config) Create Arrow Schema object for the given JDBC ResultSetMetaData.
static void	jdbcToArrowVectors(ResultSet rs, VectorSchemaRoot root, Calendar calendar) Iterate the given JDBC ResultSet object to fetch the data and transpose it to populate the given Arrow Vector objects.
static void	jdbcToArrowVectors(ResultSet rs, VectorSchemaRoot root, JdbcToArrowConfig config) Iterate the given JDBC ResultSet object to fetch the data and transpose it to populate the given Arrow Vector objects.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JdbcToArrowUtils

public JdbcToArrowUtils()

Method Detail

getUtcCalendar

public static Calendar getUtcCalendar()

Returns the instance of a {java.util.Calendar} with the UTC time zone and root locale.

jdbcToArrowSchema

public static Schema jdbcToArrowSchema(ResultSetMetaData rsmd,
                                       Calendar calendar)
                                throws SQLException

Create Arrow Schema object for the given JDBC ResultSetMetaData.

Parameters:

rsmd - The ResultSetMetaData containing the results, to read the JDBC metadata from.

calendar - The calendar to use the time zone field of, to construct Timestamp fields from.

Returns:

Schema

Throws:

SQLException - on error

jdbcToArrowSchema

public static Schema jdbcToArrowSchema(ParameterMetaData parameterMetaData,
                                       Calendar calendar)
                                throws SQLException

Create Arrow Schema object for the given JDBC ResultSetMetaData.

Parameters:

parameterMetaData - The ResultSetMetaData containing the results, to read the JDBC metadata from.

calendar - The calendar to use the time zone field of, to construct Timestamp fields from.

Returns:

Schema

Throws:

SQLException - on error

getArrowTypeFromJdbcType

public static ArrowType getArrowTypeFromJdbcType(JdbcFieldInfo fieldInfo,
                                                 Calendar calendar)

Converts the provided JDBC type to its respective ArrowType counterpart.

Parameters:

fieldInfo - the JdbcFieldInfo with information about the original JDBC type.

calendar - the Calendar to use for datetime data types.

Returns:

a new ArrowType.

jdbcToArrowSchema

public static Schema jdbcToArrowSchema(ResultSetMetaData rsmd,
                                       JdbcToArrowConfig config)
                                throws SQLException

Create Arrow Schema object for the given JDBC ResultSetMetaData.

If JdbcToArrowConfig.shouldIncludeMetadata() returns true, the following fields will be added to the FieldType.getMetadata():

• Constants.SQL_CATALOG_NAME_KEY representing ResultSetMetaData.getCatalogName(int)
• Constants.SQL_TABLE_NAME_KEY representing ResultSetMetaData.getTableName(int)
• Constants.SQL_COLUMN_NAME_KEY representing ResultSetMetaData.getColumnLabel(int)
• Constants.SQL_TYPE_KEY representing ResultSetMetaData.getColumnTypeName(int)

If any columns are of type Types.ARRAY, the configuration object will be used to look up the array sub-type field. The JdbcToArrowConfig.getArraySubTypeByColumnIndex(int) method will be checked first, followed by the JdbcToArrowConfig.getArraySubTypeByColumnName(String) method.

Parameters:

rsmd - The ResultSetMetaData containing the results, to read the JDBC metadata from.

config - The configuration to use when constructing the schema.

Returns:

Schema

Throws:

SQLException - on error

IllegalArgumentException - if rsmd contains an Types.ARRAY but the config does not have a sub-type definition for it.

jdbcToArrowVectors

public static void jdbcToArrowVectors(ResultSet rs,
                                      VectorSchemaRoot root,
                                      Calendar calendar)
                               throws SQLException,
                                      IOException

Iterate the given JDBC ResultSet object to fetch the data and transpose it to populate the given Arrow Vector objects.

Parameters:

rs - ResultSet to use to fetch the data from underlying database

root - Arrow VectorSchemaRoot object to populate

calendar - The calendar to use when reading Date, Time, or Timestamp data types from the ResultSet, or null if not converting.

Throws:

SQLException - on error

IOException

jdbcToArrowVectors

public static void jdbcToArrowVectors(ResultSet rs,
                                      VectorSchemaRoot root,
                                      JdbcToArrowConfig config)
                               throws SQLException,
                                      IOException

Iterate the given JDBC ResultSet object to fetch the data and transpose it to populate the given Arrow Vector objects.

Parameters:

rs - ResultSet to use to fetch the data from underlying database

root - Arrow VectorSchemaRoot object to populate

config - The configuration to use when reading the data.

Throws:

SQLException - on error

JdbcConsumerException - on error from VectorConsumer

IOException

getConsumer

public static JdbcConsumer getConsumer(ArrowType arrowType,
                                       int columnIndex,
                                       boolean nullable,
                                       FieldVector vector,
                                       JdbcToArrowConfig config)

Default function used for JdbcConsumerFactory. This function gets a JdbcConsumer for the given column based on the Arrow type and provided vector.

Parameters:

arrowType - Arrow type for the column.

columnIndex - Column index to fetch from the ResultSet

nullable - Whether the value is nullable or not

vector - Vector to store the consumed value

config - Associated JdbcToArrowConfig, used mainly for the Calendar.

Returns:

JdbcConsumer