ADBC
Arrow Database Connectivity
Loading...
Searching...
No Matches
Partitioned Results

Classes

struct  AdbcPartitions
 The partitions of a distributed/partitioned result set. More...
 

Functions

AdbcStatusCode AdbcStatementExecutePartitions (struct AdbcStatement *statement, struct ArrowSchema *schema, struct AdbcPartitions *partitions, int64_t *rows_affected, struct AdbcError *error)
 Execute a statement and get the results as a partitioned result set.
 

Detailed Description

Some backends may internally partition the results. These partitions are exposed to clients who may wish to integrate them with a threaded or distributed execution model, where partitions can be divided among threads or machines and fetched in parallel.

To use partitioning, execute the statement with AdbcStatementExecutePartitions to get the partition descriptors. Call AdbcConnectionReadPartition to turn the individual descriptors into ArrowArrayStream instances. This may be done on a different connection than the one the partition was created with, or even in a different process on another machine.

Drivers are not required to support partitioning.


Class Documentation

◆ AdbcPartitions

struct AdbcPartitions

The partitions of a distributed/partitioned result set.

Public Attributes

size_t num_partitions
 The number of partitions.
 
const uint8_t ** partitions
 The partitions of the result set, where each entry (up to num_partitions entries) is an opaque identifier that can be passed to AdbcConnectionReadPartition.
 
const size_t * partition_lengths
 The length of each corresponding entry in partitions.
 
void * private_data
 Opaque implementation-defined state. This field is NULLPTR iff the connection is unintialized/freed.
 
void(* release )(struct AdbcPartitions *partitions)
 Release the contained partitions.
 

Member Data Documentation

◆ num_partitions

size_t AdbcPartitions::num_partitions

The number of partitions.

◆ partition_lengths

const size_t* AdbcPartitions::partition_lengths

The length of each corresponding entry in partitions.

◆ partitions

const uint8_t** AdbcPartitions::partitions

The partitions of the result set, where each entry (up to num_partitions entries) is an opaque identifier that can be passed to AdbcConnectionReadPartition.

◆ private_data

void* AdbcPartitions::private_data

Opaque implementation-defined state. This field is NULLPTR iff the connection is unintialized/freed.

◆ release

void(* AdbcPartitions::release) (struct AdbcPartitions *partitions)

Release the contained partitions.

Unlike other structures, this is an embedded callback to make it easier for the driver manager and driver to cooperate.

Function Documentation

◆ AdbcStatementExecutePartitions()

AdbcStatusCode AdbcStatementExecutePartitions ( struct AdbcStatement * statement,
struct ArrowSchema * schema,
struct AdbcPartitions * partitions,
int64_t * rows_affected,
struct AdbcError * error )

Execute a statement and get the results as a partitioned result set.

Parameters
[in]statementThe statement to execute.
[out]schemaThe schema of the result set.
[out]partitionsThe result partitions.
[out]rows_affectedThe number of rows affected if known, else -1. Pass NULL if the client does not want this information.
[out]errorAn optional location to return an error message if necessary.
Returns
ADBC_STATUS_NOT_IMPLEMENTED if the driver does not support partitioned results