Arrow Flight RPC

Note

Flight is currently unstable. APIs are subject to change, though we don’t expect drastic changes.

Common Types

struct Action

An action to perform with the DoAction RPC.

Public Functions

arrow::Result<std::string> SerializeToString() const

Serialize this message to its wire-format representation.

Public Members

std::string type

The action type.

std::shared_ptr<Buffer> body

The action content as a Buffer.

Public Static Functions

static arrow::Result<Action> Deserialize(std::string_view serialized)

Deserialize this message from its wire-format representation.

struct ActionType

A type of action that can be performed with the DoAction RPC.

Public Functions

arrow::Result<std::string> SerializeToString() const

Serialize this message to its wire-format representation.

Public Members

std::string type

The name of the action.

std::string description

A human-readable description of the action.

Public Static Functions

static arrow::Result<ActionType> Deserialize(std::string_view serialized)

Deserialize this message from its wire-format representation.

class AddCallHeaders

A write-only wrapper around headers for an RPC call.

Public Functions

virtual void AddHeader(const std::string &key, const std::string &value) = 0

Add a header to be sent to the client.

Parameters:
  • key[in] The header name. Must be lowercase ASCII; some transports may reject invalid header names.

  • value[in] The header value. Some transports may only accept binary header values if the header name ends in “-bin”.

struct CallInfo

Information about an instance of a Flight RPC.

Public Members

FlightMethod method

The RPC method of this call.

struct Criteria

Opaque selection criteria for ListFlights RPC.

Public Functions

arrow::Result<std::string> SerializeToString() const

Serialize this message to its wire-format representation.

Public Members

std::string expression

Opaque criteria expression, dependent on server implementation.

Public Static Functions

static arrow::Result<Criteria> Deserialize(std::string_view serialized)

Deserialize this message from its wire-format representation.

struct FlightDescriptor

A request to retrieve or generate a dataset.

Public Functions

std::string ToString() const

Get a human-readable form of this descriptor.

arrow::Result<std::string> SerializeToString() const

Get the wire-format representation of this type.

Useful when interoperating with non-Flight systems (e.g. REST services) that may want to return Flight types.

Public Members

DescriptorType type

The descriptor type.

std::string cmd

Opaque value used to express a command.

Should only be defined when type is CMD

std::vector<std::string> path

List of strings identifying a particular dataset.

Should only be defined when type is PATH

Public Static Functions

static arrow::Result<FlightDescriptor> Deserialize(std::string_view serialized)

Parse the wire-format representation of this type.

Useful when interoperating with non-Flight systems (e.g. REST services) that may want to return Flight types.

struct FlightEndpoint

A flight ticket and list of locations where the ticket can be redeemed.

Public Functions

arrow::Result<std::string> SerializeToString() const

Serialize this message to its wire-format representation.

Public Members

Ticket ticket

Opaque ticket identify; use with DoGet RPC.

std::vector<Location> locations

List of locations where ticket can be redeemed.

If the list is empty, the ticket can only be redeemed on the current service where the ticket was generated

std::optional<Timestamp> expiration_time

Expiration time of this stream.

If present, clients may assume they can retry DoGet requests. Otherwise, clients should avoid retrying DoGet requests.

Public Static Functions

static arrow::Result<FlightEndpoint> Deserialize(std::string_view serialized)

Deserialize this message from its wire-format representation.

class FlightInfo

The access coordinates for retireval of a dataset, returned by GetFlightInfo.

Public Functions

arrow::Result<std::shared_ptr<Schema>> GetSchema(ipc::DictionaryMemo *dictionary_memo) const

Deserialize the Arrow schema of the dataset.

Populate any dictionary encoded fields into a DictionaryMemo for bookkeeping

Parameters:

dictionary_memo[inout] for dictionary bookkeeping, will be modified

Returns:

Arrrow result with the reconstructed Schema

inline const FlightDescriptor &descriptor() const

The descriptor associated with this flight, may not be set.

inline const std::vector<FlightEndpoint> &endpoints() const

A list of endpoints associated with the flight (dataset).

To consume the whole flight, all endpoints must be consumed

inline int64_t total_records() const

The total number of records (rows) in the dataset. If unknown, set to -1.

inline int64_t total_bytes() const

The total number of bytes in the dataset. If unknown, set to -1.

inline bool ordered() const

Whether endpoints are in the same order as the data.

arrow::Result<std::string> SerializeToString() const

Get the wire-format representation of this type.

Useful when interoperating with non-Flight systems (e.g. REST services) that may want to return Flight types.

bool Equals(const FlightInfo &other) const

Compare two FlightInfo for equality.

This will compare the serialized schema representations, NOT the logical equality of the schemas.

Public Static Functions

static arrow::Result<FlightInfo> Make(const Schema &schema, const FlightDescriptor &descriptor, const std::vector<FlightEndpoint> &endpoints, int64_t total_records, int64_t total_bytes, bool ordered = false)

Factory method to construct a FlightInfo.

static arrow::Result<std::unique_ptr<FlightInfo>> Deserialize(std::string_view serialized)

Parse the wire-format representation of this type.

Useful when interoperating with non-Flight systems (e.g. REST services) that may want to return Flight types.

struct Data
struct FlightPayload

Staging data structure for messages about to be put on the wire.

This structure corresponds to FlightData in the protocol.

Public Functions

Status Validate() const

Check that the payload can be written to the wire.

class FlightListing

An iterator to FlightInfo instances returned by ListFlights.

Subclassed by arrow::flight::SimpleFlightListing

Public Functions

virtual arrow::Result<std::unique_ptr<FlightInfo>> Next() = 0

Retrieve the next FlightInfo from the iterator.

Returns:

Arrow result with a single FlightInfo. Set to nullptr if there are none left.

enum class arrow::flight::FlightMethod : char

An enumeration of the RPC methods Flight implements.

Values:

enumerator Invalid
enumerator Handshake
enumerator ListFlights
enumerator GetFlightInfo
enumerator GetSchema
enumerator DoGet
enumerator DoPut
enumerator DoAction
enumerator ListActions
enumerator DoExchange
struct Location

A host location (a URI)

Public Functions

Location()

Initialize a blank location.

std::string ToString() const

Get a representation of this URI as a string.

std::string scheme() const

Get the scheme of this URI.

Public Static Functions

static arrow::Result<Location> Parse(const std::string &uri_string)

Initialize a location by parsing a URI string.

static arrow::Result<Location> ForGrpcTcp(const std::string &host, const int port)

Initialize a location for a non-TLS, gRPC-based Flight service from a host and port.

Parameters:
  • host[in] The hostname to connect to

  • port[in] The port

Returns:

Arrow result with the resulting location

static arrow::Result<Location> ForGrpcTls(const std::string &host, const int port)

Initialize a location for a TLS-enabled, gRPC-based Flight service from a host and port.

Parameters:
  • host[in] The hostname to connect to

  • port[in] The port

Returns:

Arrow result with the resulting location

static arrow::Result<Location> ForGrpcUnix(const std::string &path)

Initialize a location for a domain socket-based Flight service.

Parameters:

path[in] The path to the domain socket

Returns:

Arrow result with the resulting location

static arrow::Result<Location> ForScheme(const std::string &scheme, const std::string &host, const int port)

Initialize a location based on a URI scheme.

class MetadataRecordBatchReader

An interface to read Flight data with metadata.

Subclassed by arrow::flight::FlightMessageReader, arrow::flight::FlightStreamReader

Public Functions

virtual arrow::Result<std::shared_ptr<Schema>> GetSchema() = 0

Get the schema for this stream.

virtual arrow::Result<FlightStreamChunk> Next() = 0

Get the next message from Flight.

If the stream is finished, then the members of FlightStreamChunk will be nullptr.

virtual arrow::Result<std::vector<std::shared_ptr<RecordBatch>>> ToRecordBatches()

Consume entire stream as a vector of record batches.

virtual arrow::Result<std::shared_ptr<Table>> ToTable()

Consume entire stream as a Table.

struct Result

Opaque result returned after executing an action.

Public Functions

arrow::Result<std::string> SerializeToString() const

Serialize this message to its wire-format representation.

Public Static Functions

static arrow::Result<Result> Deserialize(std::string_view serialized)

Deserialize this message from its wire-format representation.

class ResultStream

An iterator to Result instances returned by DoAction.

Subclassed by arrow::flight::SimpleResultStream

Public Functions

virtual arrow::Result<std::unique_ptr<Result>> Next() = 0

Retrieve the next Result from the iterator.

Returns:

Arrow result with a single Result. Set to nullptr if there are none left.

Status Drain()

Read and drop the remaining messages to get the error (if any) from a server.

Returns:

Status OK if this is no error from a server, any other status if a server returns an error.

struct Ticket

Data structure providing an opaque identifier or credential to use when requesting a data stream with the DoGet RPC.

Public Functions

arrow::Result<std::string> SerializeToString() const

Get the wire-format representation of this type.

Useful when interoperating with non-Flight systems (e.g. REST services) that may want to return Flight types.

Public Static Functions

static arrow::Result<Ticket> Deserialize(std::string_view serialized)

Parse the wire-format representation of this type.

Useful when interoperating with non-Flight systems (e.g. REST services) that may want to return Flight types.

Clients

class FlightClient

Client class for Arrow Flight RPC services.

API experimental for now

Public Functions

Status Authenticate(const FlightCallOptions &options, std::unique_ptr<ClientAuthHandler> auth_handler)

Authenticate to the server using the given handler.

Parameters:
  • options[in] Per-RPC options

  • auth_handler[in] The authentication mechanism to use

Returns:

Status OK if the client authenticated successfully

arrow::Result<std::pair<std::string, std::string>> AuthenticateBasicToken(const FlightCallOptions &options, const std::string &username, const std::string &password)

Authenticate to the server using basic HTTP style authentication.

Parameters:
  • options[in] Per-RPC options

  • username[in] Username to use

  • password[in] Password to use

Returns:

Arrow result with bearer token and status OK if client authenticated successfully

arrow::Result<std::unique_ptr<ResultStream>> DoAction(const FlightCallOptions &options, const Action &action)

Perform the indicated action, returning an iterator to the stream of results, if any.

Parameters:
  • options[in] Per-RPC options

  • action[in] the action to be performed

Returns:

Arrow result with an iterator object for reading the returned results

arrow::Result<CancelFlightInfoResult> CancelFlightInfo(const FlightCallOptions &options, const CancelFlightInfoRequest &request)

Perform the CancelFlightInfo action, returning a CancelFlightInfoResult.

Parameters:
  • options[in] Per-RPC options

  • request[in] The CancelFlightInfoRequest

Returns:

Arrow result with a CancelFlightInfoResult

arrow::Result<FlightEndpoint> RenewFlightEndpoint(const FlightCallOptions &options, const RenewFlightEndpointRequest &request)

Perform the RenewFlightEndpoint action, returning a renewed FlightEndpoint.

Parameters:
  • options[in] Per-RPC options

  • request[in] The RenewFlightEndpointRequest

Returns:

Arrow result with a renewed FlightEndpoint

arrow::Result<std::vector<ActionType>> ListActions(const FlightCallOptions &options)

Retrieve a list of available Action types.

Parameters:

options[in] Per-RPC options

Returns:

Arrow result with the available actions

arrow::Result<std::unique_ptr<FlightInfo>> GetFlightInfo(const FlightCallOptions &options, const FlightDescriptor &descriptor)

Request access plan for a single flight, which may be an existing dataset or a command to be executed.

Parameters:
  • options[in] Per-RPC options

  • descriptor[in] the dataset request, whether a named dataset or command

Returns:

Arrow result with the FlightInfo describing where to access the dataset

arrow::Result<std::unique_ptr<SchemaResult>> GetSchema(const FlightCallOptions &options, const FlightDescriptor &descriptor)

Request schema for a single flight, which may be an existing dataset or a command to be executed.

Parameters:
  • options[in] Per-RPC options

  • descriptor[in] the dataset request, whether a named dataset or command

Returns:

Arrow result with the SchemaResult describing the dataset schema

arrow::Result<std::unique_ptr<FlightListing>> ListFlights()

List all available flights known to the server.

Returns:

Arrow result with an iterator that returns a FlightInfo for each flight

arrow::Result<std::unique_ptr<FlightListing>> ListFlights(const FlightCallOptions &options, const Criteria &criteria)

List available flights given indicated filter criteria.

Parameters:
  • options[in] Per-RPC options

  • criteria[in] the filter criteria (opaque)

Returns:

Arrow result with an iterator that returns a FlightInfo for each flight

arrow::Result<std::unique_ptr<FlightStreamReader>> DoGet(const FlightCallOptions &options, const Ticket &ticket)

Given a flight ticket and schema, request to be sent the stream.

Returns record batch stream reader

Parameters:
  • options[in] Per-RPC options

  • ticket[in] The flight ticket to use

Returns:

Arrow result with the returned RecordBatchReader

arrow::Result<DoPutResult> DoPut(const FlightCallOptions &options, const FlightDescriptor &descriptor, const std::shared_ptr<Schema> &schema)

Upload data to a Flight described by the given descriptor.

The caller must call Close() on the returned stream once they are done writing.

The reader and writer are linked; closing the writer will also close the reader. Use DoneWriting to only close the write side of the channel.

Parameters:
  • options[in] Per-RPC options

  • descriptor[in] the descriptor of the stream

  • schema[in] the schema for the data to upload

Returns:

Arrow result with a DoPutResult struct holding a reader and a writer

Status Close()

Explicitly shut down and clean up the client.

For backwards compatibility, this will be implicitly called by the destructor if not already called, but this gives the application no chance to handle errors, so it is recommended to explicitly close the client.

Since

8.0.0

Public Static Functions

static arrow::Result<std::unique_ptr<FlightClient>> Connect(const Location &location)

Connect to an unauthenticated flight service.

Parameters:

location[in] the URI

Returns:

Arrow result with the created FlightClient, OK status may not indicate that the connection was successful

static arrow::Result<std::unique_ptr<FlightClient>> Connect(const Location &location, const FlightClientOptions &options)

Connect to an unauthenticated flight service.

Parameters:
  • location[in] the URI

  • options[in] Other options for setting up the client

Returns:

Arrow result with the created FlightClient, OK status may not indicate that the connection was successful

struct DoExchangeResult
struct DoPutResult

DoPut return value.

Public Members

std::unique_ptr<FlightStreamWriter> writer

a writer to write record batches to

std::unique_ptr<FlightMetadataReader> reader

a reader for application metadata from the server

struct FlightClientOptions

Public Members

std::string tls_root_certs

Root certificates to use for validating server certificates.

std::string override_hostname

Override the hostname checked by TLS. Use with caution.

std::string cert_chain

The client certificate to use if using Mutual TLS.

std::string private_key

The private key associated with the client certificate for Mutual TLS.

std::vector<std::shared_ptr<ClientMiddlewareFactory>> middleware

A list of client middleware to apply.

int64_t write_size_limit_bytes = 0

A soft limit on the number of bytes to write in a single batch when sending Arrow data to a server.

Used to help limit server memory consumption. Only enabled if positive. When enabled, FlightStreamWriter.Write* may yield a IOError with error detail FlightWriteSizeStatusDetail.

std::vector<std::pair<std::string, std::variant<int, std::string>>> generic_options

Generic connection options, passed to the underlying transport; interpretation is implementation-dependent.

bool disable_server_verification = false

Use TLS without validating the server certificate. Use with caution.

Public Static Functions

static FlightClientOptions Defaults()

Get default options.

class FlightCallOptions

Hints to the underlying RPC layer for Arrow Flight calls.

Public Functions

FlightCallOptions()

Create a default set of call options.

Public Members

TimeoutDuration timeout

An optional timeout for this call.

Negative durations mean an implementation-defined default behavior will be used instead. This is the default value.

ipc::IpcReadOptions read_options

IPC reader options, if applicable for the call.

ipc::IpcWriteOptions write_options

IPC writer options, if applicable for the call.

std::vector<std::pair<std::string, std::string>> headers

Headers for client to add to context.

StopToken stop_token

A token to enable interactive user cancellation of long-running requests.

std::shared_ptr<MemoryManager> memory_manager

An optional memory manager to control where to allocate incoming data.

class ClientAuthHandler

An authentication implementation for a Flight service.

Authentication includes both an initial negotiation and a per-call token validation. Implementations may choose to use either or both mechanisms.

Public Functions

virtual Status Authenticate(ClientAuthSender *outgoing, ClientAuthReader *incoming) = 0

Authenticate the client on initial connection.

The client can send messages to/read responses from the server at any time.

Returns:

Status OK if authenticated successfully

virtual Status GetToken(std::string *token) = 0

Get a per-call token.

Parameters:

token[out] The token to send to the server.

class ClientMiddleware

Client-side middleware for a call, instantiated per RPC.

Middleware should be fast and must be infallible: there is no way to reject the call or report errors from the middleware instance.

Public Functions

virtual void SendingHeaders(AddCallHeaders *outgoing_headers) = 0

A callback before headers are sent.

Extra headers can be added, but existing ones cannot be read.

virtual void ReceivedHeaders(const CallHeaders &incoming_headers) = 0

A callback when headers are received from the server.

virtual void CallCompleted(const Status &status) = 0

A callback after the call has completed.

class ClientMiddlewareFactory

A factory for new middleware instances.

If added to a client, this will be called for each RPC (including Handshake) to give the opportunity to intercept the call.

It is guaranteed that all client middleware methods are called from the same thread that calls the RPC method implementation.

Public Functions

virtual void StartCall(const CallInfo &info, std::unique_ptr<ClientMiddleware> *middleware) = 0

A callback for the start of a new call.

Parameters:
  • info – Information about the call.

  • middleware[out] The middleware instance for this call. If unset, will not add middleware to this call instance from this factory.

typedef std::chrono::duration<double, std::chrono::seconds::period> arrow::flight::TimeoutDuration

A duration type for Flight call timeouts.

class FlightStreamReader : public arrow::flight::MetadataRecordBatchReader

A RecordBatchReader exposing Flight metadata and cancel operations.

Public Functions

virtual void Cancel() = 0

Try to cancel the call.

virtual arrow::Result<std::vector<std::shared_ptr<RecordBatch>>> ToRecordBatches(const StopToken &stop_token) = 0

Consume entire stream as a vector of record batches.

arrow::Result<std::shared_ptr<Table>> ToTable(const StopToken &stop_token)

Consume entire stream as a Table.

arrow::Result<std::vector<std::shared_ptr<RecordBatch>>> ToRecordBatches()

Consume entire stream as a vector of record batches.

arrow::Result<std::shared_ptr<Table>> ToTable()

Consume entire stream as a Table.

class FlightStreamWriter : public arrow::flight::MetadataRecordBatchWriter

A RecordBatchWriter that also allows sending application-defined metadata via the Flight protocol.

Public Functions

virtual Status DoneWriting() = 0

Indicate that the application is done writing to this stream.

The application may not write to this stream after calling this. This differs from closing the stream because this writer may represent only one half of a readable and writable stream.

Servers

class FlightServerBase

Skeleton RPC server implementation which can be used to create custom servers by implementing its abstract methods.

Subclassed by arrow::flight::sql::FlightSqlServerBase

Public Functions

Status Init(const FlightServerOptions &options)

Initialize a Flight server listening at the given location.

This method must be called before any other method.

Parameters:

options[in] The configuration for this server.

int port() const

Get the port that the Flight server is listening on.

This method must only be called after Init(). Will return a non-positive value if no port exists (e.g. when listening on a domain socket).

Location location() const

Get the address that the Flight server is listening on.

This method must only be called after Init().

Status SetShutdownOnSignals(const std::vector<int> sigs)

Set the server to stop when receiving any of the given signal numbers.

This method must be called before Serve().

Status Serve()

Start serving.

This method blocks until either Shutdown() is called or one of the signals registered in SetShutdownOnSignals() is received.

int GotSignal() const

Query whether Serve() was interrupted by a signal.

This method must be called after Serve() has returned.

Returns:

int the signal number that interrupted Serve(), if any, otherwise 0

Status Shutdown(const std::chrono::system_clock::time_point *deadline = NULLPTR)

Shut down the server.

Can be called from signal handler or another thread while Serve() blocks. Optionally a deadline can be set. Once the the deadline expires server will wait until remaining running calls complete.

Status Wait()

Block until server is terminated with Shutdown.

virtual Status ListFlights(const ServerCallContext &context, const Criteria *criteria, std::unique_ptr<FlightListing> *listings)

Retrieve a list of available fields given an optional opaque criteria.

Parameters:
  • context[in] The call context.

  • criteria[in] may be null

  • listings[out] the returned listings iterator

Returns:

Status

virtual Status GetFlightInfo(const ServerCallContext &context, const FlightDescriptor &request, std::unique_ptr<FlightInfo> *info)

Retrieve the schema and an access plan for the indicated descriptor.

Parameters:
  • context[in] The call context.

  • request[in] may be null

  • info[out] the returned flight info provider

Returns:

Status

virtual Status GetSchema(const ServerCallContext &context, const FlightDescriptor &request, std::unique_ptr<SchemaResult> *schema)

Retrieve the schema for the indicated descriptor.

Parameters:
  • context[in] The call context.

  • request[in] may be null

  • schema[out] the returned flight schema provider

Returns:

Status

virtual Status DoGet(const ServerCallContext &context, const Ticket &request, std::unique_ptr<FlightDataStream> *stream)

Get a stream of IPC payloads to put on the wire.

Parameters:
  • context[in] The call context.

  • request[in] an opaque ticket

  • stream[out] the returned stream provider

Returns:

Status

virtual Status DoPut(const ServerCallContext &context, std::unique_ptr<FlightMessageReader> reader, std::unique_ptr<FlightMetadataWriter> writer)

Process a stream of IPC payloads sent from a client.

Parameters:
  • context[in] The call context.

  • reader[in] a sequence of uploaded record batches

  • writer[in] send metadata back to the client

Returns:

Status

virtual Status DoExchange(const ServerCallContext &context, std::unique_ptr<FlightMessageReader> reader, std::unique_ptr<FlightMessageWriter> writer)

Process a bidirectional stream of IPC payloads.

Parameters:
  • context[in] The call context.

  • reader[in] a sequence of uploaded record batches

  • writer[in] send data back to the client

Returns:

Status

virtual Status DoAction(const ServerCallContext &context, const Action &action, std::unique_ptr<ResultStream> *result)

Execute an action, return stream of zero or more results.

Parameters:
  • context[in] The call context.

  • action[in] the action to execute, with type and body

  • result[out] the result iterator

Returns:

Status

virtual Status ListActions(const ServerCallContext &context, std::vector<ActionType> *actions)

Retrieve the list of available actions.

Parameters:
  • context[in] The call context.

  • actions[out] a vector of available action types

Returns:

Status

class FlightServerOptions

Public Members

Location location

The host & port (or domain socket path) to listen on.

Use port 0 to bind to an available port.

std::shared_ptr<ServerAuthHandler> auth_handler

The authentication handler to use.

std::vector<CertKeyPair> tls_certificates

A list of TLS certificate+key pairs to use.

bool verify_client

Enable mTLS and require that the client present a certificate.

std::string root_certificates

If using mTLS, the PEM-encoded root certificate to use.

std::vector<std::pair<std::string, std::shared_ptr<ServerMiddlewareFactory>>> middleware

A list of server middleware to apply, along with a key to identify them by.

Middleware are always applied in the order provided. Duplicate keys are an error.

std::shared_ptr<MemoryManager> memory_manager

An optional memory manager to control where to allocate incoming data.

std::function<void(void*)> builder_hook

A Flight implementation-specific callback to customize transport-specific options.

Not guaranteed to be called. The type of the parameter is specific to the Flight implementation. Users should take care to link to the same transport implementation as Flight to avoid runtime problems. See “Using Arrow C++ in your own project” in the documentation for more details.

struct CertKeyPair

A TLS certificate plus key.

Public Members

std::string pem_cert

The certificate in PEM format.

std::string pem_key

The key in PEM format.

class FlightDataStream

Interface that produces a sequence of IPC payloads to be sent in FlightData protobuf messages.

Subclassed by arrow::flight::RecordBatchStream

Public Functions

virtual arrow::Result<FlightPayload> GetSchemaPayload() = 0

Compute FlightPayload containing serialized RecordBatch schema.

class FlightMessageReader : public arrow::flight::MetadataRecordBatchReader

A reader for IPC payloads uploaded by a client.

Also allows reading application-defined metadata via the Flight protocol.

Public Functions

virtual const FlightDescriptor &descriptor() const = 0

Get the descriptor for this upload.

class FlightMetadataWriter

A writer for application-specific metadata sent back to the client during an upload.

Public Functions

virtual Status WriteMetadata(const Buffer &app_metadata) = 0

Send a message to the client.

class RecordBatchStream : public arrow::flight::FlightDataStream

A basic implementation of FlightDataStream that will provide a sequence of FlightData messages to be written to a stream.

Public Functions

explicit RecordBatchStream(const std::shared_ptr<RecordBatchReader> &reader, const ipc::IpcWriteOptions &options = ipc::IpcWriteOptions::Defaults())
Parameters:
  • reader[in] produces a sequence of record batches

  • options[in] IPC options for writing

virtual arrow::Result<FlightPayload> GetSchemaPayload() override

Compute FlightPayload containing serialized RecordBatch schema.

arrow::Result<FlightPayload> GetSchemaPayload() = 0

Compute FlightPayload containing serialized RecordBatch schema.

class ServerAuthHandler

An authentication implementation for a Flight service.

Authentication includes both an initial negotiation and a per-call token validation. Implementations may choose to use either or both mechanisms. An implementation may need to track some state, e.g. a mapping of client tokens to authenticated identities.

Subclassed by arrow::flight::NoOpAuthHandler

Public Functions

inline virtual Status Authenticate(const ServerCallContext &context, ServerAuthSender *outgoing, ServerAuthReader *incoming)

Authenticate the client on initial connection.

The server can send and read responses from the client at any time.

Parameters:
  • context[in] The call context.

  • outgoing[in] The writer for messages to the client.

  • incoming[in] The reader for messages from the client.

Returns:

Status OK if this authentication is succeeded.

inline virtual Status Authenticate(ServerAuthSender *outgoing, ServerAuthReader *incoming)

Authenticate the client on initial connection.

The server can send and read responses from the client at any time.

Deprecated:

Deprecated in 13.0.0. Implement the Authentication() with ServerCallContext version instead.

Parameters:
  • outgoing[in] The writer for messages to the client.

  • incoming[in] The reader for messages from the client.

Returns:

Status OK if this authentication is succeeded.

inline virtual Status IsValid(const ServerCallContext &context, const std::string &token, std::string *peer_identity)

Validate a per-call client token.

Parameters:
  • context[in] The call context.

  • token[in] The client token. May be the empty string if the client does not provide a token.

  • peer_identity[out] The identity of the peer, if this authentication method supports it.

Returns:

Status OK if the token is valid, any other status if validation failed

inline virtual Status IsValid(const std::string &token, std::string *peer_identity)

Validate a per-call client token.

Deprecated:

Deprecated in 13.0.0. Implement the IsValid() with ServerCallContext version instead.

Parameters:
  • token[in] The client token. May be the empty string if the client does not provide a token.

  • peer_identity[out] The identity of the peer, if this authentication method supports it.

Returns:

Status OK if the token is valid, any other status if validation failed

class ServerCallContext

Call state/contextual data.

Public Functions

virtual const std::string &peer_identity() const = 0

The name of the authenticated peer (may be the empty string)

virtual const std::string &peer() const = 0

The peer address (not validated)

virtual ServerMiddleware *GetMiddleware(const std::string &key) const = 0

Look up a middleware by key.

Do not maintain a reference to the object beyond the request body.

Returns:

The middleware, or nullptr if not found.

virtual bool is_cancelled() const = 0

Check if the current RPC has been cancelled (by the client, by a network error, etc.).

virtual const CallHeaders &incoming_headers() const = 0

The headers sent by the client for this call.

class ServerMiddleware

Server-side middleware for a call, instantiated per RPC.

Middleware should be fast and must be infallible: there is no way to reject the call or report errors from the middleware instance.

Subclassed by arrow::flight::TracingServerMiddleware

Public Functions

virtual std::string name() const = 0

Unique name of middleware, used as alternative to RTTI.

Returns:

the string name of the middleware

virtual void SendingHeaders(AddCallHeaders *outgoing_headers) = 0

A callback before headers are sent.

Extra headers can be added, but existing ones cannot be read.

virtual void CallCompleted(const Status &status) = 0

A callback after the call has completed.

class ServerMiddlewareFactory

A factory for new middleware instances.

If added to a server, this will be called for each RPC (including Handshake) to give the opportunity to intercept the call.

It is guaranteed that all server middleware methods are called from the same thread that calls the RPC method implementation.

Public Functions

virtual Status StartCall(const CallInfo &info, const ServerCallContext &context, std::shared_ptr<ServerMiddleware> *middleware)

A callback for the start of a new call.

Return a non-OK status to reject the call with the given status.

Parameters:
  • info[in] Information about the call.

  • context[in] The call context.

  • middleware[out] The middleware instance for this call. If null, no middleware will be added to this call instance from this factory.

Returns:

Status A non-OK status will reject the call with the given status. Middleware previously in the chain will have their CallCompleted callback called. Other middleware factories will not be called.

inline virtual Status StartCall(const CallInfo &info, const CallHeaders &incoming_headers, std::shared_ptr<ServerMiddleware> *middleware)

A callback for the start of a new call.

Return a non-OK status to reject the call with the given status.

Deprecated:

Deprecated in 13.0.0. Implement the StartCall() with ServerCallContext version instead.

Parameters:
  • info – Information about the call.

  • incoming_headers – Headers sent by the client for this call. Do not retain a reference to this object.

  • middleware[out] The middleware instance for this call. If null, no middleware will be added to this call instance from this factory.

Returns:

Status A non-OK status will reject the call with the given status. Middleware previously in the chain will have their CallCompleted callback called. Other middleware factories will not be called.

class SimpleFlightListing : public arrow::flight::FlightListing

A FlightListing implementation based on a vector of FlightInfo objects.

This can be iterated once, then it is consumed.

Public Functions

virtual arrow::Result<std::unique_ptr<FlightInfo>> Next() override

Retrieve the next FlightInfo from the iterator.

Returns:

Arrow result with a single FlightInfo. Set to nullptr if there are none left.

class SimpleResultStream : public arrow::flight::ResultStream

A ResultStream implementation based on a vector of Result objects.

This can be iterated once, then it is consumed.

Public Functions

virtual arrow::Result<std::unique_ptr<Result>> Next() override

Retrieve the next Result from the iterator.

Returns:

Arrow result with a single Result. Set to nullptr if there are none left.

Error Handling

Error handling uses the normal arrow::Status class, combined with a custom arrow::StatusDetail object for Flight-specific error codes.

enum class arrow::flight::FlightStatusCode : int8_t

A Flight-specific status code.

Values:

enumerator Internal

An implementation error has occurred.

enumerator TimedOut

A request timed out.

enumerator Cancelled

A request was cancelled.

enumerator Unauthenticated

We are not authenticated to the remote service.

enumerator Unauthorized

We do not have permission to make this request.

enumerator Unavailable

The remote service cannot handle this request at the moment.

enumerator Failed

A request failed for some other reason.

class FlightStatusDetail : public arrow::StatusDetail

Flight-specific error information in a Status.

Public Functions

virtual const char *type_id() const override

Return a unique id for the type of the StatusDetail (effectively a poor man’s substitute for RTTI).

virtual std::string ToString() const override

Produce a human-readable description of this status.

FlightStatusCode code() const

Get the Flight status code.

std::string extra_info() const

Get the extra error info.

std::string CodeAsString() const

Get the human-readable name of the status code.

void set_extra_info(std::string extra_info)

Set the extra error info.

Public Static Functions

static std::shared_ptr<FlightStatusDetail> UnwrapStatus(const arrow::Status &status)

Try to extract a FlightStatusDetail from any Arrow status.

Returns:

a FlightStatusDetail if it could be unwrapped, nullptr otherwise

Status arrow::flight::MakeFlightError(FlightStatusCode code, std::string message, std::string extra_info = {})

Make an appropriate Arrow status for the given Flight-specific status.

Parameters:
  • code – The Flight status code.

  • message – The message for the error.

  • extra_info – Optional extra binary info for the error (eg protobuf)

Implementing Custom Transports

Internal (but not private) interface for implementing alternate network transports in Flight.

To implement a transport, implement ServerTransport and ClientTransport, and register the desired URI schemes with TransportRegistry. Flight takes care of most of the per-RPC details; transports only handle connections and providing a I/O stream implementation (TransportDataStream).

On the server side:

  1. Applications subclass FlightServerBase and override RPC handlers.

  2. FlightServerBase::Init will look up and create a ServerTransport based on the scheme of the Location given to it.

  3. The ServerTransport will start the actual server. (For instance, for gRPC, it creates a gRPC server and registers a gRPC service.) That server will handle connections.

  4. The transport should forward incoming calls to the server to the RPC handlers defined on ServerTransport, which implements the actual RPC handler using the interfaces here. Any I/O the RPC handler needs to do is managed by transport-specific implementations of TransportDataStream.

  5. ServerTransport calls FlightServerBase for the actual application logic.

On the client side:

  1. Applications create a FlightClient with a Location.

  2. FlightClient will look up and create a ClientTransport based on the scheme of the Location given to it.

  3. When calling a method on FlightClient, FlightClient will delegate to the ClientTransport. There is some indirection, e.g. for DoGet, FlightClient only requests that the ClientTransport start the call and provide it with an I/O stream. The “Flight implementation” itself still lives in FlightClient.

Warning

EXPERIMENTAL. Subject to change.