Implementation Status#

The following tables summarize the features available in the various official Arrow libraries. All libraries currently follow version 1.0.0 of the Arrow format. See Format Versioning and Stability for details about versioning. Unless otherwise stated, the Python, R, Ruby and C/GLib libraries follow the C++ Arrow library.

Data Types#

Data type (primitive)

C++

Java

Go

JavaScript

C#

Rust

Julia

Swift

Null

Boolean

Int8/16/32/64

UInt8/16/32/64

Float16

✓ (1)

✓ (2)

✓ (3)

Float32/64

Decimal128

Decimal256

Date32/64

Time32/64

Timestamp

Duration

Interval

Fixed Size Binary

Binary

Large Binary

Utf8

Large Utf8

Binary View

Large Binary View

Utf8 View

Large Utf8 View

Data type (nested)

C++

Java

Go

JavaScript

C#

Rust

Julia

Swift

Fixed Size List

List

Large List

List View

Large List View

Struct

Map

Dense Union

Sparse Union

Data type (special)

C++

Java

Go

JavaScript

C#

Rust

Julia

Swift

Dictionary

✓ (4)

✓ (3)

Extension

Run-End Encoded

Notes:

  • (1) Casting to/from Float16 in C++ is not supported.

  • (2) Casting to/from Float16 in Java is not supported.

  • (3) Float16 support in C# is only available when targeting .NET 6+.

  • (4) Nested dictionaries not supported

See also

The Arrow Columnar Format specification.

IPC Format#

IPC Feature

C++

Java

Go

JavaScript

C#

Rust

Julia

Swift

Arrow stream format

Arrow file format

Record batches

Dictionaries

Replacement dictionaries

Delta dictionaries

✓ (1)

✓ (1)

Tensors

Sparse tensors

Buffer compression

✓ (3)

Endianness conversion

✓ (2)

✓ (2)

Custom schema metadata

Notes:

  • (1) Delta dictionaries not supported on nested dictionaries

  • (2) Data with non-native endianness can be byte-swapped automatically when reading.

  • (3) LZ4 Codec currently is quite inefficient. ARROW-11901 tracks improving performance.

Flight RPC#

Flight RPC Transport

C++

Java

Go

JavaScript

C#

Rust

Julia

Swift

gRPC transport (grpc:, grpc+tcp:)

gRPC domain socket transport (grpc+unix:)

gRPC + TLS transport (grpc+tls:)

UCX transport (ucx:)

Supported features in the gRPC transport:

Flight RPC Feature

C++

Java

Go

JavaScript

C#

Rust

Julia

Swift

All RPC methods

× (1)

Authentication handlers

✓ (2)

Call timeouts

Call cancellation

Concurrent client calls (3)

Custom middleware

RPC error codes

Supported features in the UCX transport:

Flight RPC Feature

C++

Java

Go

JavaScript

C#

Rust

Julia

Swift

All RPC methods

× (4)

Authentication handlers

Call timeouts

Call cancellation

Concurrent client calls

✓ (5)

Custom middleware

RPC error codes

Notes:

  • (1) No support for Handshake or DoExchange.

  • (2) Support using AspNetCore authentication handlers.

  • (3) Whether a single client can support multiple concurrent calls.

  • (4) Only support for DoExchange, DoGet, DoPut, and GetFlightInfo.

  • (5) Each concurrent call is a separate connection to the server (unlike gRPC where concurrent calls are multiplexed over a single connection). This will generally provide better throughput but consumes more resources both on the server and the client.

See also

The Arrow Flight RPC specification.

Flight SQL#

Note

Flight SQL is still experimental.

The feature support refers to the client/server libraries only; databases which implement the Flight SQL protocol in turn will support/not support individual features.

Feature

C++

Java

Go

JavaScript

C#

Rust

Julia

Swift

BeginSavepoint

BeginTransaction

CancelQuery

ClosePreparedStatement

CreatePreparedStatement

CreatePreparedSubstraitPlan

EndSavepoint

EndTransaction

GetCatalogs

GetCrossReference

GetDbSchemas

GetExportedKeys

GetImportedKeys

GetPrimaryKeys

GetSqlInfo

GetTables

GetTableTypes

GetXdbcTypeInfo

PreparedStatementQuery

PreparedStatementUpdate

StatementSubstraitPlan

StatementQuery

StatementUpdate

See also

The Arrow Flight SQL specification.

C Data Interface#

Feature

C++

Python

R

Rust

Go

Java

C/GLib

Ruby

Julia

C#

Swift

Schema export

Array export

Schema import

Array import

See also

The C Data Interface specification.

C Stream Interface#

Feature

C++

Python

R

Rust

Go

Java

C/GLib

Ruby

Julia

C#

Swift

Stream export

Stream import

See also

The C Stream Interface specification.

Third-Party Data Formats#

Format

C++

Java

Go

JavaScript

C#

Rust

Julia

Swift

Avro

R

CSV

R/W

R (2)

R/W

R/W

R/W

ORC

R/W

R (1)

Parquet

R/W

R (2)

R/W

R/W

Notes:

  • R = Read supported

  • W = Write supported

  • (1) Through JNI bindings. (Provided by org.apache.arrow.orc:arrow-orc)

  • (2) Through JNI bindings to Arrow C++ Datasets. (Provided by org.apache.arrow:arrow-dataset)