Implementation Status¶
The following tables summarize the features available in the various official Arrow libraries. Unless otherwise stated, the Python, R, Ruby and C/GLib libraries follow the C++ Arrow library.
Data Types¶
Data type (primitive) |
C++ |
Java |
Go |
JavaScript |
C# |
Rust |
Julia |
|---|---|---|---|---|---|---|---|
Null |
✓ |
✓ |
✓ |
✓ |
|||
Boolean |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Int8/16/32/64 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
UInt8/16/32/64 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Float16 |
✓ |
✓ |
|||||
Float32/64 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Decimal128 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
Decimal256 |
✓ |
✓ |
✓ |
✓ |
|||
Date32/64 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Time32/64 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
Timestamp |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Duration |
✓ |
✓ |
✓ |
✓ |
✓ |
||
Interval |
✓ |
✓ |
✓ |
✓ |
✓ |
||
Fixed Size Binary |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
Binary |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Large Binary |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
Utf8 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Large Utf8 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Data type (nested) |
C++ |
Java |
Go |
JavaScript |
C# |
Rust |
Julia |
|---|---|---|---|---|---|---|---|
Fixed Size List |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
List |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Large List |
✓ |
✓ |
✓ |
✓ |
|||
Struct |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Map |
✓ |
✓ |
✓ |
✓ |
✓ |
||
Dense Union |
✓ |
✓ |
✓ |
||||
Sparse Union |
✓ |
✓ |
✓ |
Data type (special) |
C++ |
Java |
Go |
JavaScript |
C# |
Rust |
Julia |
|---|---|---|---|---|---|---|---|
Dictionary |
✓ |
✓ (1) |
✓ (1) |
✓ (1) |
✓ |
||
Extension |
✓ |
✓ |
✓ |
✓ |
Notes:
(1) Nested dictionaries not supported
See also
The Arrow Columnar Format specification.
IPC Format¶
IPC Feature |
C++ |
Java |
Go |
JavaScript |
C# |
Rust |
Julia |
|---|---|---|---|---|---|---|---|
Arrow stream format |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Arrow file format |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Record batches |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Dictionaries |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
Replacement dictionaries |
✓ |
✓ |
✓ |
||||
Delta dictionaries |
✓ (1) |
✓ |
|||||
Tensors |
✓ |
||||||
Sparse tensors |
✓ |
||||||
Buffer compression |
✓ |
✓ (3) |
✓ |
✓ |
|||
Endianness conversion |
✓ (2) |
||||||
Custom schema metadata |
✓ |
✓ |
✓ |
✓ |
✓ |
Notes:
(1) Delta dictionaries not supported on nested dictionaries
(2) Data with non-native endianness can be byte-swapped automatically when reading.
(3) LZ4 Codec currently is quite inefficient. ARROW-11901 tracks improving performance.
See also
The Serialization and Interprocess Communication (IPC) specification.
Flight RPC¶
Flight RPC Feature |
C++ |
Java |
Go |
JavaScript |
C# |
Rust |
Julia |
|---|---|---|---|---|---|---|---|
gRPC transport |
✓ |
✓ |
✓ |
✓ (1) |
|||
gRPC + TLS transport |
✓ |
✓ |
✓ |
✓ |
|||
RPC error codes |
✓ |
✓ |
✓ |
✓ |
|||
Authentication handlers |
✓ |
✓ |
✓ |
✓ (2) |
|||
Custom client middleware |
✓ |
✓ |
✓ |
||||
Custom server middleware |
✓ |
✓ |
✓ |
Notes:
(1) No support for handshake or DoExchange.
(2) Support using AspNetCore authentication handlers.
See also
The Arrow Flight RPC specification.
C Data Interface¶
Feature |
C++ |
Python |
R |
Rust |
|---|---|---|---|---|
Schema export |
✓ |
✓ |
✓ |
✓ |
Array export |
✓ |
✓ |
✓ |
✓ |
Schema import |
✓ |
✓ |
✓ |
✓ |
Array import |
✓ |
✓ |
✓ |
✓ |
See also
The C Data Interface specification.
C Stream Interface (experimental)¶
Feature |
C++ |
Python |
|---|---|---|
Stream export |
✓ |
✓ |
Stream import |
✓ |
✓ |
See also
The C Stream Interface specification.
Third-Party Data Formats¶
Format |
C++ |
Java |
Go |
JavaScript |
C# |
Rust |
Julia |
|---|---|---|---|---|---|---|---|
Avro |
R |
||||||
CSV |
R |
R/W |
R/W |
R/W |
|||
ORC |
R/W |
R (2) |
|||||
Parquet |
R/W |
R (3) |
R/W (1) |
Notes:
R = Read supported
W = Write supported
(1) Nested read/write not supported.
(2) Through JNI bindings. (Provided by
org.apache.arrow.orc:arrow-orc)(3) Through JNI bindings to Arrow C++ Datasets. (Provided by
org.apache.arrow:arrow-dataset)