User Guide

• High-Level Overview
  • The physical layer
  • The one-dimensional layer
  • The two-dimensional layer
  • The compute layer
  • The IO layer
  • The Inter-Process Communication (IPC) layer
  • The file formats layer
  • The devices layer
  • The filesystem layer
• Conventions
  • Language version
  • Namespacing
  • Safe pointers
  • Immutability
  • Error reporting
• Using Arrow C++ in your own project
  • CMake
    • Basic usage
    • Available variables and targets
  • pkg-config
    • Basic usage
    • Available packages
  • A Note on Linking
  • Runtime Dependencies
• Memory Management
  • Buffers
    • Accessing Buffer Memory
    • Slicing
    • Allocating a Buffer
    • Building a Buffer
  • Memory Pools
    • Default Memory Pool
    • Overriding the Default Memory Pool
    • STL Integration
  • Devices
    • Device-Agnostic Programming
  • Memory Profiling
• Arrays
  • Building an array
    • Available strategies
    • Using ArrayBuilder and its subclasses
    • Performance
  • Size Limitations and Recommendations
  • Chunked Arrays
  • Slicing
• Data Types
  • Creating data types
  • Type Traits
  • Visitor Pattern
• Tabular Data
  • Fields
  • Schemas
  • Tables
  • Record Batches
• Compute Functions
  • The generic Compute API
    • Functions and function registry
    • Input shapes
    • Invoking functions
  • Implicit casts
    • Common numeric type
  • Available functions
    • Type categories
    • Aggregations
      • Grouped Aggregations (“group by”)
    • Element-wise (“scalar”) functions
      • Arithmetic functions
      • Bit-wise functions
      • Rounding functions
      • Logarithmic functions
      • Trigonometric functions
      • Comparisons
      • Logical functions
      • String predicates
      • String transforms
      • String padding
      • String trimming
      • String splitting
      • String component extraction
      • String joining
      • String Slicing
      • Containment tests
      • Categorizations
      • Selecting / multiplexing
      • Structural transforms
      • Conversions
      • Temporal component extraction
      • Temporal difference
      • Timezone handling
      • Random number generation
    • Array-wise (“vector”) functions
      • Associative transforms
      • Selections
      • Containment tests
      • Sorts and partitions
      • Structural transforms
• Streaming execution engine
  • Motivation
  • Overview
  • Constructing ExecPlan objects
  • Constructing ExecNode using Options
    • source
    • table_source
    • filter
    • project
    • aggregate
    • sink
    • consuming_sink
    • order_by_sink
    • select_k_sink
    • table_sink
    • scan
    • write
    • union
    • hash_join
  • Summary
• Input / output and filesystems
  • Reading binary data
  • Writing binary data
  • Filesystems
• Reading and writing the Arrow IPC format
  • Reading IPC streams and files
    • Synchronous reading
    • Event-driven reading
  • Writing IPC streams and files
  • Configuring
• Reading and Writing ORC files
  • Supported ORC features
    • Data types
    • Compression
  • Reading ORC Files
    • ORCFileReader
  • Writing ORC Files
    • ORCFileWriter
• Reading and writing Parquet files
  • Supported Parquet features
    • Page types
    • Compression
    • Encodings
    • Types
      • Physical types
      • Logical types
      • Converted types
      • Special cases
      • Roundtripping Arrow types
        • Serialization details
      • Limitations
    • Encryption
  • Reading Parquet files
    • FileReader
    • StreamReader
  • Writing Parquet files
    • WriteTable
    • StreamWriter
• Reading and Writing CSV files
  • Basic usage
  • Column names
  • Column selection
    • Interaction with column names
  • Data types
    • Dictionary inference
    • Timestamp inference/parsing
    • Nulls
    • Character encoding
  • Write Options
  • Performance
• Reading JSON files
  • Basic usage
  • Data types
• Tabular Datasets
  • Reading Datasets
    • Dataset discovery
    • Reading different file formats
    • Customizing file formats
  • Filtering data
  • Projecting columns
  • Reading and writing partitioned data
    • Different partitioning schemes
    • Partitioning performance considerations
  • Reading from other data sources
    • Reading in-memory data
    • Reading from cloud storage
  • A note on transactions & ACID guarantees
  • Full Example
• Arrow Flight RPC
  • Writing a Flight Service
  • Using the Flight Client
  • Cancellation and Timeouts
  • Enabling TLS
  • Enabling Authentication
  • Custom Middleware
  • Alternative Transports
    • UCX Transport
• Debugging code using Arrow
  • GDB extension for Arrow C++
    • Manual loading
    • Automatic loading
    • Supported classes
• Thread Management
  • Thread Pools
    • CPU vs. I/O
    • Serial Execution
    • Jemalloc Background Threads
  • Asynchronous Utilities
    • Future
• Environment Variables

previous

C++ Implementation

next

High-Level Overview