Module org.apache.arrow.c
Package org.apache.arrow.c

Class Data

java.lang.Object
org.apache.arrow.c.Data
public final class Data extends Object
Functions for working with the C data interface.

This API is EXPERIMENTAL. Note that currently only 64bit systems are supported.

  • Method Details

    • exportField

      public static void exportField(BufferAllocator allocator, Field field, DictionaryProvider provider, ArrowSchema out)
      Export Java Field using the C data interface format.
      Parameters:
      allocator - Buffer allocator for allocating C data interface fields
      field - Field object to export
      provider - Dictionary provider for dictionary encoded fields (optional)
      out - C struct where to export the field

    • exportSchema

      public static void exportSchema(BufferAllocator allocator, Schema schema, DictionaryProvider provider, ArrowSchema out)
      Export Java Schema using the C data interface format.
      Parameters:
      allocator - Buffer allocator for allocating C data interface fields
      schema - Schema object to export
      provider - Dictionary provider for dictionary encoded fields (optional)
      out - C struct where to export the field

    • exportVector

      public static void exportVector(BufferAllocator allocator, FieldVector vector, DictionaryProvider provider, ArrowArray out)
      Export Java FieldVector using the C data interface format.

      The resulting ArrowArray struct keeps the array data and buffers alive until its release callback is called by the consumer.

      Parameters:
      allocator - Buffer allocator for allocating C data interface fields
      vector - Vector object to export
      provider - Dictionary provider for dictionary encoded vectors (optional)
      out - C struct where to export the array

    • exportVector

      public static void exportVector(BufferAllocator allocator, FieldVector vector, DictionaryProvider provider, ArrowArray out, ArrowSchema outSchema)
      Export Java FieldVector using the C data interface format.

      The resulting ArrowArray struct keeps the array data and buffers alive until its release callback is called by the consumer.

      Parameters:
      allocator - Buffer allocator for allocating C data interface fields
      vector - Vector object to export
      provider - Dictionary provider for dictionary encoded vectors (optional)
      out - C struct where to export the array
      outSchema - C struct where to export the array type (optional)

    • exportTable

      public static void exportTable(BufferAllocator allocator, Table table, ArrowArray out)
      Export the current contents of a Java Table using the C data interface format.

      The table is exported as if it were a struct array. The resulting ArrowArray struct keeps the record batch data and buffers alive until its release callback is called by the consumer.

      Parameters:
      allocator - Buffer allocator for allocating C data interface fields
      table - Table to export
      out - C struct where to export the record batch

    • exportTable

      public static void exportTable(BufferAllocator allocator, Table table, DictionaryProvider provider, ArrowArray out)
      Export the current contents of a Java Table using the C data interface format.

      The table is exported as if it were a struct array. The resulting ArrowArray struct keeps the record batch data and buffers alive until its release callback is called by the consumer.

      Parameters:
      allocator - Buffer allocator for allocating C data interface fields
      table - Table to export
      provider - Dictionary provider for dictionary encoded vectors (optional)
      out - C struct where to export the record batch

    • exportTable

      public static void exportTable(BufferAllocator allocator, Table table, DictionaryProvider provider, ArrowArray out, ArrowSchema outSchema)
      Export the current contents of a Java Table using the C data interface format.

      The table is exported as if it were a struct array. The resulting ArrowArray struct keeps the record batch data and buffers alive until its release callback is called by the consumer.

      Parameters:
      allocator - Buffer allocator for allocating C data interface fields
      table - Table to export
      provider - Dictionary provider for dictionary encoded vectors (optional)
      out - C struct where to export the record batch
      outSchema - C struct where to export the record batch schema (optional)

    • exportVectorSchemaRoot

      public static void exportVectorSchemaRoot(BufferAllocator allocator, VectorSchemaRoot vsr, DictionaryProvider provider, ArrowArray out)
      Export the current contents of a Java VectorSchemaRoot using the C data interface format.

      The vector schema root is exported as if it were a struct array. The resulting ArrowArray struct keeps the record batch data and buffers alive until its release callback is called by the consumer.

      Parameters:
      allocator - Buffer allocator for allocating C data interface fields
      vsr - Vector schema root to export
      provider - Dictionary provider for dictionary encoded vectors (optional)
      out - C struct where to export the record batch

    • exportVectorSchemaRoot

      public static void exportVectorSchemaRoot(BufferAllocator allocator, VectorSchemaRoot vsr, DictionaryProvider provider, ArrowArray out, ArrowSchema outSchema)
      Export the current contents of a Java VectorSchemaRoot using the C data interface format.

      The vector schema root is exported as if it were a struct array. The resulting ArrowArray struct keeps the record batch data and buffers alive until its release callback is called by the consumer.

      Parameters:
      allocator - Buffer allocator for allocating C data interface fields
      vsr - Vector schema root to export
      provider - Dictionary provider for dictionary encoded vectors (optional)
      out - C struct where to export the record batch
      outSchema - C struct where to export the record batch schema (optional)

    • exportArrayStream

      public static void exportArrayStream(BufferAllocator allocator, ArrowReader reader, ArrowArrayStream out)
      Export a reader as an ArrowArrayStream using the C Stream Interface.
      Parameters:
      allocator - Buffer allocator for allocating C data interface fields
      reader - Reader to export
      out - C struct to export the stream

    • importField

      public static Field importField(BufferAllocator allocator, ArrowSchema schema, CDataDictionaryProvider provider)
      Equivalent to calling importField(allocator, schema, provider, true).
      Parameters:
      allocator - Buffer allocator for allocating dictionary vectors
      schema - C data interface struct representing the field [inout]
      provider - A dictionary provider will be initialized with empty dictionary vectors (optional)
      Returns:
      Imported field object
      See Also:

    • importField

      public static Field importField(BufferAllocator allocator, ArrowSchema schema, CDataDictionaryProvider provider, boolean closeImportedStructs)
      Import Java Field from the C data interface.

      The given ArrowSchema struct is released (as per the C data interface specification), even if this function fails.

      Parameters:
      allocator - Buffer allocator for allocating dictionary vectors
      schema - C data interface struct representing the field [inout]
      provider - A dictionary provider will be initialized with empty dictionary vectors (optional)
      closeImportedStructs - if true, the ArrowSchema struct will be closed when this method completes.
      Returns:
      Imported field object

    • importSchema

      public static Schema importSchema(BufferAllocator allocator, ArrowSchema schema, CDataDictionaryProvider provider)
      Equivalent to calling importSchema(allocator, schema, provider, true).
      Parameters:
      allocator - Buffer allocator for allocating dictionary vectors
      schema - C data interface struct representing the field
      provider - A dictionary provider will be initialized with empty dictionary vectors (optional)
      Returns:
      Imported schema object
      See Also:

    • importSchema

      public static Schema importSchema(BufferAllocator allocator, ArrowSchema schema, CDataDictionaryProvider provider, boolean closeImportedStructs)
      Import Java Schema from the C data interface.

      The given ArrowSchema struct is released (as per the C data interface specification), even if this function fails.

      Parameters:
      allocator - Buffer allocator for allocating dictionary vectors
      schema - C data interface struct representing the field
      provider - A dictionary provider will be initialized with empty dictionary vectors (optional)
      closeImportedStructs - if true, the ArrowSchema struct will be closed when this method completes.
      Returns:
      Imported schema object

    • importIntoVector

      public static void importIntoVector(BufferAllocator allocator, ArrowArray array, FieldVector vector, DictionaryProvider provider)
      Equivalent to calling importIntoVector(BufferAllocator, ArrowArray, FieldVector, DictionaryProvider, boolean) importIntoVector(allocator, array, vector, provider, true)}.
      Parameters:
      allocator - Buffer allocator
      array - C data interface struct holding the array data
      vector - Imported vector object [out]
      provider - Dictionary provider to load dictionary vectors to (optional)
      See Also:

    • importIntoVector

      public static void importIntoVector(BufferAllocator allocator, ArrowArray array, FieldVector vector, DictionaryProvider provider, boolean closeImportedStructs)
      Import Java vector from the C data interface.

      On successful completion, the ArrowArray struct will have been moved (as per the C data interface specification) to a private object held alive by the resulting array.

      Parameters:
      allocator - Buffer allocator
      array - C data interface struct holding the array data
      vector - Imported vector object [out]
      provider - Dictionary provider to load dictionary vectors to (optional)
      closeImportedStructs - if true, the ArrowArray struct will be closed when this method completes successfully.

    • importVector

      public static FieldVector importVector(BufferAllocator allocator, ArrowArray array, ArrowSchema schema, CDataDictionaryProvider provider)
      Equivalent to calling importVector(allocator, array, schema, provider, true).
      Parameters:
      allocator - Buffer allocator for allocating the output FieldVector
      array - C data interface struct holding the array data
      schema - C data interface struct holding the array type
      provider - Dictionary provider to load dictionary vectors to (optional)
      Returns:
      Imported vector object
      See Also:

    • importVector

      public static FieldVector importVector(BufferAllocator allocator, ArrowArray array, ArrowSchema schema, CDataDictionaryProvider provider, boolean closeImportedStructs)
      Import Java vector and its type from the C data interface.

      The ArrowArray struct has its contents moved (as per the C data interface specification) to a private object held alive by the resulting vector. The ArrowSchema struct is released, even if this function fails.

      Parameters:
      allocator - Buffer allocator for allocating the output FieldVector
      array - C data interface struct holding the array data
      schema - C data interface struct holding the array type
      provider - Dictionary provider to load dictionary vectors to (optional)
      closeImportedStructs - if true, the ArrowArray struct will be closed when this method completes successfully and the ArrowSchema struct will be always be closed.
      Returns:
      Imported vector object

    • importIntoVectorSchemaRoot

      public static void importIntoVectorSchemaRoot(BufferAllocator allocator, ArrowArray array, VectorSchemaRoot root, DictionaryProvider provider)
      Equivalent to calling importIntoVectorSchemaRoot(allocator, array, root, provider, true).
      Parameters:
      allocator - Buffer allocator
      array - C data interface struct holding the record batch data
      root - vector schema root to load into
      provider - Dictionary provider to load dictionary vectors to (optional)
      See Also:

    • importIntoVectorSchemaRoot

      public static void importIntoVectorSchemaRoot(BufferAllocator allocator, ArrowArray array, VectorSchemaRoot root, DictionaryProvider provider, boolean closeImportedStructs)
      Import record batch from the C data interface into vector schema root.

      The ArrowArray struct has its contents moved (as per the C data interface specification) to a private object held alive by the resulting vector schema root.

      The schema of the vector schema root must match the input array (undefined behavior otherwise).

      Parameters:
      allocator - Buffer allocator
      array - C data interface struct holding the record batch data
      root - vector schema root to load into
      provider - Dictionary provider to load dictionary vectors to (optional)
      closeImportedStructs - if true, the ArrowArray struct will be closed when this method completes successfully

    • importVectorSchemaRoot

      public static VectorSchemaRoot importVectorSchemaRoot(BufferAllocator allocator, ArrowSchema schema, CDataDictionaryProvider provider)
      Equivalent to calling importVectorSchemaRoot(allocator, schema, provider, true).
      Parameters:
      allocator - Buffer allocator for allocating the output VectorSchemaRoot
      schema - C data interface struct holding the record batch schema
      provider - Dictionary provider to load dictionary vectors to (optional)
      Returns:
      Imported vector schema root
      See Also:

    • importVectorSchemaRoot

      public static VectorSchemaRoot importVectorSchemaRoot(BufferAllocator allocator, ArrowSchema schema, CDataDictionaryProvider provider, boolean closeImportedStructs)
      Import Java vector schema root from a C data interface Schema.

      The type represented by the ArrowSchema struct must be a struct type array.

      The ArrowSchema struct is released, even if this function fails.

      Parameters:
      allocator - Buffer allocator for allocating the output VectorSchemaRoot
      schema - C data interface struct holding the record batch schema
      provider - Dictionary provider to load dictionary vectors to (optional)
      closeImportedStructs - if true, the ArrowSchema struct will be closed when this method completes
      Returns:
      Imported vector schema root

    • importVectorSchemaRoot

      public static VectorSchemaRoot importVectorSchemaRoot(BufferAllocator allocator, ArrowArray array, ArrowSchema schema, CDataDictionaryProvider provider)
      Equivalent to calling importVectorSchemaRoot(allocator, array, schema, provider, true).
      Parameters:
      allocator - Buffer allocator for allocating the output VectorSchemaRoot
      array - C data interface struct holding the record batch data (optional)
      schema - C data interface struct holding the record batch schema
      provider - Dictionary provider to load dictionary vectors to (optional)
      Returns:
      Imported vector schema root
      See Also:

    • importVectorSchemaRoot

      public static VectorSchemaRoot importVectorSchemaRoot(BufferAllocator allocator, ArrowArray array, ArrowSchema schema, CDataDictionaryProvider provider, boolean closeImportedStructs)
      Import Java vector schema root from the C data interface.

      The type represented by the ArrowSchema struct must be a struct type array.

      The ArrowArray struct has its contents moved (as per the C data interface specification) to a private object held alive by the resulting record batch. The ArrowSchema struct is released, even if this function fails.

      Prefer importIntoVectorSchemaRoot(org.apache.arrow.memory.BufferAllocator, org.apache.arrow.c.ArrowArray, org.apache.arrow.vector.VectorSchemaRoot, org.apache.arrow.vector.dictionary.DictionaryProvider) for loading array data while reusing the same vector schema root.

      Parameters:
      allocator - Buffer allocator for allocating the output VectorSchemaRoot
      array - C data interface struct holding the record batch data (optional)
      schema - C data interface struct holding the record batch schema
      provider - Dictionary provider to load dictionary vectors to (optional)
      closeImportedStructs - if true, the ArrowArray struct will be closed when this method completes successfully and the ArrowSchema struct will be always be closed.
      Returns:
      Imported vector schema root

    • importArrayStream

      public static ArrowReader importArrayStream(BufferAllocator allocator, ArrowArrayStream stream)
      Equivalent to calling importArrayStream(allocator, stream, true).
      Parameters:
      allocator - Buffer allocator for allocating the output data.
      stream - C stream interface struct to import.
      Returns:
      Imported reader
      See Also:

    • importArrayStream

      public static ArrowReader importArrayStream(BufferAllocator allocator, ArrowArrayStream stream, boolean closeImportedStructs)
      Import an ArrowArrayStream as an ArrowReader.

      On successful completion, the ArrowArrayStream struct will have been moved (as per the C data interface specification) to a private object held alive by the resulting ArrowReader.

      Parameters:
      allocator - Buffer allocator for allocating the output data.
      stream - C stream interface struct to import.
      closeImportedStructs - if true, the ArrowArrayStream struct will be closed when this method completes successfully
      Returns:
      Imported reader