pyarrow.concat_tables¶

pyarrow.concat_tables(tables, bool promote=False, MemoryPool memory_pool=None)¶

Concatenate pyarrow.Table objects.

If promote==False, a zero-copy concatenation will be performed. The schemas of all the Tables must be the same (except the metadata), otherwise an exception will be raised. The result Table will share the metadata with the first table.

If promote==True, any null type arrays will be casted to the type of other arrays in the column of the same name. If a table is missing a particular field, null values of the appropriate type will be generated to take the place of the missing field. The new schema will share the metadata with the first table. Each field in the new schema will share the metadata with the first table which has the field defined. Note that type promotions may involve additional allocations on the given memory_pool.

Parameters:

tablesiterable of pyarrow.Table objects: Pyarrow tables to concatenate into a single Table.
promotebool, default False: If True, concatenate tables with null-filling and null type promotion.
memory_poolMemoryPool, default None: For memory allocations, if required, otherwise use default pool.

Examples

>>> import pyarrow as pa
>>> t1 = pa.table([
...     pa.array([2, 4, 5, 100]),
...     pa.array(["Flamingo", "Horse", "Brittle stars", "Centipede"])
...     ], names=['n_legs', 'animals'])
>>> t2 = pa.table([
...     pa.array([2, 4]),
...     pa.array(["Parrot", "Dog"])
...     ], names=['n_legs', 'animals'])
>>> pa.concat_tables([t1,t2])
pyarrow.Table
n_legs: int64
animals: string
----
n_legs: [[2,4,5,100],[2,4]]
animals: [["Flamingo","Horse","Brittle stars","Centipede"],["Parrot","Dog"]]

pyarrow.concat_arrays

pyarrow.record_batch