pyarrow.concat_tables#

pyarrow.concat_tables(tables, MemoryPool memory_pool=None, unicode promote_options=u'none', **kwargs)#

Concatenate pyarrow.Table objects.

If promote_options=”none”, a zero-copy concatenation will be performed. The schemas of all the Tables must be the same (except the metadata), otherwise an exception will be raised. The result Table will share the metadata with the first table.

If promote_options=”default”, any null type arrays will be casted to the type of other arrays in the column of the same name. If a table is missing a particular field, null values of the appropriate type will be generated to take the place of the missing field. The new schema will share the metadata with the first table. Each field in the new schema will share the metadata with the first table which has the field defined. Note that type promotions may involve additional allocations on the given memory_pool.

If promote_options=”permissive”, the behavior of default plus types will be promoted to the common denominator that fits all the fields.

Parameters:

tablesiterable of pyarrow.Table objects: Pyarrow tables to concatenate into a single Table.
memory_poolMemoryPool, default None: For memory allocations, if required, otherwise use default pool.
promote_optionsstr, default none: Accepts strings “none”, “default” and “permissive”.
**kwargsdict, optional

Examples

>>> import pyarrow as pa
>>> t1 = pa.table([
...     pa.array([2, 4, 5, 100]),
...     pa.array(["Flamingo", "Horse", "Brittle stars", "Centipede"])
...     ], names=['n_legs', 'animals'])
>>> t2 = pa.table([
...     pa.array([2, 4]),
...     pa.array(["Parrot", "Dog"])
...     ], names=['n_legs', 'animals'])
>>> pa.concat_tables([t1,t2])
pyarrow.Table
n_legs: int64
animals: string
----
n_legs: [[2,4,5,100],[2,4]]
animals: [["Flamingo","Horse","Brittle stars","Centipede"],["Parrot","Dog"]]