pyarrow.concat_tables#
- pyarrow.concat_tables(tables, MemoryPool memory_pool=None, unicode promote_options=u'none', **kwargs)#
Concatenate pyarrow.Table objects.
If promote_options=”none”, a zero-copy concatenation will be performed. The schemas of all the Tables must be the same (except the metadata), otherwise an exception will be raised. The result Table will share the metadata with the first table.
If promote_options=”default”, any null type arrays will be casted to the type of other arrays in the column of the same name. If a table is missing a particular field, null values of the appropriate type will be generated to take the place of the missing field. The new schema will share the metadata with the first table. Each field in the new schema will share the metadata with the first table which has the field defined. Note that type promotions may involve additional allocations on the given
memory_pool
.If promote_options=”permissive”, the behavior of default plus types will be promoted to the common denominator that fits all the fields.
- Parameters:
- tablesiterable of
pyarrow.Table
objects
Pyarrow tables to concatenate into a single Table.
- memory_pool
MemoryPool
, defaultNone
For memory allocations, if required, otherwise use default pool.
- promote_options
str
, defaultnone
Accepts strings “none”, “default” and “permissive”.
- **kwargs
dict
, optional
- tablesiterable of
Examples
>>> import pyarrow as pa >>> t1 = pa.table([ ... pa.array([2, 4, 5, 100]), ... pa.array(["Flamingo", "Horse", "Brittle stars", "Centipede"]) ... ], names=['n_legs', 'animals']) >>> t2 = pa.table([ ... pa.array([2, 4]), ... pa.array(["Parrot", "Dog"]) ... ], names=['n_legs', 'animals']) >>> pa.concat_tables([t1,t2]) pyarrow.Table n_legs: int64 animals: string ---- n_legs: [[2,4,5,100],[2,4]] animals: [["Flamingo","Horse","Brittle stars","Centipede"],["Parrot","Dog"]]