RecordBatch class

A record batch is a collection of equal-length arrays matching a particular Schema. It is a table-like data structure that is semantically a sequence of fields, each a contiguous Arrow Array.

record_batch(..., schema = NULL)

Arguments

...	A data.frame or a named set of Arrays or vectors. If given a mixture of data.frames and vectors, the inputs will be autospliced together (see examples). Alternatively, you can provide a single Arrow IPC InputStream, Message, Buffer, or R raw object containing a Buffer.
schema	a Schema, or NULL (the default) to infer the schema from the data in .... When providing an Arrow IPC buffer, schema is required.

S3 Methods and Usage

Record batches are data-frame-like, and many methods you expect to work on a data.frame are implemented for RecordBatch. This includes [, [[, $, names, dim, nrow, ncol, head, and tail. You can also pull the data from an Arrow record batch into R with as.data.frame(). See the examples.

A caveat about the $ method: because RecordBatch is an R6 object, $ is also used to access the object's methods (see below). Methods take precedence over the table's columns. So, batch$Slice would return the "Slice" method function even if there were a column in the table called "Slice".

R6 Methods

In addition to the more R-friendly S3 methods, a RecordBatch object has the following R6 methods that map onto the underlying C++ methods:

• $Equals(other): Returns TRUE if the other record batch is equal

• $column(i): Extract an Array by integer position from the batch

• $column_name(i): Get a column's name by integer position

• $names(): Get all column names (called by names(batch))

• $GetColumnByName(name): Extract an Array by string name

• $RemoveColumn(i): Drops a column from the batch by integer position

• $selectColumns(indices): Return a new record batch with a selection of columns, expressed as 0-based integers.

• $Slice(offset, length = NULL): Create a zero-copy view starting at the indicated integer offset and going for the given length, or to the end of the table if NULL, the default.

• $Take(i): return an RecordBatch with rows at positions given by integers (R vector or Array Array) i.

• $Filter(i, keep_na = TRUE): return an RecordBatch with rows at positions where logical vector (or Arrow boolean Array) i is TRUE.

• $serialize(): Returns a raw vector suitable for interprocess communication

• $cast(target_schema, safe = TRUE, options = cast_options(safe)): Alter the schema of the record batch.

There are also some active bindings

• $num_columns

• $num_rows

• $schema

• $metadata: Returns the key-value metadata of the Schema as a named list. Modify or replace by assigning in (batch$metadata <- new_metadata). All list elements are coerced to string.

• $columns: Returns a list of Arrays

Examples

# \donttest{
batch <- record_batch(name = rownames(mtcars), mtcars)
dim(batch)
#> [1] 32 12
dim(head(batch))
#> [1]  6 12
names(batch)
#>  [1] "name" "mpg"  "cyl"  "disp" "hp"   "drat" "wt"   "qsec" "vs"   "am"  
#> [11] "gear" "carb"
batch$mpg
#> Array
#> <double>
#> [
#>   21,
#>   21,
#>   22.8,
#>   21.4,
#>   18.7,
#>   18.1,
#>   14.3,
#>   24.4,
#>   22.8,
#>   19.2,
#>   ...
#>   15.2,
#>   13.3,
#>   19.2,
#>   27.3,
#>   26,
#>   30.4,
#>   15.8,
#>   19.7,
#>   15,
#>   21.4
#> ]
batch[["cyl"]]
#> Array
#> <double>
#> [
#>   6,
#>   6,
#>   4,
#>   6,
#>   8,
#>   6,
#>   8,
#>   4,
#>   4,
#>   6,
#>   ...
#>   8,
#>   8,
#>   8,
#>   4,
#>   4,
#>   4,
#>   8,
#>   6,
#>   8,
#>   4
#> ]
as.data.frame(batch[4:8, c("gear", "hp", "wt")])
#> # A tibble: 5 x 3
#>    gear    hp    wt
#>   <dbl> <dbl> <dbl>
#> 1     3   110  3.22
#> 2     3   175  3.44
#> 3     3   105  3.46
#> 4     3   245  3.57
#> 5     4    62  3.19
# }