Options
All
  • Public
  • Public/Protected
  • All
Menu

Class Builder<T, TNull>

An abstract base class for types that construct Arrow Vectors from arbitrary JavaScript values.

A Builder is responsible for writing arbitrary JavaScript values to ArrayBuffers and/or child Builders according to the Arrow specification for each DataType, creating or resizing the underlying ArrayBuffers as necessary.

The Builder for each Arrow DataType handles converting and appending values for a given DataType. The high-level Builder.new() convenience method creates the specific Builder subclass for the supplied DataType.

Once created, Builder instances support both appending values to the end of the Builder, and random-access writes to specific indices (Builder.prototype.append(value) is a convenience method for builder.set(builder.length, value)). Appending or setting values beyond the Builder's current length may cause the builder to grow its underlying buffers or child Builders (if applicable) to accommodate the new values.

After enough values have been written to a Builder, Builder.prototype.flush() will commit the values to the underlying ArrayBuffers (or child Builders). The internal Builder state will be reset, and an instance of Data<T> is returned. Alternatively, Builder.prototype.toVector() will flush the Builder and return an instance of Vector<T> instead.

When there are no more values to write, use Builder.prototype.finish() to finalize the Builder. This does not reset the internal state, so it is necessary to call Builder.prototype.flush() or toVector() one last time if there are still values queued to be flushed.

Note: calling Builder.prototype.finish() is required when using a DictionaryBuilder, because this is when it flushes the values that have been enqueued in its internal dictionary's Builder, and creates the dictionaryVector for the Dictionary DataType.

import { Builder, Utf8 } from 'apache-arrow';

const utf8Builder = Builder.new({
    type: new Utf8(),
    nullValues: [null, 'n/a']
});

utf8Builder
    .append('hello')
    .append('n/a')
    .append('world')
    .append(null);

const utf8Vector = utf8Builder.finish().toVector();

console.log(utf8Vector.toJSON());
// > ["hello", null, "world", null]

Type parameters

  • T: DataType = any

    The DataType of this Builder.

  • TNull = any

    The type(s) of values which will be considered null-value sentinels.

Hierarchy

  • Builder

Index

Constructors

constructor

  • new Builder<T, TNull>(__namedParameters: BuilderOptions<T, TNull>): Builder<T, TNull>
  • Construct a builder with the given Arrow DataType with optional null values, which will be interpreted as "null" when set or appended to the Builder.

    Type parameters

    Parameters

    • __namedParameters: BuilderOptions<T, TNull>

    Returns Builder<T, TNull>

Properties

Readonly children

children: Builder<any, any>[]

finished

finished: boolean = false

A boolean indicating whether Builder.prototype.finish() has been called on this Builder.

readonly

length

length: number = 0

The number of values written to the Builder that haven't been flushed yet.

readonly

Optional Readonly nullValues

nullValues: null | TNull[] | readonly TNull[]

The list of null-value sentinels for this Builder. When one of these values is written to the Builder (either via Builder.prototype.set() or Builder.prototype.append()), a 1-bit is written to this Builder's underlying null BitmapBufferBuilder.

readonly

Readonly stride

stride: number

The number of elements in the underlying values TypedArray that represent a single logical element, determined by this Builder's DataType. This is 1 for most types, but is larger when the DataType is Int64, Uint64, Decimal, DateMillisecond, certain variants of Interval, Time, or Timestamp, FixedSizeBinary, and FixedSizeList.

readonly

type

type: T

The Builder's DataType instance.

readonly

Accessors

ArrayType

  • get ArrayType(): any

byteLength

  • get byteLength(): number
  • Returns number

    The aggregate length (in bytes) of the values that have been written.

nullBitmap

  • get nullBitmap(): null | Uint8Array

nullCount

  • get nullCount(): number

numChildren

  • get numChildren(): number

reservedByteLength

  • get reservedByteLength(): number
  • Returns number

    The aggregate length (in bytes) that has been reserved to write new values.

reservedLength

  • get reservedLength(): number
  • Returns number

    The aggregate number of rows that have been reserved to write new values.

typeIds

  • get typeIds(): null | Int8Array

valueOffsets

  • get valueOffsets(): null | Int32Array

values

  • get values(): null | T["TArray"]

Methods

addChild

  • addChild(child: Builder<any, any>, name?: string): void

append

  • append(value: TNull | T["TValue"]): Builder<T, TNull>
  • Appends a value (or null) to this Builder. This is equivalent to builder.set(builder.length, value).

    Parameters

    • value: TNull | T["TValue"]

      The value to append.

    Returns Builder<T, TNull>

clear

  • Clear this Builder's internal state, including child Builders if applicable, and reset the length to 0.

    Returns Builder<T, TNull>

    The cleared Builder instance.

finish

  • Finalize this Builder, and child builders if applicable.

    Returns Builder<T, TNull>

    The finalized Builder instance.

flush

  • flush(): Data<T>
  • Commit all the values that have been written to their underlying ArrayBuffers, including any child Builders if applicable, and reset the internal Builder state.

    Returns Data<T>

    A Data<T> of the buffers and childData representing the values written.

getChildAt

  • getChildAt<R>(index: number): null | Builder<R, any>
  • Retrieve the child Builder at the supplied index, or null if no child exists at that index.

    Type parameters

    Parameters

    • index: number

      The index of the child Builder to retrieve.

    Returns null | Builder<R, any>

    The child Builder at the supplied index or null.

isValid

  • isValid(value: TNull | T["TValue"]): boolean
  • Validates whether a value is valid (true), or null (false)

    Parameters

    • value: TNull | T["TValue"]

      The value to compare against null the value representations

    Returns boolean

set

  • set(index: number, value: TNull | T["TValue"]): Builder<T, TNull>
  • Write a value (or null-value sentinel) at the supplied index. If the value matches one of the null-value representations, a 1-bit is written to the null BitmapBufferBuilder. Otherwise, a 0 is written to the null BitmapBufferBuilder, and the value is passed to Builder.prototype.setValue().

    Parameters

    • index: number

      The index of the value to write.

    • value: TNull | T["TValue"]

      The value to write at the supplied index.

    Returns Builder<T, TNull>

    The updated Builder instance.

setValid

  • setValid(index: number, valid: boolean): boolean
  • Parameters

    • index: number
    • valid: boolean

    Returns boolean

setValue

  • setValue(index: number, value: T["TValue"]): void
  • Write a value to the underlying buffers at the supplied index, bypassing the null-value check. This is a low-level method that

    Parameters

    • index: number
    • value: T["TValue"]

    Returns void

toVector

  • toVector(): VectorType<T>
  • Flush the Builder and return a Vector<T>.

    Returns VectorType<T>

    A Vector<T> of the flushed values.

Static new

  • new<T, TNull>(options: BuilderOptions<T, TNull>): BuilderType<T, TNull>
  • Create a Builder instance based on the type property of the supplied options object.

    nocollapse

    Type parameters

    • T: DataType<Type, any, T> = any

      The DataType of the Builder to create.

    • TNull = any

      The type(s) of values which will be considered null-value sentinels.

    Parameters

    • options: BuilderOptions<T, TNull>

      An object with a required DataType instance and other optional parameters to be passed to the Builder subclass for the given type.

    Returns BuilderType<T, TNull>

Static throughAsyncIterable

  • throughAsyncIterable<T, TNull>(options: IterableBuilderOptions<T, TNull>): ThroughAsyncIterable<T, TNull>
  • Transform an AsyncIterable of arbitrary JavaScript values into a sequence of Arrow Vector following the chunking semantics defined in the supplied options argument.

    This function returns a function that accepts an AsyncIterable of values to transform. When called, this function returns an AsyncIterator of Vector<T>.

    The resulting AsyncIterator<Vector<T>> yields Vectors based on the queueingStrategy and highWaterMark specified in the options argument.

    • If queueingStrategy is "count" (or omitted), The AsyncIterator<Vector<T>> will flush the underlying Builder (and yield a new Vector<T>) once the Builder's length reaches or exceeds the supplied highWaterMark.
    • If queueingStrategy is "bytes", the AsyncIterator<Vector<T>> will flush the underlying Builder (and yield a new Vector<T>) once its byteLength reaches or exceeds the supplied highWaterMark.
    nocollapse

    Type parameters

    Parameters

    • options: IterableBuilderOptions<T, TNull>

      An object of properties which determine the Builder to create and the chunking semantics to use.

    Returns ThroughAsyncIterable<T, TNull>

    A function which accepts a JavaScript AsyncIterable of values to write, and returns an AsyncIterator that yields Vectors according to the chunking semantics defined in the options argument.

Static throughDOM

  • throughDOM<T, TNull>(options: BuilderTransformOptions<T, TNull>): BuilderTransform<T, TNull>
  • nocollapse

    Type parameters

    Parameters

    • options: BuilderTransformOptions<T, TNull>

    Returns BuilderTransform<T, TNull>

Static throughIterable

  • throughIterable<T, TNull>(options: IterableBuilderOptions<T, TNull>): ThroughIterable<T, TNull>
  • Transform a synchronous Iterable of arbitrary JavaScript values into a sequence of Arrow Vector following the chunking semantics defined in the supplied options argument.

    This function returns a function that accepts an Iterable of values to transform. When called, this function returns an Iterator of Vector<T>.

    The resulting Iterator<Vector<T>> yields Vectors based on the queueingStrategy and highWaterMark specified in the options argument.

    • If queueingStrategy is "count" (or omitted), The Iterator<Vector<T>> will flush the underlying Builder (and yield a new Vector<T>) once the Builder's length reaches or exceeds the supplied highWaterMark.
    • If queueingStrategy is "bytes", the Iterator<Vector<T>> will flush the underlying Builder (and yield a new Vector<T>) once its byteLength reaches or exceeds the supplied highWaterMark.
    nocollapse

    Type parameters

    Parameters

    • options: IterableBuilderOptions<T, TNull>

      An object of properties which determine the Builder to create and the chunking semantics to use.

    Returns ThroughIterable<T, TNull>

    A function which accepts a JavaScript Iterable of values to write, and returns an Iterator that yields Vectors according to the chunking semantics defined in the options argument.

Static throughNode

  • throughNode<T, TNull>(options: BuilderDuplexOptions<T, TNull>): Duplex
  • nocollapse

    Type parameters

    Parameters

    • options: BuilderDuplexOptions<T, TNull>

    Returns Duplex

Generated using TypeDoc