Compute Functions¶
Aggregations¶
|
Test whether all elements in a boolean array evaluate to true. |
|
Test whether any element in a boolean array evaluates to true. |
|
Approximate median of a numeric array with T-Digest algorithm. |
|
Count the number of null / non-null values. |
|
Count the number of unique values. |
|
Find the index of the first occurrence of a given value. |
|
Compute the minimum or maximum values of a numeric array. |
|
Compute the mean of a numeric array. |
|
Compute the minimum or maximum values of a numeric array. |
|
Compute the minimum and maximum values of a numeric array. |
|
Compute the modal (most common) values of a numeric array. |
|
Compute the product of values in a numeric array. |
|
Compute an array of quantiles of a numeric array or chunked array. |
|
Calculate the standard deviation of a numeric array. |
|
Compute the sum of a numeric array. |
|
Approximate quantiles of a numeric array with T-Digest algorithm. |
|
Calculate the variance of a numeric array. |
Cumulative Functions¶
Cumulative functions are vector functions that perform a running total on their
input and output an array containing the corresponding intermediate running values.
By default these functions do not detect overflow. They are also
available in an overflow-checking variant, suffixed _checked
, which
throws an ArrowInvalid
exception when overflow is detected.
|
Compute the cumulative sum over a numeric input. |
|
Compute the cumulative sum over a numeric input. |
Arithmetic Functions¶
By default these functions do not detect overflow. Most functions are also
available in an overflow-checking variant, suffixed _checked
, which
throws an ArrowInvalid
exception when overflow is detected.
|
Calculate the absolute value of the argument element-wise. |
|
Calculate the absolute value of the argument element-wise. |
|
Add the arguments element-wise. |
|
Add the arguments element-wise. |
|
Divide the arguments element-wise. |
|
Divide the arguments element-wise. |
|
Multiply the arguments element-wise. |
|
Multiply the arguments element-wise. |
|
Negate the argument element-wise. |
|
Negate the arguments element-wise. |
|
Raise arguments to power element-wise. |
|
Raise arguments to power element-wise. |
|
Get the signedness of the arguments element-wise. |
|
Takes the square root of arguments element-wise. |
|
Takes the square root of arguments element-wise. |
|
Subtract the arguments element-wise. |
|
Subtract the arguments element-wise. |
Bit-wise Functions¶
|
Bit-wise AND the arguments element-wise. |
|
Bit-wise negate the arguments element-wise. |
|
Bit-wise OR the arguments element-wise. |
|
Bit-wise XOR the arguments element-wise. |
|
Left shift x by y. |
|
Left shift x by y. |
|
Right shift x by y. |
|
Right shift x by y. |
Rounding Functions¶
Rounding functions displace numeric inputs to an approximate value with a simpler representation based on the rounding criterion.
|
Round up to the nearest integer. |
|
Round down to the nearest integer. |
|
Round to a given precision. |
|
Round to a given multiple. |
|
Compute the integral part. |
Logarithmic Functions¶
Logarithmic functions are also supported, and also offer _checked
variants which detect domain errors.
|
Compute natural logarithm. |
|
Compute natural logarithm. |
|
Compute base 10 logarithm. |
|
Compute base 10 logarithm. |
|
Compute natural log of (1+x). |
|
Compute natural log of (1+x). |
|
Compute base 2 logarithm. |
|
Compute base 2 logarithm. |
|
Compute base b logarithm. |
|
Compute base b logarithm. |
Trigonometric Functions¶
Trigonometric functions are also supported, and also offer _checked
variants which detect domain errors where appropriate.
|
Compute the inverse cosine. |
|
Compute the inverse cosine. |
|
Compute the inverse sine. |
|
Compute the inverse sine. |
|
Compute the inverse tangent of x. |
|
Compute the inverse tangent of y/x. |
|
Compute the cosine. |
|
Compute the cosine. |
|
Compute the sine. |
|
Compute the sine. |
|
Compute the tangent. |
|
Compute the tangent. |
Comparisons¶
These functions expect two inputs of the same type. If one of the inputs is null
they return null
.
|
Compare values for equality (x == y). |
|
Compare values for ordered inequality (x > y). |
|
Compare values for ordered inequality (x >= y). |
|
Compare values for ordered inequality (x < y). |
|
Compare values for ordered inequality (x <= y). |
|
Compare values for inequality (x != y). |
These functions take any number of arguments of a numeric or temporal type.
|
Find the element-wise maximum value. |
|
Find the element-wise minimum value. |
Logical Functions¶
These functions normally emit a null when one of the inputs is null. However, Kleene
logic variants are provided (suffixed _kleene
). See User Guide for details.
|
Logical 'and' boolean values. |
|
Logical 'and' boolean values (Kleene logic). |
|
Logical 'and not' boolean values. |
|
Logical 'and not' boolean values (Kleene logic). |
|
Invert boolean values. |
|
Logical 'or' boolean values. |
|
Logical 'or' boolean values (Kleene logic). |
|
Logical 'xor' boolean values. |
String Predicates¶
In these functions an empty string emits false in the output. For ASCII
variants (prefixed ascii_
) a string element with non-ASCII characters
emits false in the output.
The first set of functions emit true if the input contains only characters of a given class.
|
Classify strings as ASCII alphanumeric. |
|
Classify strings as ASCII alphabetic. |
|
Classify strings as ASCII decimal. |
|
Classify strings as ASCII lowercase. |
|
Classify strings as ASCII printable. |
|
Classify strings as ASCII whitespace. |
|
Classify strings as ASCII uppercase. |
|
Classify strings as alphanumeric. |
|
Classify strings as alphabetic. |
|
Classify strings as decimal. |
|
Classify strings as digits. |
|
Classify strings as lowercase. |
|
Classify strings as numeric. |
|
Classify strings as printable. |
|
Classify strings as whitespace. |
|
Classify strings as uppercase. |
The second set of functions also consider the order of characters in the string element.
|
Classify strings as ASCII titlecase. |
|
Classify strings as titlecase. |
The third set of functions examines string elements on a byte-by-byte basis.
|
Classify strings as ASCII. |
String Transforms¶
|
Capitalize the first character of ASCII input. |
|
Transform ASCII input to lowercase. |
|
Reverse ASCII input. |
|
Transform ASCII input by inverting casing. |
|
Titlecase each word of ASCII input. |
|
Transform ASCII input to uppercase. |
|
Compute string lengths. |
|
Repeat a binary string. |
|
Replace a slice of a binary string. |
|
Reverse binary input. |
|
Replace matching non-overlapping substrings with replacement. |
|
Replace matching non-overlapping substrings with replacement. |
|
Capitalize the first character of input. |
|
Compute UTF8 string lengths. |
|
Transform input to lowercase. |
|
Replace a slice of a string. |
|
Reverse input. |
|
Transform input lowercase characters to uppercase and uppercase characters to lowercase. |
|
Titlecase each word of input. |
|
Transform input to uppercase. |
String Padding¶
|
Center strings by padding with a given character. |
|
Right-align strings by padding with a given character. |
|
Left-align strings by padding with a given character. |
|
Center strings by padding with a given character. |
|
Right-align strings by padding with a given character. |
|
Left-align strings by padding with a given character. |
String Trimming¶
|
Trim leading characters. |
|
Trim leading ASCII whitespace characters. |
|
Trim trailing characters. |
|
Trim trailing ASCII whitespace characters. |
|
Trim leading and trailing characters. |
|
Trim leading and trailing ASCII whitespace characters. |
|
Trim leading characters. |
|
Trim leading whitespace characters. |
|
Trim trailing characters. |
|
Trim trailing whitespace characters. |
|
Trim leading and trailing characters. |
|
Trim leading and trailing whitespace characters. |
String Splitting¶
|
Split string according to any ASCII whitespace. |
|
Split string according to separator. |
|
Split string according to regex pattern. |
|
Split string according to any Unicode whitespace. |
String Component Extraction¶
|
Extract substrings captured by a regex pattern. |
String Joining¶
|
Join a list of strings together with a separator. |
|
Join string arguments together, with the last argument as separator. |
String Slicing¶
|
Slice binary string. |
|
Slice string. |
Containment Tests¶
|
Count occurrences of substring. |
|
Count occurrences of substring. |
|
Check if strings end with a literal pattern. |
|
Find first occurrence of substring. |
|
Find location of first match of regex pattern. |
|
Return index of each element in a set of values. |
|
Find each element in a set of values. |
|
Match strings against SQL-style LIKE pattern. |
|
Match strings against literal pattern. |
|
Match strings against regex pattern. |
|
Check if strings start with a literal pattern. |
|
Return the indices of the values in the array that are non-zero. |
Categorizations¶
|
Return true if value is finite. |
|
Return true if infinity. |
|
Return true if NaN. |
|
Return true if null (and optionally NaN). |
|
Return true if non-null. |
|
Return true if non-null, else return null. |
Selecting / Multiplexing¶
|
Choose values based on multiple conditions. |
|
Choose values from several arrays. |
|
Select the first non-null value. |
|
Choose values based on a condition. |
Conversions¶
|
Cast array values to another data type. |
|
Round temporal values up to nearest multiple of specified time unit. |
|
Round temporal values down to nearest multiple of specified time unit. |
|
Round temporal values to the nearest multiple of specified time unit. |
|
Format temporal values according to a format string. |
|
Parse timestamps. |
Temporal Component Extraction¶
|
Extract day number. |
|
Extract day of the week number. |
|
Extract day of year number. |
|
Extract hour value. |
|
Extract ISO week of year number. |
|
Extract ISO year number. |
|
Extract (ISO year, ISO week, ISO day of week) struct. |
|
Extract if year is a leap year. |
|
Extract microsecond values. |
|
Extract millisecond values. |
|
Extract minute values. |
|
Extract month number. |
|
Extract nanosecond values. |
|
Extract quarter of year number. |
|
Extract second values. |
|
Extract subsecond values. |
|
Extract US week of year number. |
|
Extract US epidemiological year number. |
|
Extract week of year number. |
|
Extract year number. |
|
Extract (year, month, day) struct. |
Temporal Difference¶
|
Compute the number of days and milliseconds between two timestamps. |
|
Compute the number of days between two timestamps. |
|
Compute the number of hours between two timestamps. |
|
Compute the number of microseconds between two timestamps. |
|
Compute the number of millisecond boundaries between two timestamps. |
|
Compute the number of minute boundaries between two timestamps. |
|
Compute the number of months, days and nanoseconds between two timestamps. |
|
Compute the number of months between two timestamps. |
|
Compute the number of nanoseconds between two timestamps. |
|
Compute the number of quarters between two timestamps. |
|
Compute the number of seconds between two timestamps. |
|
Compute the number of weeks between two timestamps. |
|
Compute the number of years between two timestamps. |
Timezone Handling¶
|
Convert naive timestamp to timezone-aware timestamp. |
Associative Transforms¶
|
Dictionary-encode array. |
|
Compute unique elements. |
|
Compute counts of unique elements. |
Selections¶
|
Filter with a boolean selection filter. |
|
Select values from an array based on indices from another array. |
|
Drop nulls from the input. |
|
Filter with a boolean selection filter. |
|
Select values (or records) from array- or table-like data given integer selection indices. |
Sorts and Partitions¶
|
Return the indices that would sort an array. |
|
Return the indices that would partition an array around a pivot. |
|
Select the indices of the first k ordered elements from the input. |
|
Return the indices that would sort an array, record batch or table. |
Structural Transforms¶
|
Carry non-null values backward to fill null slots. |
|
Carry non-null values forward to fill null slots. |
|
Compute elements using of nested list values using an index. |
|
Flatten list values. |
|
Compute parent indices of nested list values. |
|
Compute slice of list-like array. |
|
Compute list lengths. |
|
Wrap Arrays into a StructArray. |
|
Find the items corresponding to a given key in a Map. |
|
Replace items selected with a mask. |
|
Extract children of a struct or union by index. |
Compute Options¶
|
Options for the array_sort_indices function. |
|
Options for the assume_timezone function. |
|
Options for the cast function. |
|
Options for the count function. |
|
Options for the count function. |
|
Options for cumulative_sum function. |
|
Options for the day_of_week function. |
|
Options for dictionary encoding. |
|
Options for element-wise aggregate functions. |
|
Options for the extract_regex function. |
|
Options for selecting with a boolean filter. |
|
Options for the index function. |
|
Options for the binary_join_element_wise function. |
|
Options for list array slicing. |
|
Options for the make_struct function. |
|
Options for the map_lookup function. |
|
Options for looking for a substring. |
|
Options for the mode function. |
|
Options for the is_null function. |
|
Options for padding strings. |
|
Options for the partition_nth_indices function. |
|
Options for the quantile function. |
|
Options for replacing slices. |
|
Options for replacing matched substrings. |
|
Options for rounding numbers. |
|
Options for rounding temporal values. |
|
Options for rounding numbers to a multiple. |
|
Options for scalar aggregations. |
|
Options for scalar aggregations. |
|
Options for top/bottom k-selection. |
|
Options for the is_in and index_in functions. |
|
Options for slicing. |
|
Options for the sort_indices function. |
|
Options for splitting on whitespace. |
|
Options for splitting on a string pattern. |
|
Options for the strftime function. |
|
Options for the strptime function. |
|
Options for the struct_field function. |
|
Options for the take and array_take functions. |
|
Options for the tdigest function. |
|
Options for the tdigest function. |
|
Options for trimming characters from strings. |
|
Options for the variance and stddev functions. |
|
Options for the week function. |
User-Defined Functions¶
|
Register a user-defined scalar function. |
Per-invocation function context/state. |