Datafusion-Specific Functions¶
These SQL functions are specific to DataFusion, or they are well known and have functionality which is specific to DataFusion. Specifically, the to_timestamp_xx()
functions exist due to Arrow’s support for multiple timestamp resolutions.
to_timestamp
¶
to_timestamp()
is similar to the standard SQL function. It performs conversions to type Timestamp(Nanoseconds, None)
, from:
Timestamp strings
1997-01-31T09:26:56.123Z
# RCF33391997-01-31T09:26:56.123-05:00
# RCF33391997-01-31 09:26:56.123-05:00
# close to RCF3339 but with a space er than T1997-01-31T09:26:56.123
# close to RCF3339 but no timezone et specified1997-01-31 09:26:56.123
# close to RCF3339 but uses a space and timezone offset1997-01-31 09:26:56
# close to RCF3339, no fractional seconds
An Int64 array/column, values are nanoseconds since Epoch UTC
Other Timestamp() columns or values
Note that conversions from other Timestamp and Int64 types can also be performed using CAST(.. AS Timestamp)
. However, the conversion functionality here is present for consistency with the other to_timestamp_xx()
functions.
to_timestamp_millis
¶
to_timestamp_millis()
does conversions to type Timestamp(Milliseconds, None)
, from:
Timestamp strings, the same as supported by the regular timestamp() function (except the output is a timestamp of Milliseconds resolution)
1997-01-31T09:26:56.123Z
# RCF33391997-01-31T09:26:56.123-05:00
# RCF33391997-01-31 09:26:56.123-05:00
# close to RCF3339 but with a space er than T1997-01-31T09:26:56.123
# close to RCF3339 but no timezone et specified1997-01-31 09:26:56.123
# close to RCF3339 but uses a space and timezone offset1997-01-31 09:26:56
# close to RCF3339, no fractional seconds
An Int64 array/column, values are milliseconds since Epoch UTC
Other Timestamp() columns or values
Note that CAST(.. AS Timestamp)
converts to Timestamps with Nanosecond resolution; this function is the only way to convert/cast to millisecond resolution.
to_timestamp_micros
¶
to_timestamp_micros()
does conversions to type Timestamp(Microseconds, None)
, from:
Timestamp strings, the same as supported by the regular timestamp() function (except the output is a timestamp of microseconds resolution)
1997-01-31T09:26:56.123Z
# RCF33391997-01-31T09:26:56.123-05:00
# RCF33391997-01-31 09:26:56.123-05:00
# close to RCF3339 but with a space er than T1997-01-31T09:26:56.123
# close to RCF3339 but no timezone et specified1997-01-31 09:26:56.123
# close to RCF3339 but uses a space and timezone offset1997-01-31 09:26:56
# close to RCF3339, no fractional seconds
An Int64 array/column, values are microseconds since Epoch UTC
Other Timestamp() columns or values
Note that CAST(.. AS Timestamp)
converts to Timestamps with Nanosecond resolution; this function is the only way to convert/cast to microsecond resolution.
to_timestamp_seconds
¶
to_timestamp_seconds()
does conversions to type Timestamp(Seconds, None)
, from:
Timestamp strings, the same as supported by the regular timestamp() function (except the output is a timestamp of secondseconds resolution)
1997-01-31T09:26:56.123Z
# RCF33391997-01-31T09:26:56.123-05:00
# RCF33391997-01-31 09:26:56.123-05:00
# close to RCF3339 but with a space er than T1997-01-31T09:26:56.123
# close to RCF3339 but no timezone et specified1997-01-31 09:26:56.123
# close to RCF3339 but uses a space and timezone offset1997-01-31 09:26:56
# close to RCF3339, no fractional seconds
An Int64 array/column, values are seconds since Epoch UTC
Other Timestamp() columns or values
Note that CAST(.. AS Timestamp)
converts to Timestamps with Nanosecond resolution; this function is the only way to convert/cast to seconds resolution.