!
! expr - Logical not.
%
expr1 % expr2 - Returns the remainder after expr1
/expr2
.
Examples:
&
expr1 & expr2 - Returns the result of bitwise AND of expr1
and expr2
.
Examples:
*
expr1 * expr2 - Returns expr1
*expr2
.
Examples:
+
expr1 + expr2 - Returns expr1
+expr2
.
Examples:
-
expr1 - expr2 - Returns expr1
-expr2
.
Examples:
/
expr1 / expr2 - Returns expr1
/expr2
. It always performs floating point division.
Examples:
<
expr1 < expr2 - Returns true if expr1
is less than expr2
.
<=
expr1 <= expr2 - Returns true if expr1
is less than or equal to expr2
.
<=>
expr1 <=> expr2 - Returns same result as the EQUAL(=) operator for non-null operands, but returns true if both are null, false if one of the them is null.
=
expr1 = expr2 - Returns true if expr1
equals expr2
, or false otherwise.
==
expr1 == expr2 - Returns true if expr1
equals expr2
, or false otherwise.
>
expr1 > expr2 - Returns true if expr1
is greater than expr2
.
>=
expr1 >= expr2 - Returns true if expr1
is greater than or equal to expr2
.
^
expr1 ^ expr2 - Returns the result of bitwise exclusive OR of expr1
and expr2
.
Examples:
abs
abs(expr) - Returns the absolute value of the numeric value.
Examples:
acos
acos(expr) - Returns the inverse cosine (a.k.a. arccosine) of expr
if -1<=expr
<=1 or NaN otherwise.
Examples:
add_months
add_months(start_date, num_months) - Returns the date that is num_months
after start_date
.
Examples:
and
expr1 and expr2 - Logical AND.
approx_count_distinct
approx_count_distinct(expr[, relativeSD]) - Returns the estimated cardinality by HyperLogLog++. relativeSD
approx_percentile
approx_percentile(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric column col
at the given percentage. The value of percentage must be between 0.0 and 1.0. The accuracy
parameter (default: 10000) is a positive numeric literal which controls approximation accuracy at the cost of memory. Higher value of accuracy
yields better accuracy, 1.0/accuracy
is the relative error of the approximation. When percentage
is an array, each value of the percentage array must be between 0.0 and 1.0. In this case, returns the approximate percentile array of column col
Examples:
array
array(expr, ...) - Returns an array with the given elements.
Examples:
array_contains
array_contains(array, value) - Returns true if the array contains the value.
Examples:
ascii
ascii(str) - Returns the numeric value of the first character of str
.
Examples:
asin
asin(expr) - Returns the inverse sine (a.k.a. arcsine) the arc sin of expr
if -1<=expr
<=1 or NaN otherwise.
Examples:
assert_true
assert_true(expr) - Throws an exception if expr
Examples:
atan
atan(expr) - Returns the inverse tangent (a.k.a. arctangent).
Examples:
atan2
atan2(expr1, expr2) - Returns the angle in radians between the positive x-axis of a plane and the point given by the coordinates (expr1
, expr2
).
Examples:
avg
avg(expr) - Returns the mean calculated from values of a group.
base64
base64(bin) - Converts the argument from a binary bin
Examples:
bigint
bigint(expr) - Casts the value expr
to the target data type bigint
.
bin
bin(expr) - Returns the string representation of the long value expr
Examples:
binary
binary(expr) - Casts the value expr
to the target data type binary
.
bit_length
bit_length(expr) - Returns the bit length of expr
Examples:
boolean
boolean(expr) - Casts the value expr
to the target data type boolean
.
bround
bround(expr, d) - Returns expr
rounded to d
Examples:
cast
cast(expr AS type) - Casts the value expr
to the target data type type
.
Examples:
cbrt
cbrt(expr) - Returns the cube root of expr
.
Examples:
ceil
ceil(expr) - Returns the smallest integer not smaller than expr
.
Examples:
ceiling
ceiling(expr) - Returns the smallest integer not smaller than expr
.
Examples:
char
char(expr) - Returns the ASCII character having the binary equivalent to expr
. If n is larger than 256 the result is equivalent to chr(n % 256)
Examples:
char_length
char_length(expr) - Returns the character length of expr
Examples:
character_length
character_length(expr) - Returns the character length of expr
Examples:
chr
chr(expr) - Returns the ASCII character having the binary equivalent to expr
. If n is larger than 256 the result is equivalent to chr(n % 256)
Examples:
coalesce
coalesce(expr1, expr2, ...) - Returns the first non-null argument if exists. Otherwise, null.
Examples:
collect_list
collect_list(expr) - Collects and returns a list of non-unique elements.
collect_set
collect_set(expr) - Collects and returns a set of unique elements.
concat
concat(str1, str2, ..., strN) - Returns the concatenation of str1, str2, ..., strN.
Examples:
concat_ws
concat_ws(sep, [str | array(str)]+) - Returns the concatenation of the strings separated by sep
.
Examples:
conv
conv(num, from_base, to_base) - Convert num
from from_base
to to_base
.
Examples:
corr
corr(expr1, expr2) - Returns Pearson coefficient of correlation between a set of number pairs.
cos
cos(expr) - Returns the cosine of expr
.
Examples:
cosh
cosh(expr) - Returns the hyperbolic cosine of expr
.
Examples:
cot
cot(expr) - Returns the cotangent of expr
.
Examples:
count
count(*) - Returns the total number of retrieved rows, including rows containing null.
count(expr) - Returns the number of rows for which the supplied expression is non-null.
count(DISTINCT expr[, expr...]) - Returns the number of rows for which the supplied expression(s) are unique and non-null.
count_min_sketch
count_min_sketch(col, eps, confidence, seed) - Returns a count-min sketch of a column with the given esp, confidence and seed. The result is an array of bytes, which can be deserialized to a CountMinSketch
covar_pop
covar_pop(expr1, expr2) - Returns the population covariance of a set of number pairs.
covar_samp
covar_samp(expr1, expr2) - Returns the sample covariance of a set of number pairs.
crc32
crc32(expr) - Returns a cyclic redundancy check value of the expr
Examples:
cube
cume_dist
cume_dist() - Computes the position of a value relative to all values in the partition.
current_database
current_database() - Returns the current database.
Examples:
current_date
current_date() - Returns the current date at the start of query evaluation.
current_timestamp
current_timestamp() - Returns the current timestamp at the start of query evaluation.
date
date(expr) - Casts the value expr
to the target data type date
.
date_add
date_add(start_date, num_days) - Returns the date that is num_days
after start_date
.
Examples:
date_format
date_format(timestamp, fmt) - Converts timestamp
to a value of string in the format specified by the date format fmt
.
Examples:
date_sub
date_sub(start_date, num_days) - Returns the date that is num_days
before start_date
.
Examples:
datediff
datediff(endDate, startDate) - Returns the number of days from startDate
to endDate
.
Examples:
day
day(date) - Returns the day of month of the date/timestamp.
Examples:
dayofmonth
dayofmonth(date) - Returns the day of month of the date/timestamp.
Examples:
dayofweek
dayofweek(date) - Returns the day of the week for date/timestamp (1 = Sunday, 2 = Monday, ..., 7 = Saturday).
Examples:
dayofyear
dayofyear(date) - Returns the day of year of the date/timestamp.
Examples:
decimal
decimal(expr) - Casts the value expr
to the target data type decimal
.
decode
decode(bin, charset) - Decodes the first argument using the second argument character set.
Examples:
degrees
degrees(expr) - Converts radians to degrees.
Examples:
dense_rank
dense_rank() - Computes the rank of a value in a group of values. The result is one plus the previously assigned rank value. Unlike the function rank, dense_rank will not produce gaps in the ranking sequence.
double
double(expr) - Casts the value expr
to the target data type double
.
e
e() - Returns Euler's number, e.
Examples:
elt
elt(n, str1, str2, ...) - Returns the n
-th string, e.g., returns str2
when n
Examples:
encode
encode(str, charset) - Encodes the first argument using the second argument character set.
Examples:
exp
exp(expr) - Returns e to the power of expr
.
Examples:
explode
explode(expr) - Separates the elements of array expr
into multiple rows, or the elements of map expr
Examples:
explode_outer
explode_outer(expr) - Separates the elements of array expr
into multiple rows, or the elements of map expr
Examples:
expm1
expm1(expr) - Returns exp(expr
) - 1.
Examples:
factorial
factorial(expr) - Returns the factorial of expr
. expr
Examples:
find_in_set
find_in_set(str, str_array) - Returns the index (1-based) of the given string (str
) in the comma-delimited list (str_array
). Returns 0, if the string was not found or if the given string (str
) contains a comma.
Examples:
first
first(expr[, isIgnoreNull]) - Returns the first value of expr
for a group of rows. If isIgnoreNull
first_value
first_value(expr[, isIgnoreNull]) - Returns the first value of expr
for a group of rows. If isIgnoreNull
is true, returns only non-null values.
float
float(expr) - Casts the value expr
to the target data type float
.
floor
floor(expr) - Returns the largest integer not greater than expr
.
Examples:
format_number
format_number(expr1, expr2) - Formats the number expr1
like '#,###,###.##', rounded to expr2
decimal places. If expr2
Examples:
format_string
format_string(strfmt, obj, ...) - Returns a formatted string from printf-style format strings.
Examples:
from_json
from_json(jsonStr, schema[, options]) - Returns a struct value with the given jsonStr
and schema
.
Examples:
Since: 2.2.0
from_unixtime
from_unixtime(unix_time, format) - Returns unix_time
in the specified format
.
Examples:
from_utc_timestamp
from_utc_timestamp(timestamp, timezone) - Given a timestamp, which corresponds to a certain time of day in UTC, returns another timestamp that corresponds to the same time of day in the given timezone.
Examples:
get_json_object
get_json_object(json_txt, path) - Extracts a json object from path
.
Examples:
greatest
greatest(expr, ...) - Returns the greatest value of all parameters, skipping null values.
Examples:
grouping
grouping_id
hash
hash(expr1, expr2, ...) - Returns a hash value of the arguments.
Examples:
hex
hex(expr) - Converts expr
Examples:
hour
hour(timestamp) - Returns the hour component of the string/timestamp.
Examples:
hypot
hypot(expr1, expr2) - Returns sqrt(expr1
2 + expr2
2).
Examples:
if
if(expr1, expr2, expr3) - If expr1
evaluates to true, then returns expr2
; otherwise returns expr3
.
Examples:
ifnull
ifnull(expr1, expr2) - Returns expr2
if expr1
is null, or expr1
Examples:
in
expr1 in(expr2, expr3, ...) - Returns true if expr
initcap
initcap(str) - Returns str
Examples:
inline
inline(expr) - Explodes an array of structs into a table.
Examples:
inline_outer
inline_outer(expr) - Explodes an array of structs into a table.
Examples:
input_file_block_length
input_file_block_length() - Returns the length of the block being read, or -1 if not available.
input_file_block_start
input_file_block_start() - Returns the start offset of the block being read, or -1 if not available.
input_file_name
input_file_name() - Returns the name of the file being read, or empty string if not available.
instr
instr(str, substr) - Returns the (1-based) index of the first occurrence of substr
in str
.
Examples:
int
int(expr) - Casts the value expr
to the target data type int
.
isnan
isnan(expr) - Returns true if expr
Examples:
isnotnull
isnotnull(expr) - Returns true if expr
Examples:
isnull
isnull(expr) - Returns true if expr
Examples:
java_method
java_method(class, method[, arg1[, arg2 ..]]) - Calls a method with reflection.
Examples:
json_tuple
json_tuple(jsonStr, p1, p2, ..., pn) - Returns a tuple like the function get_json_object, but it takes multiple names. All the input parameters and output column types are string.
Examples:
kurtosis
kurtosis(expr) - Returns the kurtosis value calculated from values of a group.
lag
lag(input[, offset[, default]]) - Returns the value of input
at the offset
th row before the current row in the window. The default value of offset
is 1 and the default value of default
is null. If the value of input
at the offset
th row is null, null is returned. If there is no such offset row (e.g., when the offset is 1, the first row of the window does not have any previous row), default
last
last(expr[, isIgnoreNull]) - Returns the last value of expr
for a group of rows. If isIgnoreNull
last_day
last_day(date) - Returns the last day of the month which the date belongs to.
Examples:
last_value
last_value(expr[, isIgnoreNull]) - Returns the last value of expr
for a group of rows. If isIgnoreNull
lcase
lcase(str) - Returns str
Examples:
lead
lead(input[, offset[, default]]) - Returns the value of input
at the offset
th row after the current row in the window. The default value of offset
is 1 and the default value of default
is null. If the value of input
at the offset
th row is null, null is returned. If there is no such an offset row (e.g., when the offset is 1, the last row of the window does not have any subsequent row), default
least
least(expr, ...) - Returns the least value of all parameters, skipping null values.
Examples:
left
left(str, len) - Returns the leftmost len
(len
can be string type) characters from the string str
,if len
Examples:
length
length(expr) - Returns the character length of expr
Examples:
levenshtein
levenshtein(str1, str2) - Returns the Levenshtein distance between the two given strings.
Examples:
like
str like pattern - Returns true if str matches pattern, null if any arguments are null, false otherwise.
Arguments:
- str - a string expression
- pattern - a string expression. The pattern is a string which is matched literally, with exception to the following special symbols:
_ matches any one character in the input (similar to . in posix regular expressions)
% matches zero or more characters in the input (similar to .* in posix regular expressions)
The escape character is '\'. If an escape character precedes a special symbol or another escape character, the following character is matched literally. It is invalid to escape any other character.
Since Spark 2.0, string literals are unescaped in our SQL parser. For example, in order to match "\abc", the pattern should be "\abc".
When SQL config 'spark.sql.parser.escapedStringLiterals' is enabled, it fallbacks to Spark 1.6 behavior regarding string literal parsing. For example, if the config is enabled, the pattern to match "\abc" should be "\abc".
Examples:
Note:
Use RLIKE to match with standard regular expressions.
ln
ln(expr) - Returns the natural logarithm (base e) of expr
.
Examples:
locate
locate(substr, str[, pos]) - Returns the position of the first occurrence of substr
in str
after position pos
. The given pos
Examples:
log
log(base, expr) - Returns the logarithm of expr
with base
.
Examples:
log10
log10(expr) - Returns the logarithm of expr
Examples:
log1p
log1p(expr) - Returns log(1 + expr
).
Examples:
log2
log2(expr) - Returns the logarithm of expr
Examples:
lower
lower(str) - Returns str
Examples:
lpad
lpad(str, len, pad) - Returns str
, left-padded with pad
to a length of len
. If str
is longer than len
, the return value is shortened to len
Examples:
ltrim
ltrim(str) - Removes the leading and trailing space characters from str
.
Examples:
map
map(key0, value0, key1, value1, ...) - Creates a map with the given key/value pairs.
Examples:
map_keys
map_keys(map) - Returns an unordered array containing the keys of the map.
Examples:
map_values
map_values(map) - Returns an unordered array containing the values of the map.
Examples:
max
max(expr) - Returns the maximum value of expr
.
md5
md5(expr) - Returns an MD5 128-bit checksum as a hex string of expr
.
Examples:
mean
mean(expr) - Returns the mean calculated from values of a group.
min
min(expr) - Returns the minimum value of expr
.
minute
minute(timestamp) - Returns the minute component of the string/timestamp.
Examples:
mod
expr1 mod expr2 - Returns the remainder after expr1
/expr2
.
Examples:
monotonically_increasing_id
monotonically_increasing_id() - Returns monotonically increasing 64-bit integers. The generated ID is guaranteed to be monotonically increasing and unique, but not consecutive. The current implementation puts the partition ID in the upper 31 bits, and the lower 33 bits represent the record number within each partition. The assumption is that the data frame has less than 1 billion partitions, and each partition has less than 8 billion records.
month
month(date) - Returns the month component of the date/timestamp.
Examples:
months_between
months_between(timestamp1, timestamp2) - Returns number of months between timestamp1
and timestamp2
.
Examples:
named_struct
named_struct(name1, val1, name2, val2, ...) - Creates a struct with the given field names and values.
Examples:
nanvl
nanvl(expr1, expr2) - Returns expr1
if it's not NaN, or expr2
Examples:
negative
negative(expr) - Returns the negated value of expr
.
Examples:
next_day
next_day(start_date, day_of_week) - Returns the first date which is later than start_date
Examples:
not
not expr - Logical not.
now
now() - Returns the current timestamp at the start of query evaluation.
ntile
ntile(n) - Divides the rows for each window partition into n
buckets ranging from 1 to at most n
.
nullif
nullif(expr1, expr2) - Returns null if expr1
equals to expr2
, or expr1
Examples:
nvl
nvl(expr1, expr2) - Returns expr2
if expr1
is null, or expr1
Examples:
nvl2
nvl2(expr1, expr2, expr3) - Returns expr2
if expr1
is not null, or expr3
Examples:
octet_length
octet_length(expr) - Returns the byte length of expr
Examples:
or
expr1 or expr2 - Logical OR.
parse_url
parse_url(url, partToExtract[, key]) - Extracts a part from a URL.
Examples:
percent_rank
percent_rank() - Computes the percentage ranking of a value in a group of values.
percentile
percentile(col, percentage [, frequency]) - Returns the exact percentile value of numeric column col
at the given percentage. The value of percentage must be between 0.0 and 1.0. The value of frequency should be positive integralpercentile(col, array(percentage1 [, percentage2]...) [, frequency]) - Returns the exact percentile value array of numeric column col
percentile_approx
percentile_approx(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric column col
at the given percentage. The value of percentage must be between 0.0 and 1.0. The accuracy
parameter (default: 10000) is a positive numeric literal which controls approximation accuracy at the cost of memory. Higher value of accuracy
yields better accuracy, 1.0/accuracy
is the relative error of the approximation. When percentage
is an array, each value of the percentage array must be between 0.0 and 1.0. In this case, returns the approximate percentile array of column col
Examples:
pi
pi() - Returns pi.
Examples:
pmod
pmod(expr1, expr2) - Returns the positive value of expr1
mod expr2
.
Examples:
posexplode
posexplode(expr) - Separates the elements of array expr
into multiple rows with positions, or the elements of map expr
Examples:
posexplode_outer
posexplode_outer(expr) - Separates the elements of array expr
into multiple rows with positions, or the elements of map expr
Examples:
position
position(substr, str[, pos]) - Returns the position of the first occurrence of substr
in str
after position pos
. The given pos
Examples:
positive
positive(expr) - Returns the value of expr
.
pow
pow(expr1, expr2) - Raises expr1
to the power of expr2
.
Examples:
power
power(expr1, expr2) - Raises expr1
to the power of expr2
.
Examples:
printf
printf(strfmt, obj, ...) - Returns a formatted string from printf-style format strings.
Examples:
quarter
quarter(date) - Returns the quarter of the year for date, in the range 1 to 4.
Examples:
radians
radians(expr) - Converts degrees to radians.
Examples:
rand
rand([seed]) - Returns a random value with independent and identically distributed (i.i.d.) uniformly distributed values in [0, 1).
Examples:
randn
randn([seed]) - Returns a random value with independent and identically distributed (i.i.d.) values drawn from the standard normal distribution.
Examples:
rank
rank() - Computes the rank of a value in a group of values. The result is one plus the number of rows preceding or equal to the current row in the ordering of the partition. The values will produce gaps in the sequence.
reflect
reflect(class, method[, arg1[, arg2 ..]]) - Calls a method with reflection.
Examples:
regexp_extract
regexp_extract(str, regexp[, idx]) - Extracts a group that matches regexp
.
Examples:
regexp_replace
regexp_replace(str, regexp, rep) - Replaces all substrings of str
that match regexp
with rep
.
Examples:
repeat
repeat(str, n) - Returns the string which repeats the given string value n times.
Examples:
replace
replace(str, search[, replace]) - Replaces all occurrences of search
with replace
.
Arguments:
- str - a string expression
- search - a string expression. If
search
- is not found in
str
- ,
str
- replace - a string expression. If
replace
- is not specified or is an empty string, nothing replaces the string that is removed from
str
- .
Examples:
reverse
reverse(str) - Returns the reversed given string.
Examples:
right
right(str, len) - Returns the rightmost len
(len
can be string type) characters from the string str
,if len
Examples:
rint
rint(expr) - Returns the double value that is closest in value to the argument and is equal to a mathematical integer.
Examples:
rlike
str rlike regexp - Returns true if str
matches regexp
, or false otherwise.
Arguments:
- str - a string expression
- regexp - a string expression. The pattern string should be a Java regular expression.
Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser. For example, to match "\abc", a regular expression for
regexp
- There is a SQL config 'spark.sql.parser.escapedStringLiterals' that can be used to fallback to the Spark 1.6 behavior regarding string literal parsing. For example, if the config is enabled, the
regexp
Examples:
Note:
Use LIKE to match with simple string pattern.
rollup
round
round(expr, d) - Returns expr
rounded to d
Examples:
row_number
row_number() - Assigns a unique, sequential number to each row, starting with one, according to the ordering of rows within the window partition.
rpad
rpad(str, len, pad) - Returns str
, right-padded with pad
to a length of len
. If str
is longer than len
, the return value is shortened to len
Examples:
rtrim
rtrim(str) - Removes the trailing space characters from str
.
Examples:
second
second(timestamp) - Returns the second component of the string/timestamp.
Examples:
sentences
sentences(str[, lang, country]) - Splits str
Examples:
sha
sha(expr) - Returns a sha1 hash value as a hex string of the expr
.
Examples:
sha1
sha1(expr) - Returns a sha1 hash value as a hex string of the expr
.
Examples:
sha2
sha2(expr, bitLength) - Returns a checksum of SHA-2 family as a hex string of expr
. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256.
Examples:
shiftleft
shiftleft(base, expr) - Bitwise left shift.
Examples:
shiftright
shiftright(base, expr) - Bitwise (signed) right shift.
Examples:
shiftrightunsigned
shiftrightunsigned(base, expr) - Bitwise unsigned right shift.
Examples:
sign
sign(expr) - Returns -1.0, 0.0 or 1.0 as expr
Examples:
signum
signum(expr) - Returns -1.0, 0.0 or 1.0 as expr
Examples:
sin
sin(expr) - Returns the sine of expr
.
Examples:
sinh
sinh(expr) - Returns the hyperbolic sine of expr
.
Examples:
size
size(expr) - Returns the size of an array or a map. Returns -1 if null.
Examples:
skewness
skewness(expr) - Returns the skewness value calculated from values of a group.
smallint
smallint(expr) - Casts the value expr
to the target data type smallint
.
sort_array
sort_array(array[, ascendingOrder]) - Sorts the input array in ascending or descending order according to the natural ordering of the array elements.
Examples:
soundex
soundex(str) - Returns Soundex code of the string.
Examples:
space
space(n) - Returns a string consisting of n
Examples:
spark_partition_id
spark_partition_id() - Returns the current partition id.
split
split(str, regex) - Splits str
around occurrences that match regex
.
Examples:
sqrt
sqrt(expr) - Returns the square root of expr
.
Examples:
stack
stack(n, expr1, ..., exprk) - Separates expr1
, ..., exprk
into n
Examples:
std
std(expr) - Returns the sample standard deviation calculated from values of a group.
stddev
stddev(expr) - Returns the sample standard deviation calculated from values of a group.
stddev_pop
stddev_pop(expr) - Returns the population standard deviation calculated from values of a group.
stddev_samp
stddev_samp(expr) - Returns the sample standard deviation calculated from values of a group.
str_to_map
str_to_map(text[, pairDelim[, keyValueDelim]]) - Creates a map after splitting the text into key/value pairs using delimiters. Default delimiters are ',' for pairDelim
and ':' for keyValueDelim
.
Examples:
string
string(expr) - Casts the value expr
to the target data type string
.
struct
struct(col1, col2, col3, ...) - Creates a struct with the given field values.
substr
substr(str, pos[, len]) - Returns the substring of str
that starts at pos
and is of length len
, or the slice of byte array that starts at pos
and is of length len
.
Examples:
substring
substring(str, pos[, len]) - Returns the substring of str
that starts at pos
and is of length len
, or the slice of byte array that starts at pos
and is of length len
.
Examples:
substring_index
substring_index(str, delim, count) - Returns the substring from str
before count
occurrences of the delimiter delim
. If count
is positive, everything to the left of the final delimiter (counting from the left) is returned. If count
is negative, everything to the right of the final delimiter (counting from the right) is returned. The function substring_index performs a case-sensitive match when searching for delim
.
Examples:
sum
sum(expr) - Returns the sum calculated from values of a group.
tan
tan(expr) - Returns the tangent of expr
.
Examples:
tanh
tanh(expr) - Returns the hyperbolic tangent of expr
.
Examples:
timestamp
timestamp(expr) - Casts the value expr
to the target data type timestamp
.
tinyint
tinyint(expr) - Casts the value expr
to the target data type tinyint
.
to_date
to_date(date_str[, fmt]) - Parses the date_str
expression with the fmt
expression to a date. Returns null with invalid input. By default, it follows casting rules to a date if the fmt
Examples:
to_json
to_json(expr[, options]) - Returns a json string with a given struct value
Examples:
Since: 2.2.0
to_timestamp
to_timestamp(timestamp[, fmt]) - Parses the timestamp
expression with the fmt
expression to a timestamp. Returns null with invalid input. By default, it follows casting rules to a timestamp if the fmt
Examples:
to_unix_timestamp
to_unix_timestamp(expr[, pattern]) - Returns the UNIX timestamp of the given time.
Examples:
to_utc_timestamp
to_utc_timestamp(timestamp, timezone) - Given a timestamp, which corresponds to a certain time of day in the given timezone, returns another timestamp that corresponds to the same time of day in UTC.
Examples:
translate
translate(input, from, to) - Translates the input
string by replacing the characters present in the from
string with the corresponding characters in the to
Examples:
trim
trim(str) - Removes the leading and trailing space characters from str
.
Examples:
trunc
trunc(date, fmt) - Returns date
with the time portion of the day truncated to the unit specified by the format model fmt
.
Examples:
ucase
ucase(str) - Returns str
Examples:
unbase64
unbase64(str) - Converts the argument from a base 64 string str
Examples:
unhex
unhex(expr) - Converts hexadecimal expr
Examples:
unix_timestamp
unix_timestamp([expr[, pattern]]) - Returns the UNIX timestamp of current or specified time.
Examples:
upper
upper(str) - Returns str
Examples:
uuid
uuid() - Returns an universally unique identifier (UUID) string. The value is returned as a canonical UUID 36-character string.
Examples:
var_pop
var_pop(expr) - Returns the population variance calculated from values of a group.
var_samp
var_samp(expr) - Returns the sample variance calculated from values of a group.
variance
variance(expr) - Returns the sample variance calculated from values of a group.
weekofyear
weekofyear(date) - Returns the week of the year of the given date. A week is considered to start on a Monday and week 1 is the first week with >3 days.
Examples:
when
CASE WHEN expr1 THEN expr2 [WHEN expr3 THEN expr4]* [ELSE expr5] END - When expr1
= true, returns expr2
; when expr3
= true, return expr4
; else return expr5
.
window
xpath
xpath(xml, xpath) - Returns a string array of values within the nodes of xml that match the XPath expression.
Examples:
xpath_boolean
xpath_boolean(xml, xpath) - Returns true if the XPath expression evaluates to true, or if a matching node is found.
Examples:
xpath_double
xpath_double(xml, xpath) - Returns a double value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric.
Examples:
xpath_float
xpath_float(xml, xpath) - Returns a float value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric.
Examples:
xpath_int
xpath_int(xml, xpath) - Returns an integer value, or the value zero if no match is found, or a match is found but the value is non-numeric.
Examples:
xpath_long
xpath_long(xml, xpath) - Returns a long integer value, or the value zero if no match is found, or a match is found but the value is non-numeric.
Examples:
xpath_number
xpath_number(xml, xpath) - Returns a double value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric.
Examples:
xpath_short
xpath_short(xml, xpath) - Returns a short integer value, or the value zero if no match is found, or a match is found but the value is non-numeric.
Examples:
xpath_string
xpath_string(xml, xpath) - Returns the text contents of the first xml node that matches the XPath expression.
Examples:
year
year(date) - Returns the year component of the date/timestamp.
Examples:
|
expr1 | expr2 - Returns the result of bitwise OR of expr1
and expr2
.
Examples:
~
~ expr - Returns the result of bitwise NOT of expr
.
Examples:
原文:https://spark-test.github.io/sparksqldoc/