Apache DataFusion SQL Query Engine
docs: Document missing SQL data types (#24471)
Nuno Faria733b195726e0
diff --git a/docs/source/user-guide/sql/data_types.md b/docs/source/user-guide/sql/data_types.mdindex 502193df41..2e75fce6c7 100644--- a/docs/source/user-guide/sql/data_types.md+++ b/docs/source/user-guide/sql/data_types.md@@ -21,7 +21,7 @@DataFusion uses Arrow, and thus the Arrow type system, for queryexecution. The SQL types from-[sqlparser-rs](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/src/ast/data_type.rs#L27)+[sqlparser-rs](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/src/ast/data_type.rs#L49)are mapped to [Arrow data types](https://docs.rs/arrow/latest/arrow/datatypes/enum.DataType.html) according to the following table.This mapping occurs when defining the schema in a `CREATE EXTERNAL TABLE` command or when performing a SQL `CAST` operation.@@ -67,32 +67,39 @@ select arrow_cast(now(), 'Timestamp(Second, None)') as "now()";## Character Types-| SQL DataType | Arrow DataType |-| ------------ | -------------- |-| `CHAR` | `Utf8View` |-| `VARCHAR` | `Utf8View` |-| `TEXT` | `Utf8View` |-| `STRING` | `Utf8View` |+| SQL DataType | Arrow DataType |+| ------------------- | -------------- |+| `CHAR` | `Utf8View` |+| `VARCHAR` | `Utf8View` |+| `TEXT` | `Utf8View` |+| `STRING` | `Utf8View` |+| `CHARACTER VARYING` | `Utf8View` |+| `CHAR VARYING` | `Utf8View` |+| `NVARCHAR` | `Utf8View` |By default, string types are mapped to `Utf8View`. This can be configured using the `datafusion.sql_parser.map_string_types_to_utf8view` setting. When set to `false`, string types are mapped to `Utf8` instead.+Note that `CHAR`, `VARCHAR`, `STRING`, `CHARACTER VARYING`, `CHAR VARYING`, and `NVARCHAR` can+optionally receive the maximum length (e.g., `VARCHAR(n)`). However, this limit is ignored. To raise+an error when the length is passed, set `datafusion.sql_parser.support_varchar_with_length` to+`false`.+## Numeric Types-| SQL DataType | Arrow DataType |-| ------------------------------------------------ | :----------------------------- |-| `TINYINT` | `Int8` |-| `SMALLINT` | `Int16` |-| `INT` or `INTEGER` | `Int32` |-| `BIGINT` | `Int64` |-| `TINYINT UNSIGNED` | `UInt8` |-| `SMALLINT UNSIGNED` | `UInt16` |-| `INT UNSIGNED` or `INTEGER UNSIGNED` | `UInt32` |-| `BIGINT UNSIGNED` | `UInt64` |-| `FLOAT` | `Float32` |-| `REAL` | `Float32` |-| `DOUBLE` | `Float64` |-| `DECIMAL(precision, scale)` where precision ≤ 38 | `Decimal128(precision, scale)` |-| `DECIMAL(precision, scale)` where precision > 38 | `Decimal256(precision, scale)` |+| SQL DataType | Arrow DataType |+| ----------------------------------------------------------------------------- | :----------------------------- |+| `TINYINT` | `Int8` |+| `SMALLINT` or `INT2` | `Int16` |+| `INT` or `INTEGER` or `INT4` | `Int32` |+| `BIGINT` or `INT8` | `Int64` |+| `TINYINT UNSIGNED` or `UTINYINT` | `UInt8` |+| `SMALLINT UNSIGNED` or `INT2 UNSIGNED` or `USMALLINT` | `UInt16` |+| `INT UNSIGNED` or `INTEGER UNSIGNED` or `INT4 UNSIGNED` | `UInt32` |+| `BIGINT UNSIGNED` or `INT8 UNSIGNED` or `UBIGINT` | `UInt64` |+| `FLOAT` or `FLOAT4` or `REAL` | `Float32` |+| `DOUBLE` or `DOUBLE PRECISION` or `FLOAT8` | `Float64` |+| `DECIMAL(precision, scale) or NUMERIC(precision, scale)` where precision ≤ 38 | `Decimal128(precision, scale)` |+| `DECIMAL(precision, scale) or NUMERIC(precision, scale)` where precision > 38 | `Decimal256(precision, scale)` |The maximum supported precision for `DECIMAL` types is 76.@@ -107,32 +114,46 @@ The maximum supported precision for `DECIMAL` types is 76.## Boolean Types-| SQL DataType | Arrow DataType |-| ------------ | :------------- |-| `BOOLEAN` | `Boolean` |+| SQL DataType | Arrow DataType |+| ------------------- | :------------- |+| `BOOLEAN` or `BOOL` | `Boolean` |## Binary Types| SQL DataType | Arrow DataType || ------------ | :------------- || `BYTEA` | `Binary` |+| `BLOB` | `Binary` |+| `BINARY` | `Binary` |+| `VARBINARY` | `Binary` |+| `BYTES` | `Binary` |You can create binary literals using a hex string literal such as`X'1234'` to create a `Binary` value of two bytes, `0x12` and `0x34`.+## Nested Types++| SQL DataType | Arrow DataType |+| --------------------------------- | :------------------------------ |+| `ARRAY<T>` or `T[]` | `List(T)` |+| `T[n]` | `FixedSizeList(n x T)` |+| `MAP(k, v)` | `Map(k, v)` |+| `STRUCT<[c0] T0[, ..., [cn] Tn]>` | `Struct(c0: T0[, ..., cn: Tn])` |++While columns defined with `STRUCT` require all rows to have the same schema, `MAP` imposes no such+restriction.++Also note that `STRUCT` can be declared without defining the field names. In that case, they will be+set to `c0`, `c1`, ..., `cn`.+## Unsupported SQL Types| SQL Data Type | Arrow DataType || ------------- | :------------------ || `UUID` | _Not yet supported_ |-| `BLOB` | _Not yet supported_ || `CLOB` | _Not yet supported_ |-| `BINARY` | _Not yet supported_ |-| `VARBINARY` | _Not yet supported_ || `REGCLASS` | _Not yet supported_ |-| `NVARCHAR` | _Not yet supported_ || `CUSTOM` | _Not yet supported_ |-| `ARRAY` | _Not yet supported_ || `ENUM` | _Not yet supported_ || `SET` | _Not yet supported_ || `DATETIME` | _Not yet supported_ |