Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,7 @@ currency_time currency rate
<span class="label label-danger">注意</span> 理论上讲任意都能用作时态表并在基于处理时间的时态表 Join 中使用,但当前支持作为时态表的普通表必须实现接口 `LookupableTableSource`。接口 `LookupableTableSource` 的实例只能作为时态表用于基于处理时间的时态 Join 。

通过 `LookupableTableSource` 定义的表意味着该表具备了在运行时通过一个或多个 key 去查询外部存储系统的能力,当前支持在 基于处理时间的时态表 join 中使用的表包括
[JDBC]({{< ref "docs/connectors/table/jdbc" >}}), [HBase]({{< ref "docs/connectors/table/hbase" >}}) 和 [Hive]({{< ref "docs/connectors/table/hive/hive_read_write" >}}#temporal-table-join)
[JDBC]({{< ref "docs/connectors/table/jdbc" >}}), [HBase]({{< ref "docs/connectors/table/hbase" >}}) 和 Hive。

另请参阅 [LookupableTableSource]({{< ref "docs/dev/table/sourcesSinks" >}}#lookup-table-source)页面了解更多信息。

Expand Down
6 changes: 0 additions & 6 deletions docs/content.zh/docs/connectors/table/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,12 +102,6 @@ Flink natively support various connectors. The following tables list all availab
<td>Bounded Scan, Lookup</td>
<td>Streaming Sink, Batch Sink</td>
</tr>
<tr>
<td><a href="{{< ref "docs/connectors/table/hive/overview" >}}">Apache Hive</a></td>
<td><a href="{{< ref "docs/connectors/table/hive/overview" >}}#supported-hive-versions">Supported Versions</a></td>
<td>Unbounded Scan, Bounded Scan, Lookup</td>
<td>Streaming Sink, Batch Sink</td>
</tr>
<tr>
<td><a href="{{< ref "docs/connectors/table/mongodb" >}}">MongoDB</a></td>
<td>3.6.x & 4.x & 5.x & 6.0.x</td>
Expand Down
2 changes: 1 addition & 1 deletion docs/content.zh/docs/dev/table/modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ or influence how connectors of temporary tables should be created without a corr
### HiveModule

The `HiveModule` provides Hive built-in functions as Flink's system functions to SQL and Table API users.
Flink's [Hive documentation]({{< ref "docs/connectors/table/hive/hive_functions" >}}) provides full details on setting up the module.
Flink's Hive documentation provides full details on setting up the module.

### User-Defined Module

Expand Down
2 changes: 1 addition & 1 deletion docs/content.zh/docs/sql/catalogs.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Catalog 提供了元数据信息,例如数据库、表、分区、视图以及
### HiveCatalog

`HiveCatalog` 有两个用途:作为原生 Flink 元数据的持久化存储,以及作为读写现有 Hive 元数据的接口。
Flink 的 [Hive 文档]({{< ref "docs/connectors/table/hive/overview" >}}) 提供了有关设置 `HiveCatalog` 以及访问现有 Hive 元数据的详细信息。
Flink 的 Hive 文档 提供了有关设置 `HiveCatalog` 以及访问现有 Hive 元数据的详细信息。


<span class="label label-danger">警告</span> Hive Metastore 以小写形式存储所有元数据对象名称。而 `GenericInMemoryCatalog` 区分大小写。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,14 @@ Flink 目前支持两种 SQL 方言: `default` 和 `hive`。你需要先切换
{{< hint warning >}}
**Note:**

- 为了使用 Hive 方言, 你必须首先添加和 Hive 相关的依赖. 请参考 [Hive dependencies]({{< ref "docs/connectors/table/hive/overview" >}}#dependencies) 如何添加这些依赖。
- 请确保当前的 Catalog 是 [HiveCatalog]({{< ref "docs/connectors/table/hive/hive_catalog" >}}). 否则, 将使用 Flink 的默认方言。
- 为了使用 Hive 方言, 你必须首先添加和 Hive 相关的依赖. 请参考 Hive dependencies 如何添加这些依赖。
- 请确保当前的 Catalog 是 HiveCatalog. 否则, 将使用 Flink 的默认方言。
在启动了 [HiveServer2 Endpoint]({{< ref "docs/sql/hive-compatibility/hiveserver2" >}}) 的 SQL Gateway 下,默认当前的 Catalog 就是 HiveCatalog。
- 为了实现更好的语法和语义的兼容,强烈建议首先加载 [HiveModule]({{< ref "docs/connectors/table/hive/hive_functions" >}}#use-hive-built-in-functions-via-hivemodule) 并将其放在 Module 列表的首位,以便在函数解析时优先使用 Hive 内置函数。
- 为了实现更好的语法和语义的兼容,强烈建议首先加载 HiveModule 并将其放在 Module 列表的首位,以便在函数解析时优先使用 Hive 内置函数。
请参考文档 [here]({{< ref "docs/dev/table/modules" >}}#how-to-load-unload-use-and-list-modules) 来将 HiveModule 放在 Module 列表的首。
在启动了 HiveServer2 endpoint 的 SQL Gateway,HiveModule 已经被加载进来了。
- Hive 方言只支持 `db.table` 这种两级的标识符,不支持带有 Catalog 名字的标识符。
- 虽然所有 Hive 版本支持相同的语法,但是一些特定的功能是否可用仍取决于你使用的 [Hive 版本]({{< ref "docs/connectors/table/hive/overview" >}}#支持的hive版本)。例如,更新数据库位置
- 虽然所有 Hive 版本支持相同的语法,但是一些特定的功能是否可用仍取决于你使用的 Hive 版本。例如,更新数据库位置
只在 Hive-2.4.0 或更高版本支持。
- Hive 方言主要是在批模式下使用的,某些 Hive 的语法([Sort/Cluster/Distributed BY]({{< ref "docs/sql/hive-compatibility/hive-dialect/queries/sort-cluster-distribute-by" >}}), [Transform]({{< ref "docs/sql/hive-compatibility/hive-dialect/queries/transform" >}}), 等)还没有在流模式下支持。
{{< /hint >}}
Expand Down
6 changes: 3 additions & 3 deletions docs/content.zh/docs/sql/hive-compatibility/hiveserver2.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ wire protocol and allows users to interact (e.g. submit Hive SQL) with Flink SQL

Setting Up
----------------
Before the trip of the SQL Gateway with the HiveServer2 Endpoint, please prepare the required [dependencies]({{< ref "docs/connectors/table/hive/overview#dependencies" >}}).
Before the trip of the SQL Gateway with the HiveServer2 Endpoint, please prepare the required dependencies.

### Configure HiveServer2 Endpoint

Expand Down Expand Up @@ -206,7 +206,7 @@ HiveServer2 Protocol Compatibility

The Flink SQL Gateway with HiveServer2 Endpoint aims to provide the same experience compared to the HiveServer2 of Apache Hive.
Therefore, HiveServer2 Endpoint automatically initialize the environment to have more consistent experience for Hive users:
- create the [Hive Catalog]({{< ref "docs/connectors/table/hive/hive_catalog.md" >}}) as the default catalog;
- create the Hive Catalog as the default catalog;
- use Hive built-in function by loading Hive function module and place it first in the [function module]({{< ref "docs/dev/table/modules/index.md" >}}) list;
- switch to the Hive dialect (`table.sql-dialect = hive`);
- switch to batch execution mode (`execution.runtime-mode = BATCH`);
Expand Down Expand Up @@ -313,4 +313,4 @@ Supported Types

The HiveServer2 Endpoint is built on the Hive2 now and supports all Hive2 available types. For Hive-compatible tables, the HiveServer2 Endpoint
obeys the same rule as the HiveCatalog to convert the Flink types to Hive Types and serialize them to the thrift object. Please refer to
the [HiveCatalog]({{< ref "docs/connectors/table/hive/hive_catalog#supported-types" >}}) for the type mappings.
the HiveCatalog for the type mappings.
2 changes: 1 addition & 1 deletion docs/content.zh/docs/sql/reference/utility/jar.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ ADD JAR '<path_to_filename>.jar'

### 限制

请不要通过 `ADD JAR` 语句来加载 Hive 的source、sink、function、catalog。这是 Hive connector 的一个已知限制,且会在将来版本中修复。当前,建议跟随这个指南来[安装 Hive 的集成]({{< ref "docs/connectors/table/hive/overview" >}}#dependencies)
请不要通过 `ADD JAR` 语句来加载 Hive 的source、sink、function、catalog。这是 Hive connector 的一个已知限制,且会在将来版本中修复。当前,建议跟随这个指南来安装 Hive 的集成。

<a name="show-jars"></a>

Expand Down
6 changes: 0 additions & 6 deletions docs/content/docs/connectors/table/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,12 +102,6 @@ Flink natively support various connectors. The following tables list all availab
<td>Bounded Scan, Lookup</td>
<td>Streaming Sink, Batch Sink</td>
</tr>
<tr>
<td><a href="{{< ref "docs/connectors/table/hive/overview" >}}">Apache Hive</a></td>
<td><a href="{{< ref "docs/connectors/table/hive/overview" >}}#supported-hive-versions">Supported Versions</a></td>
<td>Unbounded Scan, Bounded Scan, Lookup</td>
<td>Streaming Sink, Batch Sink</td>
</tr>
<tr>
<td><a href="{{< ref "docs/connectors/table/mongodb" >}}">MongoDB</a></td>
<td>3.6.x & 4.x & 5.x & 6.x & 7.0.x </td>
Expand Down
2 changes: 1 addition & 1 deletion docs/content/docs/dev/table/modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ or influence how connectors of temporary tables should be created without a corr
### HiveModule

The `HiveModule` provides Hive built-in functions as Flink's system functions to SQL and Table API users.
Flink's [Hive documentation]({{< ref "docs/connectors/table/hive/hive_functions" >}}) provides full details on setting up the module.
Flink's Hive documentation provides full details on setting up the module.

### User-Defined Module

Expand Down
2 changes: 1 addition & 1 deletion docs/content/docs/sql/catalogs.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ See [JdbcCatalog documentation]({{< ref "docs/connectors/table/jdbc" >}}) for mo
### HiveCatalog

The `HiveCatalog` serves two purposes; as persistent storage for pure Flink metadata, and as an interface for reading and writing existing Hive metadata.
Flink's [Hive documentation]({{< ref "docs/connectors/table/hive/overview" >}}) provides full details on setting up the catalog and interfacing with an existing Hive installation.
Flink's Hive documentation provides full details on setting up the catalog and interfacing with an existing Hive installation.


{{< hint warning >}} The Hive Metastore stores all meta-object names in lower case. This is unlike `GenericInMemoryCatalog` which is case-sensitive
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,16 @@ statement you execute. There's no need to restart a session to use a different d
{{< hint warning >}}
**Note:**

- To use Hive dialect, you have to add dependencies related to Hive. Please refer to [Hive dependencies]({{< ref "docs/connectors/table/hive/overview" >}}#dependencies) for how to add the dependencies.
- Please make sure the current catalog is [HiveCatalog]({{< ref "docs/connectors/table/hive/hive_catalog" >}}). Otherwise, it will fall back to Flink's `default` dialect.
- To use Hive dialect, you have to add dependencies related to Hive. Please refer to Hive dependencies for how to add the dependencies.
- Please make sure the current catalog is HiveCatalog. Otherwise, it will fall back to Flink's `default` dialect.
When using SQL Gateway configured with [HiveServer2 Endpoint]({{< ref "docs/sql/interfaces/sql-gateway/hiveserver2" >}}), the current catalog will be a HiveCatalog by default.
- In order to have better syntax and semantic compatibility, it’s highly recommended to load [HiveModule]({{< ref "docs/connectors/table/hive/hive_functions" >}}#use-hive-built-in-functions-via-hivemodule) and
- In order to have better syntax and semantic compatibility, it’s highly recommended to load HiveModule and
place it first in the module list, so that Hive built-in functions can be picked up during function resolution.
Please refer [here]({{< ref "docs/dev/table/modules" >}}#how-to-load-unload-use-and-list-modules) for how to change resolution order.
But when using SQL Gateway configured with HiveServer2 Endpoint, the Hive module will be loaded automatically.
- Hive dialect only supports 2-part identifiers, so you can't specify catalog for an identifier.
- While all Hive versions support the same syntax, whether a specific feature is available still depends on the
[Hive version]({{< ref "docs/connectors/table/hive/overview" >}}#supported-hive-versions) you use. For example, updating database
Hive version you use. For example, updating database
location is only supported in Hive-2.4.0 or later.
- The Hive dialect is mainly used in batch mode. Some Hive's syntax ([Sort/Cluster/Distributed BY]({{< ref "docs/sql/hive-compatibility/hive-dialect/queries/sort-cluster-distribute-by" >}}), [Transform]({{< ref "docs/sql/hive-compatibility/hive-dialect/queries/transform" >}}), etc.) haven't been supported in streaming mode yet.
{{< /hint >}}
Expand Down
6 changes: 3 additions & 3 deletions docs/content/docs/sql/interfaces/sql-gateway/hiveserver2.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ It is recommended to use the HiveServer2 Endpoint with a Hive Catalog and Hive d

Setting Up
----------------
Before the trip of the SQL Gateway with the HiveServer2 Endpoint, please prepare the required [dependencies]({{< ref "docs/connectors/table/hive/overview#dependencies" >}}).
Before the trip of the SQL Gateway with the HiveServer2 Endpoint, please prepare the required dependencies.

### Configure HiveServer2 Endpoint

Expand Down Expand Up @@ -209,7 +209,7 @@ HiveServer2 Protocol Compatibility

The Flink SQL Gateway with HiveServer2 Endpoint aims to provide the same experience compared to the HiveServer2 of Apache Hive.
Therefore, HiveServer2 Endpoint automatically initialize the environment to have more consistent experience for Hive users:
- create the [Hive Catalog]({{< ref "docs/connectors/table/hive/hive_catalog.md" >}}) as the default catalog;
- create the Hive Catalog as the default catalog;
- use Hive built-in function by loading Hive function module and place it first in the [function module]({{< ref "docs/dev/table/modules/index.md" >}}) list;
- switch to the Hive dialect (`table.sql-dialect = hive`);
- switch to batch execution mode (`execution.runtime-mode = BATCH`);
Expand Down Expand Up @@ -316,4 +316,4 @@ Supported Types

The HiveServer2 Endpoint is built on the Hive2 now and supports all Hive2 available types. For Hive-compatible tables, the HiveServer2 Endpoint
obeys the same rule as the HiveCatalog to convert the Flink types to Hive Types and serialize them to the thrift object. Please refer to
the [HiveCatalog]({{< ref "docs/connectors/table/hive/hive_catalog#supported-types" >}}) for the type mappings.
the HiveCatalog for the type mappings.
2 changes: 1 addition & 1 deletion docs/content/docs/sql/reference/utility/jar.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ ADD JAR '<path_to_filename>.jar'
Add a JAR file to the list of resources, it supports adding the jar locates in a local or remote [file system]({{< ref "docs/deployment/filesystems/overview" >}}). The added JAR file can be listed using [`SHOW JARS`](#show-jars) statements.

### Limitation
Please don't use `ADD JAR` statements to load Hive source/sink/function/catalog. This is a known limitation of Hive connector and will be fixed in the future version. Currently, it's recommended to follow this [instruction]({{< ref "docs/connectors/table/hive/overview" >}}#dependencies) to setup Hive integration.
Please don't use `ADD JAR` statements to load Hive source/sink/function/catalog. This is a known limitation of Hive connector and will be fixed in the future version. Currently, it's recommended to follow this instruction to setup Hive integration.

## SHOW JARS

Expand Down