From a2b63fd7731a91d8045a924b834690b53669ac0e Mon Sep 17 00:00:00 2001 From: Sergey Chernov Date: Thu, 29 Oct 2026 13:44:28 -0700 Subject: [PATCH 1/2] Documented usage of Spring JDBC Template --- .gitignore | 1 + docs/clickhouse-docs/spring-io-integration.md | 150 ++++++++++++++++++ .../examples/SignalIngestionTests.java | 39 +++++ 3 files changed, 190 insertions(+) create mode 100644 docs/clickhouse-docs/spring-io-integration.md diff --git a/.gitignore b/.gitignore index b6dd9374c..b5cc91a03 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,4 @@ +.m2 # MacOS .DS_Store diff --git a/docs/clickhouse-docs/spring-io-integration.md b/docs/clickhouse-docs/spring-io-integration.md new file mode 100644 index 000000000..8a7760670 --- /dev/null +++ b/docs/clickhouse-docs/spring-io-integration.md @@ -0,0 +1,150 @@ +--- +sidebarTitle: 'Spring JdbcTemplate' +keywords: ['java', 'jdbc', 'spring', 'spring boot', 'JdbcTemplate', 'RowSet', 'ZonedDateTime'] +description: 'Using the ClickHouse JDBC driver with Spring JdbcTemplate' +slug: /integrations/language-clients/java/spring-jdbc-template +title: 'Using the ClickHouse JDBC driver with Spring JdbcTemplate' +doc_type: 'guide' +integration: + - support_level: 'core' + - category: 'language_client' +--- + +Spring's `JdbcTemplate` works with the ClickHouse JDBC driver like with any other JDBC driver. However, ClickHouse queries +often return large result sets, so the way the result is read matters. This guide describes which `JdbcTemplate` APIs to +prefer, which to avoid for large results, and how to read `DateTime`/`DateTime64` values. + +## Configuration {#configuration} + +With Spring Boot, the driver is configured through the regular `spring.datasource` properties. Driver-specific properties +are passed with `data-source-properties` of the connection pool: + +```yaml +spring: + datasource: + url: jdbc:clickhouse://localhost:8123/default + username: default + password: + driver-class-name: com.clickhouse.jdbc.ClickHouseDriver + hikari: + data-source-properties: + # driver properties go here, for example: + jdbc_type_mappings: DateTime64=java.time.ZonedDateTime,DateTime=java.time.ZonedDateTime +``` + +## Reading results {#reading-results} + +`JdbcTemplate` has three groups of read APIs. They differ in how much of the result is kept in memory. + +| API | Example | Memory | +| --- | --- | --- | +| Callback (`RowCallbackHandler`, `ResultSetExtractor`) | `query(sql, handler, args...)` | One row at a time. The driver streams the result. | +| List (`RowMapper`) | `query(sql, rowMapper, args...)`, `queryForList(...)` | All mapped objects. | +| Row set (`SqlRowSet`) | `queryForRowSet(sql, args...)` | All rows, copied into a `CachedRowSet`. | + +### Callback API (recommended for large results) {#callback-api} + +A `RowCallbackHandler` is called for every row while the driver reads the response from the server. Nothing is +accumulated, so memory usage does not depend on the number of rows. This is the preferred way to export, aggregate or +forward large result sets: + +```java +jdbcTemplate.query(""" + SELECT device_id, value, event_time + FROM iot_signals + WHERE location_id = ? + ORDER BY event_time + """, + (ResultSet rs) -> { + String deviceId = rs.getString("device_id"); + double value = rs.getDouble("value"); + ZonedDateTime eventTime = rs.getObject("event_time", ZonedDateTime.class); + // process the row + }, + locationId); +``` + +Use `ResultSetExtractor` when the whole result should be reduced to a single value (for example, a summary object), and +the `ResultSet` should be iterated manually. + +### List API (recommended for small and medium results) {#list-api} + +A `RowMapper` maps each row to an object and `JdbcTemplate` returns a `List` of them. It is the most convenient API, and +is fine when the result fits in memory comfortably (for example, a page of rows or an aggregated report): + +```java +record Signal(String deviceId, double value, ZonedDateTime eventTime) {} + +List signals = jdbcTemplate.query( + "SELECT device_id, value, event_time FROM iot_signals WHERE location_id = ? LIMIT 1000", + (rs, rowNum) -> new Signal( + rs.getString("device_id"), + rs.getDouble("value"), + rs.getObject("event_time", ZonedDateTime.class)), + locationId); +``` + +Use `LIMIT` or aggregation on the server side to keep the list size bounded. + +### Row set API {#row-set-api} + +`queryForRowSet()` returns a `SqlRowSet` that is backed by a `javax.sql.rowset.CachedRowSet`. The row set reads the +**whole** result into memory before the first row is returned, and every value is stored as a Java object. For large +results this means high memory usage and possible `OutOfMemoryError`. Prefer the [callback API](#callback-api) for such +cases. + +The row set API is still useful when a disconnected, scrollable result is needed (for example, to pass small results +between layers). In this case, note how timestamps are cached. + +#### Reading `DateTime` values from a row set {#row-set-date-time} + +`CachedRowSet` fills itself by calling `ResultSet#getObject(int)` for every column, so it stores whatever class the driver +maps the column to by default. For `DateTime` and `DateTime64` it is `java.sql.Timestamp`. Additionally, the JDK +`CachedRowSet` implementation doesn't support `getObject(column, Class)`: calling `getObject("event_time", ZonedDateTime.class)` +on a `CachedRowSet` (or on `SqlRowSet`) fails with `Not supported yet`. + +To get `java.time.ZonedDateTime` values from a row set, configure the driver to map timestamp types to `ZonedDateTime` +with the `jdbc_type_mappings` property: + +```yaml +spring: + datasource: + hikari: + data-source-properties: + jdbc_type_mappings: DateTime64=java.time.ZonedDateTime,DateTime=java.time.ZonedDateTime +``` + +Then the row set caches `ZonedDateTime` objects that can be read with `getObject(column)`: + +```java +SqlRowSet rows = jdbcTemplate.queryForRowSet( + "SELECT device_id, event_time FROM iot_signals WHERE location_id = ? LIMIT 100", locationId); +while (rows.next()) { + ZonedDateTime eventTime = (ZonedDateTime) rows.getObject("event_time"); + // ... +} +``` + +`ZonedDateTime` is more portable and convenient than `Timestamp`: it keeps the time zone of the value, it can be +converted to any other zone or to `Instant`/`LocalDateTime` explicitly, and its `toString()` doesn't depend on the JVM +default time zone. + + +`jdbc_type_mappings` affects every `getObject(column)` call on the connection, not only row sets. +With this mapping, `getTimestamp()` on a row set fails with `ClassCastException` because the cached value is a +`ZonedDateTime`. Read such columns with `getObject(column)` only. + + +## Avoid `java.sql.Timestamp` {#avoid-timestamp} + + +`java.sql.Timestamp` is a complicated class. It extends `java.util.Date` but stores nanoseconds separately, it has no +time zone while its `toString()` and `valueOf()` use the JVM default time zone, and its `equals()` is not symmetric +with `java.util.Date`. The same ClickHouse value can look different depending on the JVM time zone, which is a common +source of bugs. + +Prefer `java.time.ZonedDateTime` (`rs.getObject(column, ZonedDateTime.class)`), or `java.time.LocalDateTime` +when the time zone is known to the application. +See the [Date/Time values guide](/integrations/language-clients/java/jdbc_date_time_guide) for how the driver +converts `DateTime`/`DateTime64` values and how time zones affect them. + diff --git a/examples/demo-spring-service/src/test/java/com/clickhouse/examples/SignalIngestionTests.java b/examples/demo-spring-service/src/test/java/com/clickhouse/examples/SignalIngestionTests.java index fc1b15600..617408744 100755 --- a/examples/demo-spring-service/src/test/java/com/clickhouse/examples/SignalIngestionTests.java +++ b/examples/demo-spring-service/src/test/java/com/clickhouse/examples/SignalIngestionTests.java @@ -20,12 +20,15 @@ import org.springframework.jdbc.core.JdbcTemplate; import org.testcontainers.junit.jupiter.Testcontainers; +import java.sql.ResultSet; import java.time.Duration; import java.time.Instant; +import java.time.ZonedDateTime; import java.time.temporal.ChronoUnit; import java.util.Map; import java.util.UUID; import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; import static org.assertj.core.api.Assertions.assertThat; @@ -152,6 +155,42 @@ SELECT max(value) assertThat(durationSamples).isNotNull().isGreaterThanOrEqualTo(1.0); } + @Test + void selectDataViaRowSet() { + int eventCount = 100; + UUID locationId = UUID.randomUUID(); + Instant start = Instant.now().minusSeconds(60).truncatedTo(ChronoUnit.MILLIS); + + for (int i = 0; i < eventCount; i++) { + boolean temperature = i % 2 == 0; + postSignal( + locationId, + "rowset-device-" + (i % 5), + temperature ? "TEMPERATURE" : "HUMIDITY", + i, + temperature ? "C" : "%", + start.plusMillis(i * 10L)); + } + + AtomicInteger read = new AtomicInteger(); + jdbc.query(""" + SELECT device_id, signal_type, value, unit, event_time + FROM iot_signals + WHERE location_id = ? + ORDER BY event_time + """, new Object[] {locationId}, (ResultSet rs) -> { + int i = read.getAndIncrement(); + boolean temperature = i % 2 == 0; + assertThat(rs.getString("device_id")).isEqualTo("rowset-device-" + (i % 5)); + assertThat(rs.getString("signal_type")).isEqualTo(temperature ? "TEMPERATURE" : "HUMIDITY"); + assertThat(rs.getDouble("value")).isEqualTo(i); + assertThat(rs.getString("unit")).isEqualTo(temperature ? "C" : "%"); + assertThat(rs.getObject("event_time", ZonedDateTime.class).toInstant()) + .isEqualTo(start.plusMillis(i * 10L)); + }); + assertThat(read.get()).isEqualTo(eventCount); + } + @Test void acceptsReconciliationBatchAndStoresItAsynchronously() throws InterruptedException { UUID firstId = UUID.randomUUID(); From 45ab3d694d13379e428879125d8f648ad15bcc1e Mon Sep 17 00:00:00 2001 From: Sergey Chernov Date: Fri, 30 Oct 2026 11:54:15 -0700 Subject: [PATCH 2/2] Added new page and updated the documentation about spring integration --- docs/clickhouse-docs/navigation.json | 3 +- docs/clickhouse-docs/spring-io-integration.md | 173 ++++++------------ 2 files changed, 54 insertions(+), 122 deletions(-) diff --git a/docs/clickhouse-docs/navigation.json b/docs/clickhouse-docs/navigation.json index b8fae20c4..49579c233 100644 --- a/docs/clickhouse-docs/navigation.json +++ b/docs/clickhouse-docs/navigation.json @@ -6,7 +6,8 @@ "integrations/language-clients/java/client", "integrations/language-clients/java/jdbc", "integrations/language-clients/java/r2dbc", - "integrations/language-clients/java/date-time-guide" + "integrations/language-clients/java/date-time-guide", + "integrations/language-clients/java/spring-io-integration" ], "root": "integrations/language-clients/java/index" } diff --git a/docs/clickhouse-docs/spring-io-integration.md b/docs/clickhouse-docs/spring-io-integration.md index 8a7760670..2d7e99005 100644 --- a/docs/clickhouse-docs/spring-io-integration.md +++ b/docs/clickhouse-docs/spring-io-integration.md @@ -1,60 +1,72 @@ --- -sidebarTitle: 'Spring JdbcTemplate' +sidebarTitle: 'Spring integration' keywords: ['java', 'jdbc', 'spring', 'spring boot', 'JdbcTemplate', 'RowSet', 'ZonedDateTime'] -description: 'Using the ClickHouse JDBC driver with Spring JdbcTemplate' +description: 'Integrating Spring applications with ClickHouse' slug: /integrations/language-clients/java/spring-jdbc-template -title: 'Using the ClickHouse JDBC driver with Spring JdbcTemplate' +title: 'Integrating Spring with ClickHouse' doc_type: 'guide' integration: - support_level: 'core' - category: 'language_client' --- -Spring's `JdbcTemplate` works with the ClickHouse JDBC driver like with any other JDBC driver. However, ClickHouse queries -often return large result sets, so the way the result is read matters. This guide describes which `JdbcTemplate` APIs to -prefer, which to avoid for large results, and how to read `DateTime`/`DateTime64` values. +This guide covers different aspects of integrating Spring applications with ClickHouse, including configuration, +data-access APIs, type conversion, and limitations of the Spring abstractions involved. -## Configuration {#configuration} +## `JdbcTemplate` {#jdbc-template} -With Spring Boot, the driver is configured through the regular `spring.datasource` properties. Driver-specific properties -are passed with `data-source-properties` of the connection pool: +`JdbcTemplate` is Spring's abstraction for executing JDBC operations and managing their resources. This section focuses +on its read APIs: callbacks, row mappers, convenience query methods, and row sets. Choosing an API matters because each +one hides different logic for iterating, mapping, buffering, and returning data from the driver's `ResultSet`. -```yaml -spring: - datasource: - url: jdbc:clickhouse://localhost:8123/default - username: default - password: - driver-class-name: com.clickhouse.jdbc.ClickHouseDriver - hikari: - data-source-properties: - # driver properties go here, for example: - jdbc_type_mappings: DateTime64=java.time.ZonedDateTime,DateTime=java.time.ZonedDateTime -``` - -## Reading results {#reading-results} +### Ways to read data {#jdbc-template-reading-data} -`JdbcTemplate` has three groups of read APIs. They differ in how much of the result is kept in memory. +`JdbcTemplate` provides several fetching APIs over the same JDBC driver result: -| API | Example | Memory | +| API | Interaction with the driver result | Specific behavior or limitation | | --- | --- | --- | -| Callback (`RowCallbackHandler`, `ResultSetExtractor`) | `query(sql, handler, args...)` | One row at a time. The driver streams the result. | -| List (`RowMapper`) | `query(sql, rowMapper, args...)`, `queryForList(...)` | All mapped objects. | -| Row set (`SqlRowSet`) | `queryForRowSet(sql, args...)` | All rows, copied into a `CachedRowSet`. | +| `RowCallbackHandler` | Receives the live `ResultSet` at the current row | Spring advances the cursor; the callback does not return a value | +| `ResultSetExtractor` | Receives the complete live `ResultSet` | The extractor controls cursor iteration and constructs the return value | +| `RowMapper` | Receives the live `ResultSet` once per row | Spring collects every mapped object into the returned `List` | +| `queryForObject`, `queryForList`, `queryForMap` | Use Spring's built-in result mapping | The application does not choose which JDBC getters perform the conversion | +| `queryForRowSet` (`SqlRowSet`) | Copies values from the driver result into a `CachedRowSet` | Reads are disconnected from the driver and use the cached Java objects | + +Prefer these `query` overloads because they operate on the driver's live `ResultSet` without first copying it into a row +set: -### Callback API (recommended for large results) {#callback-api} +- `RowCallbackHandler` for efficient, independent row-by-row processing. +- `ResultSetExtractor` for full control over iteration and result construction. +- `RowMapper` when returning a list is intentional. Direct access to the JDBC result makes type conversion more + predictable and portable than relying on a particular `CachedRowSet` implementation. -A `RowCallbackHandler` is called for every row while the driver reads the response from the server. Nothing is -accumulated, so memory usage does not depend on the number of rows. This is the preferred way to export, aggregate or -forward large result sets: + +**Timestamp handling with `queryForRowSet()`** + +`queryForRowSet()` is different from the other `queryFor*` methods: it reads the entire driver `ResultSet` into a +disconnected `CachedRowSet` before returning. The `SqlRowSet` returned by Spring does not support +`getObject(column, Class)`, so a call such as `getObject("event_time", ZonedDateTime.class)` fails with +`Not supported yet`. + +For `DateTime` and `DateTime64`, configure +`jdbc_type_mappings=DateTime64=java.time.ZonedDateTime,DateTime=java.time.ZonedDateTime` in the connection pool's +`data-source-properties`. The cached value can then be read with the untyped +`(ZonedDateTime) rowSet.getObject("event_time")` call. This mapping applies to every `getObject(column)` call on the +connection, not only to row sets. See the +[Date/Time values guide](/integrations/language-clients/java/jdbc_date_time_guide) for details. + + +The following example uses the recommended callback API. `JdbcTemplate` advances the cursor and invokes the callback +once for each row; the callback should not call `next()`: ```java -jdbcTemplate.query(""" - SELECT device_id, value, event_time - FROM iot_signals - WHERE location_id = ? - ORDER BY event_time - """, +public void processSignals(JdbcTemplate jdbcTemplate, long locationId) { + String sql = "SELECT device_id, value, event_time " + + "FROM iot_signals " + + "WHERE location_id = ? " + + "ORDER BY event_time"; + + jdbcTemplate.query( + sql, (ResultSet rs) -> { String deviceId = rs.getString("device_id"); double value = rs.getDouble("value"); @@ -62,89 +74,8 @@ jdbcTemplate.query(""" // process the row }, locationId); -``` - -Use `ResultSetExtractor` when the whole result should be reduced to a single value (for example, a summary object), and -the `ResultSet` should be iterated manually. - -### List API (recommended for small and medium results) {#list-api} - -A `RowMapper` maps each row to an object and `JdbcTemplate` returns a `List` of them. It is the most convenient API, and -is fine when the result fits in memory comfortably (for example, a page of rows or an aggregated report): - -```java -record Signal(String deviceId, double value, ZonedDateTime eventTime) {} - -List signals = jdbcTemplate.query( - "SELECT device_id, value, event_time FROM iot_signals WHERE location_id = ? LIMIT 1000", - (rs, rowNum) -> new Signal( - rs.getString("device_id"), - rs.getDouble("value"), - rs.getObject("event_time", ZonedDateTime.class)), - locationId); -``` - -Use `LIMIT` or aggregation on the server side to keep the list size bounded. - -### Row set API {#row-set-api} - -`queryForRowSet()` returns a `SqlRowSet` that is backed by a `javax.sql.rowset.CachedRowSet`. The row set reads the -**whole** result into memory before the first row is returned, and every value is stored as a Java object. For large -results this means high memory usage and possible `OutOfMemoryError`. Prefer the [callback API](#callback-api) for such -cases. - -The row set API is still useful when a disconnected, scrollable result is needed (for example, to pass small results -between layers). In this case, note how timestamps are cached. - -#### Reading `DateTime` values from a row set {#row-set-date-time} - -`CachedRowSet` fills itself by calling `ResultSet#getObject(int)` for every column, so it stores whatever class the driver -maps the column to by default. For `DateTime` and `DateTime64` it is `java.sql.Timestamp`. Additionally, the JDK -`CachedRowSet` implementation doesn't support `getObject(column, Class)`: calling `getObject("event_time", ZonedDateTime.class)` -on a `CachedRowSet` (or on `SqlRowSet`) fails with `Not supported yet`. - -To get `java.time.ZonedDateTime` values from a row set, configure the driver to map timestamp types to `ZonedDateTime` -with the `jdbc_type_mappings` property: - -```yaml -spring: - datasource: - hikari: - data-source-properties: - jdbc_type_mappings: DateTime64=java.time.ZonedDateTime,DateTime=java.time.ZonedDateTime -``` - -Then the row set caches `ZonedDateTime` objects that can be read with `getObject(column)`: - -```java -SqlRowSet rows = jdbcTemplate.queryForRowSet( - "SELECT device_id, event_time FROM iot_signals WHERE location_id = ? LIMIT 100", locationId); -while (rows.next()) { - ZonedDateTime eventTime = (ZonedDateTime) rows.getObject("event_time"); - // ... } ``` -`ZonedDateTime` is more portable and convenient than `Timestamp`: it keeps the time zone of the value, it can be -converted to any other zone or to `Instant`/`LocalDateTime` explicitly, and its `toString()` doesn't depend on the JVM -default time zone. - - -`jdbc_type_mappings` affects every `getObject(column)` call on the connection, not only row sets. -With this mapping, `getTimestamp()` on a row set fails with `ClassCastException` because the cached value is a -`ZonedDateTime`. Read such columns with `getObject(column)` only. - - -## Avoid `java.sql.Timestamp` {#avoid-timestamp} - - -`java.sql.Timestamp` is a complicated class. It extends `java.util.Date` but stores nanoseconds separately, it has no -time zone while its `toString()` and `valueOf()` use the JVM default time zone, and its `equals()` is not symmetric -with `java.util.Date`. The same ClickHouse value can look different depending on the JVM time zone, which is a common -source of bugs. - -Prefer `java.time.ZonedDateTime` (`rs.getObject(column, ZonedDateTime.class)`), or `java.time.LocalDateTime` -when the time zone is known to the application. -See the [Date/Time values guide](/integrations/language-clients/java/jdbc_date_time_guide) for how the driver -converts `DateTime`/`DateTime64` values and how time zones affect them. - +For the complete API and callback contracts, see the +[Spring JDBC documentation](https://docs.spring.io/spring-framework/reference/data-access/jdbc/core.html).