diff --git a/iotdb-thingsboard-table/docs/migration-guide.md b/iotdb-thingsboard-table/docs/migration-guide.md new file mode 100644 index 0000000..344ed91 --- /dev/null +++ b/iotdb-thingsboard-table/docs/migration-guide.md @@ -0,0 +1,392 @@ + + +# Migration Guide: ThingsBoard storage on IoTDB Table Mode + +This guide is for an operator who wants to move a ThingsBoard deployment's +telemetry, latest-value, and (optionally) entity-attribute storage onto Apache +IoTDB 2.x Table Mode using the `iotdb-thingsboard-table` module. It covers +version pairing, step-by-step enablement, the ThingsBoard-to-IoTDB data-model +mapping, coexistence with the existing storage backend, rollback, and the +limitations to plan for. + +For configuration reference and the full limitation list see the module +[`README.md`](../README.md); for day-to-day operation see +[`docs/user-guide.md`](user-guide.md). This guide focuses on the one-time +cut-over. + +## Audience and scope + +The module is a ThingsBoard DAO backend built on IoTDB's table-session API. It +adds three independently activated routes: + +- **historical telemetry** (`TimeseriesDao`) → the `telemetry` table; +- **latest value** (`TimeseriesLatestDao`) → derived from `telemetry` plus a + per-key `telemetry_latest` overlay table; +- **entity attributes** (`AttributesDao`, opt-in) → the `entity_attributes` + table. + +**The module is additive and default-inert.** Putting the jar on the ThingsBoard +classpath changes nothing on its own: every route is gated behind an explicit +activation selector, and with no selector set the activation conditions all +return `false`, no IoTDB bean or session pool is created, and ThingsBoard keeps +using its existing storage backend. Enabling a route is a deliberate +configuration step, described below. + +The timeseries route implements time-bucketed aggregation over the IoTDB +`telemetry` table — both the fixed-width millisecond `date_bin` path and the +timezone-aware calendar buckets (`WEEK` / `WEEK_ISO` / `MONTH` / `QUARTER`) — in +addition to raw read, write, and delete. Aggregation is therefore **not** a +reason to keep the host backend for the telemetry route. + +## Version pairing + +Pin these versions so the module, the IoTDB client, and the IoTDB server line +up. The module overrides **no** shared reactor versions; the IoTDB client and +TsFile versions are inherited from the `iotdb-extras` parent reactor. + +| Component | Coordinate / image | Version | Where it is fixed | +| --- | --- | --- | --- | +| Module | `org.apache.iotdb:iotdb-thingsboard-table` | `2.0.4-SNAPSHOT` (parent version) | module `pom.xml` `` | +| Parent reactor | `org.apache.iotdb:iotdb-extras-parent` | `2.0.4-SNAPSHOT` | module `pom.xml` `` | +| IoTDB session client | `org.apache.iotdb:iotdb-session` | `2.0.5` | inherited from parent `iotdb.version`; module declares the dependency with no `` | +| TsFile | `org.apache.tsfile:tsfile` | `2.1.1` | inherited from parent `tsfile.version` (transitive of `iotdb-session`) | +| Guava | `com.google.guava:guava` | `32.1.2-jre` | inherited from parent `guava.version`; `provided` scope | +| Bean-validation API | `jakarta.validation:jakarta.validation-api` | `3.0.2` | module override (`jakarta.* ` namespace, `provided` scope) to match the Spring Boot 3 runtime host | +| ThingsBoard host | (ThingsBoard distribution) | `4.3.1.2` | module `pom.xml` `thingsboard.version`; SPI surface verified against this tag | +| IoTDB server (tested) | `apache/iotdb` standalone image | `2.0.8` | integration tests run against `apache/iotdb:2.0.8-standalone` | + +Notes: + +- The module compiles against the **2.0.5** table-session client; its + integration tests exercise the real write path against an **IoTDB 2.0.8** + standalone server, so the 2.0.5-client / 2.0.8-server RPC path is the + validated pairing. Any IoTDB 2.x server that speaks the same table-session RPC + is a candidate, but 2.0.8 is the version the module is tested against. +- The build parent itself targets an older Spring line; the module deploys into + ThingsBoard 4.3.x, which runs Spring Boot 3.5.x / Spring 6 / JDK 17 (the + `jakarta.*` namespace). That is why `jakarta.validation-api` is overridden to + `3.0.2` and why the module requires **JDK 17+** to build and run. The + Hibernate Validator provider is supplied by the ThingsBoard runtime classpath, + not bundled by the module. +- If you cannot determine a version from this table, read it from the module + `pom.xml` and the `iotdb-extras` parent `pom.xml` rather than assuming a value. + +## Before you begin + +- A reachable IoTDB 2.x server in **Table Mode** (the tested server is IoTDB + 2.0.8). Have its host, port, and credentials ready. +- A ThingsBoard 4.3.1.2 build you control (the module is consumed as a + compile/runtime dependency of the ThingsBoard application; it is not a + drop-in for a pre-built binary distribution). +- JDK 17+ for the ThingsBoard build that includes this module. +- Decide which routes you are enabling: telemetry only, telemetry + latest, and + whether to opt in to the stretch attributes route. + +## Activation selectors + +Each route is controlled by its own selector. Until a route's selector is set, +that route stays inert. + +| Route | Required properties | Default state | +| --- | --- | --- | +| Historical telemetry (raw read/write/delete) | `database.ts.type=iotdb-table` **and** `iotdb.ts.experimental-raw-only=true` | inert | +| Latest value | `database.ts.type=iotdb-table` **and** `database.ts_latest.type=iotdb-table` **and** `iotdb.ts.experimental-raw-only=true` | inert | +| Entity attributes (stretch) | `database.attributes.type=iotdb-table` (independent of the timeseries selectors); also requires `iotdb.attributes.cluster_mode` set to `sticky-routing` or `disabled` | inert | + +The latest route deliberately also requires the timeseries selector: the latest +value is derived from the `telemetry` table that only the IoTDB writer +populates, so activating latest without the IoTDB write path would read from a +table the module never fills. The attribute selector is fully independent of the +timeseries selectors. + +## Step-by-step enablement + +### 1. Add the module to the ThingsBoard build + +Build and install `org.apache.iotdb:iotdb-thingsboard-table` and add it as a +dependency of the ThingsBoard application so its classes are on the runtime +classpath. The module is a Spring Boot auto-configuration +(`IoTDBTableConfiguration`) registered through +`META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports`, +so it activates without the host application having to component-scan +`org.apache.iotdb.extras`. + +In the `iotdb-extras` reactor the module lives behind a named opt-in profile and +requires JDK 17+: + +```bash +# from the iotdb-extras repository root +mvn -pl iotdb-thingsboard-table -am -P with-thingsboard clean install +``` + +At this point the jar is present but **inert** — no selector is set yet, so +ThingsBoard still uses its existing backend. + +### 2. Bootstrap the schema + +On startup the module runs an idempotent bootstrap (gated by +`iotdb.schema.bootstrap`, default `true`) that reads `schema-iotdb-table.sql` +from the classpath and creates the target database plus the `telemetry` and +`entity_attributes` tables on a fresh IoTDB before the first write. When the +latest route is active, a **second** bootstrap reads +`schema-iotdb-table-latest.sql` to create the `telemetry_latest` overlay table. +Both bootstraps use `CREATE ... IF NOT EXISTS`, so they are safe to re-run. + +If you manage the schema out of band, set `iotdb.schema.bootstrap=false` and +apply `schema-iotdb-table.sql` (and `schema-iotdb-table-latest.sql` when the +latest route is enabled) yourself before first start. + +### 3. Set the activation properties for the routes you want + +Add the selectors from the table above to the ThingsBoard configuration. For +example, to route both historical telemetry and latest values to IoTDB: + +```properties +database.ts.type=iotdb-table +database.ts_latest.type=iotdb-table +iotdb.ts.experimental-raw-only=true +``` + +To additionally opt in to the stretch attributes route (only on a ThingsBoard +build that exposes the attribute selector — no shipped release does yet, see +*Limitations*): + +```properties +database.attributes.type=iotdb-table +iotdb.attributes.cluster_mode=sticky-routing +``` + +`iotdb.attributes.cluster_mode` is mandatory when the attribute route is active +and must be `sticky-routing` (per-identity writes pinned to one node) or +`disabled` (single-node / acknowledged best-effort); any other value, including +the empty default, fails construction fast, because the attribute write path +converges only within a single JVM. + +### 4. Point the module at IoTDB and size the pools + +Set the connection and pool properties (bound from `iotdb.*`): + +```properties +iotdb.host= +iotdb.port=6667 +iotdb.username= +iotdb.password= +iotdb.database=thingsboard +iotdb.session-pool-size=8 +``` + +The defaults are `127.0.0.1:6667`, credentials `root`/`root`, database +`thingsboard`, and a session pool of `8`. `iotdb.database` must be a valid IoTDB +identifier (letter or underscore, then letters/digits/underscores). The write +path and read path have their own pools, tunable under `iotdb.ts.save.*` and +`iotdb.ts.read.*`: + +- Write path: `iotdb.ts.save.batch-size` (default `500`), + `iotdb.ts.save.max-linger-ms` (`20`), `iotdb.ts.save.queue-capacity` + (`50000`), and retry/backoff settings. The writer uses a single dedicated + flush worker, so `iotdb.ts.save.flush-threads` is fixed at `1` (any other + value fails binding). +- Read path: `iotdb.ts.read.threads` (default `4`) and + `iotdb.ts.read.queue-capacity` (`10000`). + +### 5. Set retention (optional, table-level TTL) + +Physical retention in IoTDB Table Mode is a **table-level** property expressed in +**milliseconds** as a bare long literal, or the keyword `INF` (never expire) / +`DEFAULT` (inherit the database default, which is `INF` on a fresh node). The +shipped `telemetry` table is created `WITH (TTL=DEFAULT)`. To enable a concrete +retention, either edit the schema before bootstrap, e.g. 7 days: + +```sql +CREATE TABLE telemetry (...) WITH (TTL=604800000); +``` + +or change it at runtime on the live table: + +```sql +ALTER TABLE telemetry SET PROPERTIES TTL=604800000; -- 7 days, in ms +ALTER TABLE telemetry SET PROPERTIES TTL=DEFAULT; -- back to the db default +``` + +Quoted numbers (`'604800000'`) and duration forms (`'7d'`) are rejected by IoTDB +2.0.8. The `telemetry_latest` overlay table is created `WITH (TTL='INF')` and is +**exempt** from the `telemetry` TTL: setting a retention window on `telemetry` +does not evict overlay rows, so bound the overlay separately if needed (see +*Limitations*). + +## Data-model mapping + +ThingsBoard's relational `ts_kv` stores one row per `(entity_id, key, ts)` with +the value in one typed column (`bool_v` / `long_v` / `dbl_v` / `str_v` / +`json_v`) and `key` dictionary-encoded. The IoTDB `telemetry` table keys each row +by a TAG tuple plus the built-in `time` column, with the value in the matching +FIELD column. Exactly one value column is non-null per row, preserving the +source shape. + +### `ts_kv` → `telemetry` + +| ThingsBoard `ts_kv` | IoTDB `telemetry` | IoTDB column kind | +| --- | --- | --- | +| `entity_id` | `entity_id` | TAG | +| *(entity type)* | `entity_type` | TAG | +| *(tenant)* | `tenant_id` | TAG | +| `key` (dictionary id → string) | `key` (resolved to the string key) | TAG | +| `ts` | `time` (built-in) | TIME | +| `bool_v` | `bool_v` (`BOOLEAN`) | FIELD | +| `long_v` | `long_v` (`INT64`) | FIELD | +| `dbl_v` | `double_v` (`DOUBLE`) | FIELD | +| `str_v` | `str_v` (`STRING`) | FIELD | +| `json_v` | `json_v` (`TEXT`) | FIELD | + +### `ts_kv_latest` → `telemetry_latest` + +Do **not** migrate `ts_kv_latest` as a separate store. The latest value is read +from **both** the historical `telemetry` table (derived, newest row per key) and +a minimal per-key `telemetry_latest` overlay, merged by the maximum timestamp +per key. Once history is in `telemetry`, the derived latest follows +automatically; the overlay is written by the module on every `saveLatest` going +forward. The overlay table has the same TAG tuple +(`entity_type`, `tenant_id`, `key`, `entity_id`) and FIELD columns as +`telemetry`, plus an explicit `time TIMESTAMP TIME` column. + +### `attribute_kv` → `entity_attributes` (stretch route only) + +Applies only if the attribute route is activated. Each identity tuple +`(tenant_id, entity_type, entity_id, attribute_scope, key)` holds exactly one +current row at `time = lastUpdateTs`, with one typed FIELD set. + +| ThingsBoard attributes | IoTDB `entity_attributes` | IoTDB column kind | +| --- | --- | --- | +| `entity_id` | `entity_id` | TAG | +| *(entity type)* | `entity_type` | TAG | +| *(tenant)* | `tenant_id` | TAG | +| attribute scope | `attribute_scope` (`CLIENT_SCOPE` / `SERVER_SCOPE` / `SHARED_SCOPE`) | TAG | +| attribute key | `key` | TAG | +| `last_update_ts` | `time` | TIME | +| boolean / long / double / string / json value | `bool_v` / `long_v` / `double_v` / `str_v` / `json_v` | FIELD | + +## Migrating existing data (backfill) + +The module supplies the write/read DAO surface but not an ETL tool; the backfill +is operator-driven. + +1. **Stand up IoTDB** in Table Mode and bootstrap the schema (Step 2). Set a + concrete `telemetry` TTL now (Step 5) if you want retention, before + backfilling. +2. **Backfill** historical `ts_kv` into `telemetry`: read from the source + backend, resolve the `key` dictionary id to the string key, supply + `tenant_id` / `entity_type` for each `entity_id`, and write via batched + `Tablet` inserts using the value-column mapping above. The module's own write + path batches 500 rows per flush, a reasonable starting batch size. +3. **Verify** a sample: spot-check historical reads and latest values for a set + of `(entity, key)` pairs against the source before the cut-over. +4. **Switch** ThingsBoard to the IoTDB routes by setting the selectors (Step 3) + and restart. New writes flow into `telemetry`; the derived latest is + immediately correct because it reads the same table, and the overlay is + written from the first new `saveLatest` onward. + +Idempotency: re-running the backfill for the same `(tags, time)` overwrites the +typed column in place (IoTDB same-timestamp overwrite), so re-runs converge +rather than duplicate. A value whose **type** changed at the same timestamp is +subject to the same-timestamp limitation below. + +For a zero-downtime cut-over, a dual-write or read-old / write-new bridge during +backfill is an operator concern beyond this guide. + +## Coexistence and rollback + +Each route is independently activated and guarded: + +- **Independent activation.** Enabling telemetry does not enable attributes, and + vice versa. You can route telemetry + latest to IoTDB while attributes stay in + the host entity database (the default Phase-1 posture). +- **Fail-fast conflict guard.** When a route is enabled but a conflicting + non-IoTDB host DAO bean of the same SPI type is also present, startup fails + fast with a clear message rather than silently shadowing one DAO with another. + The historical (`TimeseriesDao`), latest (`TimeseriesLatestDao`), and attribute + (`AttributesDao`) routes each have their own guard. Make sure the host backend + for a route is removed/disabled when you point that route at IoTDB. + +### Rollback + +To roll a route back, **unset its selector(s)** and restart. The route's +activation condition then returns `false`, the IoTDB DAO and its session pool are +not created, the conflict guard does not fire, and the host backend's DAO +resumes serving that route. + +What is and isn't reversible: + +- **Configuration is fully reversible.** Unsetting the selectors returns + ThingsBoard to the host backend with no schema changes required on the IoTDB + side; the IoTDB tables simply stop receiving new writes. +- **Data written to IoTDB stays in IoTDB.** Rolling back the configuration does + not copy IoTDB-resident telemetry/latest/attributes back into the host + backend. If the host backend was serving the route during the IoTDB window, + those two stores will have diverged for that period. Plan a reverse backfill + (or a dual-write window) if you need the host backend to be current after a + rollback. + +## Limitations to plan for + +These are the documented current-scope limitations (see the DAO Javadoc and the +module `README.md` for the authoritative list): + +- **`version` is always `null`** on the latest and attribute routes. IoTDB has no + SQL sequence (same as the Cassandra backend); this is type-correct and + contract-legal, but ThingsBoard notifications that key off a non-null version + are not driven in Phase-1. +- **Same-timestamp cross-store type change.** A single + `(tenant, entity, key, timestamp)` point whose value *type* changes between two + separate flushes can land two non-null typed columns. The behavior is + **fail-fast, not silent**: a raw read of that one point throws an + `IllegalStateException` rather than returning a wrong value; every other point + is unaffected. The latest overlay wins an exact-timestamp tie, continuing the + same limitation on the latest route. +- **Overlay TTL=`INF` growth.** `telemetry_latest` never physically expires and + has no entity-level cleanup, so under unbounded key cardinality it grows + without bound (one row per identity; bounded for normal key sets). Bound it + separately with a TTL on `telemetry_latest` if needed. +- **Per-save `ttl` is not honored as physical retention.** The + `TimeseriesDao.save(..., long ttl)` per-data-point TTL cannot be reconciled + with table-wide IoTDB TTL; the module uses it only for ThingsBoard's + storage-accounting, never as a physical-retention directive. Set physical + retention on the table (Step 5). +- **The attributes route is a stretch / Phase-2 opt-in.** No shipped ThingsBoard + release exposes a `database.attributes.type` selector yet (open question, + tracked upstream), so in a real Phase-1 deployment the selector is unset and + attributes stay in the host entity database. When activated, `save` is a + non-atomic tag-only delete-then-insert under a per-identity in-JVM lock that + converges only within one JVM; `findNextBatch` is unsupported + (`UnsupportedOperationException`), and `findAllKeysByDeviceProfileId` with a + non-null profile returns an empty list (matching the non-relational backend). + +## Verifying the cut-over + +After switching a route to IoTDB: + +- Confirm the target tables exist and carry data: `SHOW TABLES` in the IoTDB + database, and spot-check rows in `telemetry` (and `telemetry_latest` when the + latest route is on). +- Confirm the effective TTL via `information_schema.tables` (the `ttl(ms)` + column) or `SHOW TABLES` (the `TTL(ms)` column) if you set retention. +- Read back a few `(entity, key)` latest values through ThingsBoard and compare + against the pre-cut-over source sample. diff --git a/iotdb-thingsboard-table/docs/user-guide.md b/iotdb-thingsboard-table/docs/user-guide.md new file mode 100644 index 0000000..2c19743 --- /dev/null +++ b/iotdb-thingsboard-table/docs/user-guide.md @@ -0,0 +1,315 @@ + + +# IoTDB ThingsBoard Table — User Guide + +This guide explains how to configure, activate, and operate the IoTDB Table Mode +storage backend for ThingsBoard. For the architecture overview, the +compile-only ThingsBoard SPI surface, build/test instructions, and the design +rationale, see the module [`README.md`](../README.md). + +## 1. What the module is + +`iotdb-thingsboard-table` is a storage backend that lets a ThingsBoard +deployment persist telemetry, latest values, and (optionally) entity attributes +in Apache IoTDB 2.x relational **Table Mode** instead of the default +Cassandra / SQL backends. It implements ThingsBoard's `TimeseriesDao`, +`TimeseriesLatestDao`, and `AttributesDao` SPIs and ships as a Spring Boot +auto-configuration, so it activates inside a real ThingsBoard deployment without +the host application having to component-scan the module's package. + +The module is built against the reactor's IoTDB 2.0.5 table-session client and +its integration tests exercise the real write path against an +`apache/iotdb:2.0.8-standalone` server. It targets ThingsBoard `4.3.1.2` and +requires JDK 17+. + +**It is default-inert.** None of the three DAOs activate unless the operator +sets the explicit selector properties below. With no selector set, the module +contributes no beans and ThingsBoard keeps using its configured backends. + +## 2. Prerequisites + +- A running Apache IoTDB 2.x node with relational Table Mode, reachable from the + ThingsBoard host over the IoTDB client RPC port. +- The target IoTDB database (default `thingsboard`) and its tables. On a fresh + node the module's idempotent startup bootstrap creates them for you (see + [§5](#5-schema-and-bootstrap)); operators who manage the schema out-of-band + can disable the bootstrap and pre-create the tables from the shipped SQL. +- IoTDB credentials with permission to create the database/tables (when the + bootstrap is enabled) and to read/write the tables. + +## 3. Activation + +Each path is selected independently and is inert until its selector is present. +Selector values are matched case-insensitively (trimmed). + +| Path | Required properties | Default state | +| --- | --- | --- | +| Historical timeseries (`TimeseriesDao`) | `database.ts.type=iotdb-table` **and** `iotdb.ts.experimental-raw-only=true` | inert | +| Latest telemetry (`TimeseriesLatestDao`) | all of `database.ts.type=iotdb-table`, `iotdb.ts.experimental-raw-only=true`, **and** `database.ts_latest.type=iotdb-table` | inert | +| Entity attributes (`AttributesDao`) | `database.attributes.type=iotdb-table` (independent of the timeseries selectors) | inert (opt-in / stretch) | + +Notes: + +- **Timeseries** requires both `database.ts.type=iotdb-table` and the explicit + experimental opt-in `iotdb.ts.experimental-raw-only=true`; setting only one of + them leaves the path inert. +- **Latest** additionally requires `database.ts_latest.type=iotdb-table`. The + timeseries selectors are required as well because the latest value is derived + from the `telemetry` table that only the IoTDB writer populates — the latest + path can never activate without that writer. +- **Attributes** is an opt-in stretch feature. No shipped + ThingsBoard release exposes a `database.attributes.type` selector yet (open + question Q6 / ThingsBoard Discussion #15296), so a real deployment normally + leaves it unset and attributes keep flowing to the host entity database. When + it is activated, `iotdb.attributes.cluster-mode` must also be set or the DAO + fails fast at startup (see [§6](#6-configuration-reference)). + +When activated, the DAOs share a single module-owned IoTDB table session pool. + +### Conflict guards + +When a selector is on, the module fails startup fast if a conflicting non-IoTDB +DAO bean of the same SPI type is also present (for example, another +`TimeseriesDao` while `database.ts.type=iotdb-table`). This is deliberate: it +prevents the module from silently shadowing, or being shadowed by, a different +backend. Remove the conflicting backend or unset the IoTDB selector. + +## 4. The three DAOs + +### 4.1 Timeseries DAO (`TimeseriesDao`) + +- **Write** (`save`): telemetry points are accepted onto a bounded in-memory + queue and flushed by a single writer thread as multi-row `Tablet` inserts into + the `telemetry` table. Writes are asynchronous and batched; the call returns a + future. Duplicate `(tenant, entity, key, timestamp)` saves within one flush are + collapsed so the last write wins. +- **Raw read** (`findAllAsync` with no aggregation, or an interval below 1): + returns the stored points for the query window, ordered and limited as + requested. +- **Aggregated read** (`findAllAsync` with a positive interval): time-bucketed + aggregation matching ThingsBoard's contract, in two forms: + - **Fixed-width millisecond buckets** anchored at the query start timestamp, + using IoTDB's `date_bin` primitive (supports `AVG`, `SUM`, `COUNT`, `MIN`, + `MAX`). + - **Timezone-aware calendar buckets** for `WEEK`, `WEEK_ISO`, `MONTH`, and + `QUARTER` intervals, where each calendar boundary is computed in the query's + timezone and one bounded aggregate query is issued per bucket. +- **Delete** (`remove`): deletes points for an identity within the query's + half-open time window. +- **Retention hooks**: `cleanup` is a no-op and `savePartition` does nothing, + because IoTDB Table Mode keeps no per-partition bookkeeping and physical + retention is a table-level property (see [§7](#7-retention--ttl)). + +### 4.2 Latest telemetry DAO (`TimeseriesLatestDao`) + +Active only when all three timeseries+latest selectors are set. + +- **Read** (`findLatest`, `findLatestOpt`, `findAllLatest`): the latest value per + `(tenant, entity, key)` is read from **both** the historical `telemetry` table + (derived via `ORDER BY time DESC LIMIT 1` / `LAST_BY`, engine-accelerated by + IoTDB's last cache) **and** the `telemetry_latest` overlay table, then merged + by the maximum timestamp per key (the overlay wins an exact tie). The merge is + not additive, so a key present in both stores is never double-counted. +- **Write** (`saveLatest`): writes the per-key overlay row as a delete-then-insert + under a per-identity lock. It is **max-timestamp-wins**: a backdated + `saveLatest` whose timestamp is not newer than the stored latest is skipped, so + an out-of-order write never regresses the latest. The overlay is written on + every (non-backdated) call so a latest-only write — one with no paired + historical `save`, e.g. the EntityView telemetry-copy path — is not lost. +- **Delete** (`removeLatest`): snapshots the merged latest under the per-identity + lock; when that latest falls inside the half-open `[startTs, endTs)` window it + deletes the overlay row, and when `rewriteLatestIfDeleted` is set it resurrects + the next-older historical value (from `telemetry`) back into the overlay. +- **Key discovery** (`findAllKeysByEntityIds` / `…Async`): returns the distinct + telemetry keys for the entity set, unioned across `telemetry` and the + `telemetry_latest` overlay, so a latest-only key is still discoverable. + `findAllKeysByDeviceProfileId` returns the tenant-wide distinct keys for a null + (all-profiles) device profile and an empty list for a specific profile (the + tables carry no device-profile membership). + +### 4.3 Entity attributes DAO (`AttributesDao`) — opt-in / stretch + +Inert by default; activated only by `database.attributes.type=iotdb-table`. + +- **Write** (`save`): each identity tuple + `(tenant_id, entity_type, entity_id, attribute_scope, key)` holds exactly one + current row. `save` is a tag-only `DELETE` (no time predicate) followed by an + `INSERT` at `time = lastUpdateTs` with one typed value column set, both under a + per-identity lock so concurrent same-identity writes converge to a single row. +- **Read** (`find`, `findAll`, and the by-keys / by-entity-ids variants): + synchronous `SELECT`s over the `entity_attributes` table. +- **Delete** (`removeAll`, `removeAllWithVersions`): one future per key, each a + tag-only `DELETE`. `removeAllByEntityId` is a best-effort select-then-delete + under an entity-level write lock. + +## 5. Schema and bootstrap + +The module uses one table per path. All three share the same five typed value +columns — `bool_v BOOLEAN`, `long_v INT64`, `double_v DOUBLE`, `str_v STRING`, +`json_v TEXT` — of which **exactly one is non-null per row**. + +| Table | Purpose | TAG columns (in order) | TTL | +| --- | --- | --- | --- | +| `telemetry` | historical telemetry | `entity_type`, `tenant_id`, `key`, `entity_id` | `DEFAULT` | +| `telemetry_latest` | per-key latest overlay | `entity_type`, `tenant_id`, `key`, `entity_id` | `INF` | +| `entity_attributes` | entity attributes | `attribute_scope`, `entity_type`, `tenant_id`, `key`, `entity_id` | `INF` | + +The shipped DDL lives in +[`schema-iotdb-table.sql`](../src/main/resources/schema-iotdb-table.sql) +(`telemetry` + `entity_attributes`) and +[`schema-iotdb-table-latest.sql`](../src/main/resources/schema-iotdb-table-latest.sql) +(`telemetry_latest`). + +**Bootstrap.** When `iotdb.schema.bootstrap` is `true` (the default), the module +runs an idempotent startup bootstrap that reads the schema SQL from the classpath +and creates the database and tables before the first write. Every statement uses +`CREATE … IF NOT EXISTS`, so re-runs are harmless. The base schema is created +whenever the timeseries or attributes path is active; the `telemetry_latest` +overlay is created by a second bootstrap only when the latest path is active. +Set `iotdb.schema.bootstrap=false` to manage the schema out-of-band. + +## 6. Configuration reference + +All module knobs are bound from the `iotdb.*` prefix; the three `database.*` +selectors and `iotdb.ts.experimental-raw-only` are the activation switches from +[§3](#3-activation). Defaults are shown. + +### Connection + +| Property | Default | Notes | +| --- | --- | --- | +| `iotdb.host` | `127.0.0.1` | IoTDB node host. | +| `iotdb.port` | `6667` | IoTDB client RPC port (1–65535). | +| `iotdb.database` | `thingsboard` | Target Table Mode database. Must be a valid IoTDB identifier (a letter or underscore, then letters/digits/underscores). | +| `iotdb.username` | `root` | Set a real credential in production. | +| `iotdb.password` | `root` | Set a real credential in production. | +| `iotdb.session-pool-size` | `8` | Table session pool size (1–1024). Keep it at least `iotdb.ts.read.threads + iotdb.ts.save.flush-threads`, or the module logs a warning that reads/flushes may wait for a session. | +| `iotdb.connection-timeout-ms` | `5000` | Connection timeout (minimum 100). | +| `iotdb.enable-compression` | `false` | RPC compression. | +| `iotdb.default-ttl-ms` | `-1` | ThingsBoard storage data-point accounting only; **not** an IoTDB physical-retention setting. `-1` disables the default. | +| `iotdb.schema.bootstrap` | `true` | Run the idempotent startup schema bootstrap. Set `false` to manage the schema yourself. | + +### Write path (`iotdb.ts.save.*`) + +| Property | Default | Notes | +| --- | --- | --- | +| `batch-size` | `500` | Rows per `Tablet` insert. | +| `max-linger-ms` | `20` | Maximum time a partial batch waits to fill before flushing. | +| `queue-capacity` | `50000` | Bounded accept queue; over-capacity saves are rejected (back-pressure), not blocked. | +| `flush-threads` | `1` | Fixed at 1 (single-writer ordering). Any other value fails binding with a clear error. | +| `shutdown-drain-timeout-ms` | `5000` | Graceful-drain window on shutdown. Keep `≥ max-linger-ms`. | +| `retry-max-attempts` | `3` | Per-batch attempts on a transient IoTDB error. | +| `retry-initial-backoff-ms` | `50` | Initial retry backoff. | +| `retry-max-backoff-ms` | `1000` | Maximum retry backoff (exponential between the two). | + +### Read path (`iotdb.ts.read.*`) + +These also size the bounded executor used by the latest and attribute DAOs. + +| Property | Default | Notes | +| --- | --- | --- | +| `threads` | `4` | Bounded read/IO executor size. | +| `queue-capacity` | `10000` | Read/IO task queue bound; an over-capacity task is rejected with a queue-full error. | + +### Attributes (`iotdb.attributes.*`) — only when the attribute DAO is active + +| Property | Default | Notes | +| --- | --- | --- | +| `iotdb.attributes.cluster-mode` | *(empty)* | Required when `database.attributes.type=iotdb-table`. Must be `sticky-routing` (writes for an identity pinned to one node) or `disabled` (single-node / acknowledged best-effort). Any other value, including the empty default, fails fast at startup, because the per-identity write lock converges writes only within a single JVM. (The module also accepts the equivalent `iotdb.attributes.cluster_mode` spelling.) | + +## 7. Retention / TTL + +Physical retention is a **table-level** IoTDB property, expressed in +milliseconds, set by the operator on the schema — not a per-data-point setting. +The shipped `telemetry` table is declared `WITH (TTL=DEFAULT)` (inherit the +database default, which is `INF` on a fresh node). To enable a concrete +retention, edit the schema before bootstrap or run +`ALTER TABLE telemetry SET PROPERTIES TTL=` on the live table. + +Two consequences to be aware of: + +- The per-save `ttl` argument carried by `TimeseriesDao.save(..., ttl)` is **not** + applied as physical retention per row — IoTDB Table Mode TTL is table-wide. The + module uses the per-save `ttl` (and `iotdb.default-ttl-ms`) only for + ThingsBoard's storage data-point accounting. +- The `telemetry_latest` overlay is declared `WITH (TTL='INF')` and is exempt + from the `telemetry` TTL, so a retention window on `telemetry` does not evict + overlay rows. Operators who need to bound the overlay must set a TTL on + `telemetry_latest` separately. + +See the README's *Retention / TTL* section for the full mechanism and the +syntax IoTDB accepts. + +## 8. Operational notes + +- **Back-pressure.** The write accept queue (`iotdb.ts.save.queue-capacity`) is + bounded. When it is full, a `save` is rejected with a queue-full error rather + than blocking the caller — size the queue for your ingest rate. +- **Retries.** A batch that hits a transient IoTDB condition (connection error, + or a server-side overload / timeout / dispatch status) is retried up to + `retry-max-attempts` with exponential backoff. Permanent errors (parse, type, + schema) fail fast without consuming the retry budget. +- **Graceful shutdown.** On shutdown the writer stops accepting new saves and + drains already-accepted writes within `iotdb.ts.save.shutdown-drain-timeout-ms`. + If the window expires, the remaining pending writes are failed. Shutdown is + at-least-once: if the timeout races an insert that has already started, the + write may still commit after its future has failed, so tolerate a duplicate or + uncertain final batch. +- **Session pool sizing.** Keep `iotdb.session-pool-size` at least as large as + the sum of read threads and flush threads (see [§6](#6-configuration-reference)). + +## 9. Known limitations (Phase-1 residuals) + +These are documented behaviors of the current module, not defects to work around +silently: + +- **Latest/attribute `version` is always `null`.** IoTDB has no SQL sequence, so + `saveLatest`, attribute `save`, and `removeAllWithVersions` return a `null` + version (type-correct, matching the Cassandra backend). ThingsBoard + notifications that key off a non-null version are therefore not driven. +- **Same-timestamp cross-store type change.** The writer collapses duplicate + `(tenant, entity, key, timestamp)` saves within a single flush, but it does not + defend against a same-timestamp save whose value *type* changes across two + separate flushes. Because each typed value lands in its own column, that single + point can end up with two non-null typed columns. The behavior is fail-fast, + not silent: a read of that one point throws rather than returning a wrong value; + every other point is unaffected. In the latest path, the overlay wins an exact + timestamp tie. +- **Overlay growth.** `telemetry_latest` is `TTL='INF'` with no entity-level + cleanup, so under unbounded key cardinality it grows without bound (one row per + identity; bounded for normal key sets). +- **Latest delete eventual consistency.** A purely telemetry-derived (full-save) + latest can transiently still be read from `telemetry` until the separate + historical delete commits. +- **Single-JVM write convergence.** The per-identity locks that serialize the + latest-overlay and attribute writes are in-JVM, so cross-node single-writer + safety is the operator's responsibility. The attribute path makes this explicit + via the required `iotdb.attributes.cluster-mode` acknowledgement. +- **Unsupported SPI operations.** The attribute relational-migration helper + `findNextBatch` throws `UnsupportedOperationException`: it is a relational + keyset-pagination helper with no IoTDB equivalent, and it is the only method in + the module that throws. The batch latest read (`findLatestByEntityIds` / + `…Async`, new in ThingsBoard 4.3.1.2) instead returns empty, matching + `CassandraBaseTimeseriesLatestDao` — it backs the `includeSamples` branch of + `POST /api/entitiesQuery/find/keys`, where a synchronous throw would surface as + an HTTP 500. A specific (non-null) device-profile key lookup likewise returns + an empty list. diff --git a/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableConfig.java b/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableConfig.java index 2155f0c..915d352 100644 --- a/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableConfig.java +++ b/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableConfig.java @@ -84,9 +84,13 @@ public class IoTDBTableConfig { @Data public static class Ts { /** - * Explicit opt-in for the raw-only backend. This backend currently implements write, raw read, - * and delete only; time-bucketed aggregation is outside the current scope, so it is not - * production-complete and must be enabled deliberately. + * Explicit opt-in for this backend. It implements write, raw read, time-bucketed aggregation + * (both the fixed-width millisecond {@code date_bin} path and the timezone-aware calendar + * buckets) and delete. The property name predates the aggregation support and is kept for + * compatibility. It remains experimental because of a known Phase-1 relaxation: a single {@code + * (tenant, entity, key, timestamp)} point whose value type changes across two separate flushes + * can leave two non-null typed columns, and a raw read of that one point then fails fast — so + * the backend must still be enabled deliberately. */ private boolean experimentalRawOnly = false; diff --git a/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDao.java b/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDao.java index 216ffcd..a958859 100644 --- a/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDao.java +++ b/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDao.java @@ -80,7 +80,7 @@ public class IoTDBTableTimeseriesDao extends IoTDBTableBaseDao private static final long SECONDS_PER_DAY = 86400L; private static final String TABLE_NAME = IoTDBTableTimeseriesWriter.TABLE_NAME; - // Result-set column aliases for the time-bucketed aggregation read path (design doc §3.3). + // Result-set column aliases for the time-bucketed aggregation read path. private static final String BUCKET_TS_COLUMN = "bucket_ts"; private static final String AGG_NUM_COLUMN = "agg_num"; private static final String AGG_STR_COLUMN = "agg_str";