Add the TC-1 ingestion-throughput benchmark for the Table Mode write path

iotdb-thingsboard-table/docker-compose.bench.yml

@@ -0,0 +1,66 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+# TC-1 ingestion-throughput SMOKE-profile bench stack.
+#
+# This is the smoke profile: a single, clean-volume IoTDB 2.0.8 node for a fast,
+# reproducible local benchmark run. It mirrors docker-compose.test.yml but ships
+# only the IoTDB service on a dedicated fresh volume so each run starts from an
+# empty store. The benchmark IT (IoTDBTableIngestionBenchmarkIT) provisions its
+# own throwaway Testcontainer, so this compose file is only for an out-of-band
+# manual smoke run against a standalone node.
+#
+# The FULL multi-backend profile (Cassandra / PostgreSQL / TimescaleDB on a
+# dedicated host, contributor-run) is later-scope and is not defined here.
+#
+# Do not hardcode passwords or local hostnames; pass IOTDB_USERNAME /
+# IOTDB_PASSWORD via the environment.
+
+services:
+  iotdb:
+    image: apache/iotdb:2.0.8-standalone
+    container_name: iotdb-table-bench
+    environment:
+      IOTDB_USERNAME: ${IOTDB_USERNAME:?set IOTDB_USERNAME}
+      IOTDB_PASSWORD: ${IOTDB_PASSWORD:?set IOTDB_PASSWORD}
+      # Bind the client RPC service to all interfaces so the mapped host port is
+      # reachable from the benchmark client (default 127.0.0.1 only listens on the
+      # container loopback).
+      dn_rpc_address: 0.0.0.0
+    ports:
+      - "${IOTDB_RPC_PORT:-6667}:6667"
+    volumes:
+      # Fresh, dedicated volume so every smoke run starts from an empty store.
+      # Reset between runs with: docker compose -f docker-compose.bench.yml down -v
+      - iotdb-bench-data:/iotdb/data
+    networks:
+      - tb-iotdb-bench
+    healthcheck:
+      test: ["CMD-SHELL", "bash -ec ': >/dev/tcp/127.0.0.1/6667'"]
+      interval: 10s
+      timeout: 5s
+      retries: 12
+      start_period: 30s
+
+networks:
+  tb-iotdb-bench:
+    driver: bridge
+
+volumes:
+  iotdb-bench-data:

iotdb-thingsboard-table/docs/benchmarks/README.md

@@ -0,0 +1,170 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# IoTDB ThingsBoard Table — Benchmarks
+
+This directory documents the performance test cases for the IoTDB Table Mode
+ThingsBoard storage backend. It currently covers **TC-1 (ingestion
+throughput)**, the write-throughput case for this backend.
+
+## Two profiles
+
+Each test case is defined with two profiles:
+
+- **Smoke profile** — small, fast, reproducible on a laptop in well under ten
+  minutes; single tenant; not wired into required CI, though it is CI-eligible
+  wherever Docker is available. Its purpose is to exercise the real write path
+  end to end and guard against gross regressions. **This is what is implemented
+  today** (`IoTDBTableIngestionBenchmarkIT`).
+- **Full profile** — a dedicated host, contributor-run, multi-backend
+  comparison (Cassandra / PostgreSQL / TimescaleDB). It is *not* in CI and is
+  **later-scope**; its report lives in [`report.md`](report.md) (placeholder
+  until a fresh run is recorded).
+
+## TC-1 — Ingestion throughput
+
+Design intent: 1,000 devices writing simultaneously via 50 concurrent threads in
+500-entry batches; measure records/sec and error rate. The calendar target is
+**> 10,000 writes/sec**.
+
+That **> 10K writes/sec figure is the full-profile headline on a dedicated
+host.** A cold single-node container on a laptop or CI runner will not reach it,
+so the smoke profile does not assert it.
+
+### What the smoke benchmark does
+
+`IoTDBTableIngestionBenchmarkIT` drives the **real** save path — the same code
+ThingsBoard uses in production:
+
+```
+dao.save(tenant, entity, tsKvEntry, ttl)
+  -> writer.enqueue(...)           bounded ArrayBlockingQueue (capacity 50,000)
+  -> single flush worker           batches up to 500 rows, maxLingerMs 20
+  -> Tablet insert                 multi-row table-session insert
+  -> real IoTDB 2.0.8              apache/iotdb:2.0.8-standalone Testcontainer
+```
+
+It runs `SAVER_THREADS = 50` concurrent threads, each writing
+`ROWS_PER_THREAD = 600` rows (30,000 rows total) with a distinct
+`(entity, key, timestamp)` per write so nothing is deduplicated away. The
+production save defaults are used unchanged (batchSize 500, queueCapacity
+50,000, maxLingerMs 20, flushThreads 1, sessionPoolSize 8); only the retry
+backoff is shortened so a transient cold-start blip does not stretch the
+measured window. The total row count is kept below the queue capacity so the
+run is free of back-pressure rejects without changing the real defaults.
+
+### What it measures and asserts
+
+Measured and logged:
+
+- **records/sec** — `totalRows / wall-clock seconds`, timed from the first
+  `save()` to all save futures completing.
+- **error rate** — `failedFutures / totalRows`.
+- **writer stats** — `dao.stats()`: `enqueued`, `flushed`, `flushFailures`,
+  `retries`, `rejectsFull`, `rejectsShutdown`, `queueDepth`.
+- **persisted-sample count** — a handful of rows are read back from IoTDB to
+  prove real ingestion, not just future completion.
+
+Asserted:
+
+- error rate `== 0` and zero failed save futures;
+- `flushFailures == 0`, `rejectsFull == 0`, `rejectsShutdown == 0`;
+- `flushed == totalRows` (every distinct row reached IoTDB);
+- the sampled rows are readable back from IoTDB;
+- throughput `>=` a **conservative smoke floor of 1,000 rows/sec**.
+
+#### Why the floor is 1,000 rows/sec, not 10,000
+
+The smoke floor only guards against gross regressions and proves correctness on
+a cold, shared, single-node container. It is intentionally an order of magnitude
+below the full-profile headline so the test is not flaky on laptops or CI. Raise
+it only alongside a measured full-profile report — never to chase the headline
+number on CI.
+
+### How to run it
+
+The benchmark is named `*IT.java`, so the unit `mvn test` run never executes it;
+it only test-compiles there. Integration tests in this module run via the
+**`iotdb-table-it` profile** (maven-failsafe-plugin), which requires Docker.
+
+The benchmark is tagged `@Tag("benchmark")` and `@Tag("integration")`. Use the
+JUnit tag filter to select it. Run **only** the benchmark:
+
+```bash
+cd iotdb-thingsboard-table
+mvn -ntp -Piotdb-table-it verify -Dgroups=benchmark
+```
+
+`-Dgroups=benchmark` runs only the `@Tag("benchmark")` test among the failsafe
+`**/*IT.java` set, so the functional ITs are skipped and only the throughput
+benchmark runs. Use `-Dgroups=benchmark`, **not**
+`-Dtest=IoTDBTableIngestionBenchmarkIT`: a global `-Dtest=` overrides the
+include/exclude filters of the surefire executions too, which can pull a Docker
+IT into the unit `test` phase. The tag filter is applied on top of the file
+patterns instead.
+
+The benchmark also runs as part of the normal integration-test gate
+(`mvn -ntp -Piotdb-table-it verify`): it is a deliberately cheap (~10 s, 30,000
+rows) **throughput-regression guard** that asserts only a conservative floor of
+1,000 records/sec, so it stays non-flaky on shared CI while still catching a
+write-path performance regression. It needs no benchmark-specific pom wiring. If
+you want a purely functional gate, exclude its tag:
+
+```bash
+cd iotdb-thingsboard-table
+mvn -ntp -Piotdb-table-it verify -DexcludedGroups=benchmark
+```
+
+> **Gate note.** A bare `mvn -Piotdb-table-it verify` runs the benchmark as one
+> of the `**/*IT.java` set (it is `@Tag("benchmark")`) — this is intended, as it
+> guards the throughput floor. Use `-Dgroups=benchmark` to run *only* it and read
+> the throughput number; use `-DexcludedGroups=benchmark` to skip it.
+
+The measured records/sec and the full writer-stats report are emitted to the
+test log at INFO and to stdout, so the figure is captured even when no SLF4J
+binding is on the test classpath.
+
+If Docker is unavailable the test is skipped
+(`@Testcontainers(disabledWithoutDocker = true)`); it never fails the build for
+lack of Docker.
+
+### Smoke stack
+
+The benchmark IT manages its own throwaway `apache/iotdb:2.0.8-standalone`
+Testcontainer, so no external stack is required to run it. For a manual run
+against a standalone node instead of the throwaway container, the module's
+[`../../docker-compose.test.yml`](../../docker-compose.test.yml) brings up an
+IoTDB service (among the full ThingsBoard test stack):
+
+```bash
+IOTDB_USERNAME=<iotdb-user> IOTDB_PASSWORD=<iotdb-password> \
+  docker compose -f docker-compose.test.yml up -d iotdb
+
+# reset to an empty store between runs
+docker compose -f docker-compose.test.yml down -v
+```
+
+## Full-profile report
+
+The full-profile multi-backend report is deferred; see
+[`report.md`](report.md). The smoke-profile records/sec from a fresh run is also
+filled in there by whoever runs the benchmark — the numbers are never committed
+ahead of an actual run.

iotdb-thingsboard-table/docs/benchmarks/report.md

@@ -0,0 +1,91 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# TC-1 Ingestion-Throughput Benchmark Report
+
+Methodology is documented in [`README.md`](README.md). This file records the
+measured numbers. **No figure here is committed ahead of an actual run** — every
+value below is filled in from a fresh local run, not copied from a previous one.
+
+## Smoke profile (`IoTDBTableIngestionBenchmarkIT`)
+
+Single-node `apache/iotdb:2.0.8-standalone` Testcontainer; 50 concurrent saver
+threads; 600 rows/thread (30,000 rows total); production save defaults
+(batchSize 500, queueCapacity 50,000, maxLingerMs 20, flushThreads 1,
+sessionPoolSize 8). records/sec is timed from the first `save()` to all save
+futures completing.
+
+How to reproduce:
+
+```bash
+cd iotdb-thingsboard-table
+mvn -ntp -Piotdb-table-it verify -Dgroups=benchmark
+```
+
+Results (fresh runs, 2026-08-03):
+
+| Field | Value |
+| --- | --- |
+| Date | 2026-08-03 |
+| Host | Apple Silicon laptop (macOS) via Docker Desktop / Testcontainers — a shared dev host, not a dedicated benchmark machine |
+| IoTDB image | `apache/iotdb:2.0.8-standalone` (Testcontainers-managed) |
+| Total rows | 30,000 |
+| Saver threads | 50 |
+| Batch size | 500 |
+| Queue capacity | 50,000 |
+| Elapsed (s) | 0.55 |
+| **records/sec** | **54,292** |
+| Error rate | 0.0000 |
+| flushed / flushFailures / rejectsFull | 30,000 / 0 / 0 |
+| retries / rejectsShutdown / queueDepth | 0 / 0 / 0 |
+
+A second back-to-back run on the same host and configuration measured **61,936 rows/sec** in 0.48 s (error rate 0; flushed / flushFailures / rejectsFull = 30,000 / 0 / 0). Both runs are stable; the conservative **54,292** figure anchors the ≈ 5.4× / ≈ 54× ratios below.
+
+> The smoke run asserts only a conservative floor of 1,000 rows/sec on a cold,
+> shared, single-node container (so it stays non-flaky as a CI regression guard).
+> The observed **54,292 rows/sec** is ≈5.4× the **> 10,000 writes/sec** design
+> target and ≈54× the 1,000 rows/sec smoke floor — but treat it as a
+> regression-guard / peak figure, **not** as a pass of the sustained full-profile
+> target. That target is defined for 1,000 devices on a dedicated host (see the
+> full profile below); this smoke run drives only 50 distinct devices (one per
+> saver thread), and because `entity_id` is a TAG column the device cardinality
+> materially changes the write workload. The 30,000 rows also fit entirely in the
+> 50,000-row queue (so `rejectsFull=0`, no back-pressure) and are drained by a
+> single flush worker. The sustained 1,000-device > 10K target therefore remains
+> the deferred full profile, not something this smoke run validates.
+
+## Full profile (deferred / later-scope)
+
+> **Status: deferred / later-scope.**
+
+This section will hold the **full-profile** TC-1 ingestion-throughput results:
+1,000 devices, 50 concurrent threads, 500-entry batches, run on a dedicated
+host, with the **> 10,000 writes/sec** target and a multi-backend comparison
+(Cassandra / PostgreSQL / TimescaleDB).
+
+The full profile is not in CI and is run by a contributor on dedicated
+hardware. To be filled in:
+
+- Hardware and IoTDB topology (single node vs. cluster).
+- Dataset: device count, keys per device, batch size, total rows.
+- Measured records/sec, error rate, and p50 / p99 batch flush latency.
+- Per-backend comparison table.
+- Tuning notes (session pool size, flush threads, queue capacity, linger).