HDDS-15894. Use deleteRange to truncate Recon DB tables

[chihsuan]

What changes were proposed in this pull request?

ReconDBProvider.truncateTable currently iterates over every table entry and issues one point delete per key. This makes clearing a populated table require O(n) delete operations.

This change replaces the per-entry deletion loop with a fixed number of operations:

1. Use a key-only iterator to find the first and last keys.
2. Delete the range [firstKey, lastKey).
3. Delete lastKey separately because deleteRange excludes its end key.

The method is also made generic to avoid using raw Table and KeyValue types.

A real RocksDB-backed test covers null, empty, single-entry, and multi-entry tables. It also verifies that the table remains usable after truncation.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15894

How was this patch tested?

The focused Recon DB provider and caller tests passed:

mvn -o -pl :ozone-recon test \
  -Dtest='TestReconDBProvider,TestReconContainerMetadataManagerImpl,TestReconNamespaceSummaryManagerImpl' \
  -DskipShade -DskipRecon -DskipDocs

Result: 18 tests passed with no failures or errors.

Repository checkstyle also passed for all modules:

./hadoop-ozone/dev-support/checks/checkstyle.sh

Generated-by: Codex (GPT-5)

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[szetszwo]

@chihsuan , thanks for working on this! Please see the comments inlined.

[szetszwo]

Let's move it to Table, make it non-static and rename it to clear().

[szetszwo]

Let's add a new deleteRange method to including the lastKey. Google suggestion:

[szetszwo]

Tried it a little bit:

diff --git a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeTable.java b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeTable.java
index ed7a05027e..23e807cb88 100644
--- a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeTable.java
+++ b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeTable.java
@@ -68,6 +68,11 @@ public void deleteRange(KEY beginKey, KEY endKey) throws RocksDatabaseException,
     table.deleteRange(beginKey, endKey);
   }
 
+  @Override
+  public void deleteRange(KEY beginKey, KEY endKey, boolean includeEndKey) throws RocksDatabaseException, CodecException {
+    table.deleteRange(beginKey, endKey, includeEndKey);
+  }
+
   @Override
   public void deleteWithBatch(BatchOperation batch, KEY key) throws CodecException {
     table.deleteWithBatch(batch, key);
diff --git a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/SchemaOneDeletedBlocksTable.java b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/SchemaOneDeletedBlocksTable.java
index 913d6e30d1..2ee943ee9c 100644
--- a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/SchemaOneDeletedBlocksTable.java
+++ b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/SchemaOneDeletedBlocksTable.java
@@ -78,6 +78,11 @@ public void deleteRange(String beginKey, String endKey) throws RocksDatabaseExce
     super.deleteRange(prefix(beginKey), prefix(endKey));
   }
 
+  @Override
+  public void deleteRange(String beginKey, String endKey, boolean includeEndKey) throws RocksDatabaseException, CodecException {
+    super.deleteRange(beginKey, endKey, includeEndKey);
+  }
+
   @Override
   public boolean isExist(String key) throws RocksDatabaseException, CodecException {
     return super.isExist(prefix(key));
diff --git a/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java b/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java
index e1fc7297d4..a83f7f03bc 100644
--- a/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java
+++ b/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java
@@ -199,6 +199,25 @@ public void deleteRange(byte[] beginKey, byte[] endKey) throws RocksDatabaseExce
     db.deleteRange(family, beginKey, endKey);
   }
 
+  @Override
+  public void deleteRange(byte[] beginKey, byte[] endKey, boolean includeEndKey) throws RocksDatabaseException, CodecException {
+    if (includeEndKey) {
+      boolean found = false;
+      for (int i = 0; i < endKey.length && !found; i++) {
+        if (endKey[i] < Byte.MAX_VALUE) {
+          endKey[i]++;
+        }
+      }
+      if (!found) {
+        final byte[] tmp = new byte[endKey.length + 1];
+        System.arraycopy(endKey, 0, tmp, 0, endKey.length);
+        tmp[endKey.length - 1] = 0;
+        endKey = tmp;
+      }
+    }
+    deleteRange(beginKey, endKey);
+  }
+
   void deleteWithBatch(BatchOperation batch, CodecBuffer key) {
     if (batch instanceof RDBBatchOperation) {
       ((RDBBatchOperation) batch).delete(family, key);
diff --git a/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/Table.java b/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/Table.java
index 400547a0e3..92f309e214 100644
--- a/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/Table.java
+++ b/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/Table.java
@@ -134,13 +134,33 @@ default VALUE getReadCopy(KEY key) throws RocksDatabaseException, CodecException
   void deleteWithBatch(BatchOperation batch, KEY key) throws CodecException;
 
   /**
-   * Deletes a range of keys from the metadata store.
+   * Deletes a range of keys from this table.
    *
-   * @param beginKey start metadata key
-   * @param endKey end metadata key
+   * @param beginKey start key (inclusive)
+   * @param endKey end key (inclusive or exclusive, depending on includeEndKey)
+   * @param includeEndKey including end key?
    */
+  void deleteRange(KEY beginKey, KEY endKey, boolean includeEndKey) throws RocksDatabaseException, CodecException;
+
+  /** The same as deleteRange(beginKey, endKey, false). */
   void deleteRange(KEY beginKey, KEY endKey) throws RocksDatabaseException, CodecException;
 
+  /** Deletes all entries from this table. */
+  default void clear() throws RocksDatabaseException, CodecException {
+    final KEY beginKey;
+    final KEY endKey;
+    try (TableIterator<KEY, KEY> keyIterator = keyIterator()) {
+      if (!keyIterator.hasNext()) {
+        return;
+      }
+      beginKey = keyIterator.next();
+      keyIterator.seekToLast();
+      endKey = keyIterator.next();
+    }
+
+    deleteRange(beginKey, endKey, true);
+  }
+
   /** The same as iterator(null, KEY_AND_VALUE). */
   default KeyValueIterator<KEY, VALUE> iterator() throws RocksDatabaseException, CodecException {
     return iterator(null, IteratorType.KEY_AND_VALUE);
diff --git a/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/TypedTable.java b/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/TypedTable.java
index 59e924529c..e24ceb8d4d 100644
--- a/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/TypedTable.java
+++ b/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/TypedTable.java
@@ -397,6 +397,11 @@ public void deleteRange(KEY beginKey, KEY endKey) throws RocksDatabaseException,
     rawTable.deleteRange(encodeKey(beginKey), encodeKey(endKey));
   }
 
+  @Override
+  public void deleteRange(KEY beginKey, KEY endKey, boolean includeEndKey) throws RocksDatabaseException, CodecException {
+    rawTable.deleteRange(encodeKey(beginKey), encodeKey(endKey), includeEndKey);
+  }
+
   @Override
   public KeyValueIterator<KEY, VALUE> iterator(KEY prefix, IteratorType type)
       throws RocksDatabaseException, CodecException {
diff --git a/hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/db/InMemoryTestTable.java b/hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/db/InMemoryTestTable.java
index 1dbb502971..a76c6c0f25 100644
--- a/hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/db/InMemoryTestTable.java
+++ b/hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/db/InMemoryTestTable.java
@@ -95,6 +95,11 @@ public void deleteRange(KEY beginKey, KEY endKey) {
     map.subMap(beginKey, endKey).clear();
   }
 
+  @Override
+  public void deleteRange(KEY beginKey, KEY endKey, boolean includeEndKey) throws RocksDatabaseException, CodecException {
+    map.subMap(beginKey, true, endKey, includeEndKey).clear();
+  }
+
   @Override
   public KeyValueIterator<KEY, VALUE> iterator(KEY prefix, IteratorType type) {
     throw new UnsupportedOperationException();

[chihsuan]

Thanks for the sketch! will move it into Table as clear() with the deleteRange overload.

[echonesis]

Thanks @chihsuan for the patch.
I may be missing a production call path, but it looks like truncateTable is only called during task reprocessing.
ReconTaskControllerImpl.reInitializeTasks() first calls getStagedReconDBProvider(), which deletes the existing .staged directory and creates a new Recon DB. The staged tasks are then constructed against this newly created DB before their reprocess() methods call truncateTable.

Would these tables therefore always be empty when truncateTable is reached in production? If so, both the previous per-key deletion and the new deleteRange path would return immediately, and the populated-table test would cover a state that does not occur through the production reprocessing flow.
Is there another call path where truncateTable operates on an existing populated DB? Otherwise, would it make more sense to remove the now-redundant truncate calls/helper instead?

[chihsuan]

Is there another call path where truncateTable operates on an existing populated DB? Otherwise, would it make more sense to remove the now-redundant truncate calls/helper instead?

Thanks for checking! @echonesis Yes, I noticed the same and mentioned it on the Jira. Since HDDS-13553 the callers always get a fresh staged DB. I think it might still be worth fixing for future non-staged callers, and it's moving into Table.clear() as a general utility anyway. Removing the redundant calls is a fair point, but I'd prefer a separate Jira for that.