[Question] Database size increases significantly after sync compared to snapshot

[NabiKAZ]

I have a question about database size optimization.

A snapshot of the TRON Lite FullNode database was around 70 GB. After syncing new blocks for a few days, the database size increased to around 100 GB.

• 70 GB from day one until now
• Two or three days of syncing 30 GB!!!

Why does the synced database become much larger than the snapshot? Is the snapshot generated after compaction or with some optimization process?

Is there any recommended method to reduce the size of an existing Lite FullNode database (for example, RocksDB compaction or cleanup be like snapshot existed.)?
Meanwhile, I only need the wallet balances.

[GrothenDI]

The initial snapshot contains only the latest entire network state and the most recent 65,536 blocks. Assuming that the size of the entire network state remains essentially unchanged and that the additional 30 GB comes entirely from the accumulated size of newly synchronized blocks, an increase of 30 GB within two or three days is completely normal. Given an average block size of approximately 0.1 MB, a 30 GB increase over two days corresponds to a synchronization speed of about 1.77 blocks per second. With the recommended hardware configuration, the synchronization speed is approximately 4 blocks per second.

1. To minimize the disk space occupied by database, do not enable the vm.saveInternalTx configuration when start your node. When enabled, this option stores internal transactions, which increases the database size. It is disabled by default.

2. If you are only interested in latest account balances, you can periodically prune the database. For details, see the lite-fullnode-maintenance. The node must be stopped before pruning

3. If you need historical account balances, make sure that storage.balance.history.lookup is enabled during synchronization, this option is disabled by default. You should also exercise caution when pruning the node database, because only the most recent 65,536 blocks will be retained after pruning, queries for an account's balance at a block height outside the retained range will fail.

[waynercheung]

The growth is expected for a Lite FullNode, although the additional 30 GB should be verified against the block-height difference and node configuration.

A Lite FullNode snapshot is a pruned dataset containing the latest network state and the most recent 65,536 blocks. It is not simply a fully compacted database. After startup, the node continues to store new blocks, indexes, transaction data, and transaction results. Since old history is not automatically pruned, the database grows over time similarly to a regular FullNode. While the node is catching up a backlog, daily growth can be several times higher than the chain's real-time rate; once fully synchronized, it should fall closer to the real-time accumulation rate.

Compaction can reclaim obsolete database versions and storage overhead, but it cannot remove valid historical blocks or transaction data. Therefore, compaction alone cannot reduce the database to snapshot size. The Toolkit does not provide a general operator-facing db compact command; the supported way to reduce the database is Lite pruning.

Please check the block-height difference and database usage:

du -sh output-directory/database/* | sort -h

Also check:

• vm.saveInternalTx
• vm.saveFeaturedInternalTx
• storage.transHistory.switch
• storage.balance.history.lookup
• whether the snapshot and database sizes use the same format, such as extracted versus compressed data

If only current account balances are required, consider disabling:

  storage {
    transHistory.switch = "off"
    balance.history.lookup = false
  }

  vm {
    saveInternalTx = false
    saveFeaturedInternalTx = false
  }

With storage.transHistory.switch = "off", new transaction result records are not stored. Consequently, transaction-info-based APIs such as gettransactioninfobyid, eth_getTransactionReceipt, and historical eth_getLogs may not return data for transactions processed while it is disabled. Existing records are not deleted automatically.

To reduce an existing database, stop the node and run:

  java -jar Toolkit.jar db lite \
    -o split \
    -t snapshot \
    --fn-data-path /path/to/output-directory/database \
    --dataset-path /path/to/prune-output \
    --exclude-historical-balance

The command creates a new snapshot dataset and requires additional free disk space. Keep the original database until the generated snapshot has been validated, then replace the node's database directory with the generated snapshot directory.

--exclude-historical-balance removes the balance-trace and account-trace databases. Use it only when historical balance queries are not required. It is available since GreatVoyage-v4.8.2; omit it on older versions.

After pruning, ordinary block and transaction history outside the latest 65,536-block window will not be available. Current account state and balances remain available, but historical balance queries require the trace databases to be retained. Periodic pruning is required, and the node must be stopped during the operation.