Skip to content

feat(test-benchmark): strengthen and consolidate TSTORE benchmark - #3441

Open
chfast wants to merge 1 commit into
ethereum:forks/amsterdamfrom
chfast:bench/tstore-unique-keys-gas
Open

feat(test-benchmark): strengthen and consolidate TSTORE benchmark#3441
chfast wants to merge 1 commit into
ethereum:forks/amsterdamfrom
chfast:bench/tstore-unique-keys-gas

Conversation

@chfast

@chfast chfast commented Aug 25, 2026

Copy link
Copy Markdown
Member

Description

Rework test_tstore into a single parametrized worst-case benchmark: key = GAS (unique slot per op, no memory counter) or PUSH0 (fixed slot), value = COINBASE (a nonzero op) so each write actually inserts an entry. A zero value lets clients skip the insert and silently hide store growth, so the test asserts the block coinbase is nonzero.

The tload parameter adds a transient read (TLOAD and TSTORE share the same 100 gas cost):

  • none: TSTORE only.
  • hit: seed the value once, store it under the key and read it back, feeding the result into the next TSTORE so the read is used, not discarded.
  • miss: store the key, then read a fresh, never-written key so TLOAD misses on the growing store; the value stays COINBASE (nonzero).

This subsumes the bloatnet test_tstore_same_key (fixed key) and test_tstore_unique_keys (unique key), which are removed along with their now-empty module.

Related Issues or PRs

N/A.

Checklist

  • Ran fast static checks to avoid CI fails, see Code Standards & Verifying Changes: just static
  • PR title has the form <type>(<area>): <title>, where <type> and <area> come from an appropriate C-<type>, respectively A-<area>, label. The title should match the target squash commit message.

Rework test_tstore into a single parametrized worst-case benchmark:
key = GAS (unique slot per op, no memory counter) or PUSH0 (fixed slot),
value = COINBASE (a nonzero op) so each write actually inserts an entry.
A zero value lets clients skip the insert and silently hide store growth,
so the test asserts the block coinbase is nonzero.

The tload parameter adds a transient read (TLOAD and TSTORE share the same
100 gas cost):
- none: TSTORE only.
- hit: seed the value once, store it under the key and read it back,
  feeding the result into the next TSTORE so the read is used, not
  discarded.
- miss: store the key, then read a fresh, never-written key so TLOAD
  misses on the growing store; the value stays COINBASE (nonzero).

This subsumes the bloatnet test_tstore_same_key (fixed key) and
test_tstore_unique_keys (unique key), which are removed along with their
now-empty module.
@chfast

chfast commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

Benchmark results — geth & besu

Filled fixture executed at Osaka, 300M gas via block-test
(evm blocktest / evmtool block-test), single machine, best-of-3 (geth) /
best-of-2 (besu).

Clients: geth evm 1.17.2-stable, besu evmtool 26.8-develop.

key tload geth MGas/s besu MGas/s
unique (GAS) none 141 39
unique (GAS) hit 270 56
unique (GAS) miss 288 56
fixed (PUSH0) none 1222 385
fixed (PUSH0) hit 1231 462
fixed (PUSH0) miss 1254 455

The unique (distinct transient slot per op) variants are ~9x slower than the
fixed baseline — the intended worst case. tload=hit/miss add a TLOAD, so
fewer TSTOREs run per unit gas and the per-tx transient store grows less,
giving higher throughput than tload=none.

Notes:

  • Single-machine, relative numbers (not hardware-normalized); besu is a locally
    built dev evmtool, so treat its absolutes as indicative.
  • Amsterdam was tried first but no client runner supports it yet (geth's t8n /
    blocktest excludes it), so these are on Osaka.

@codecov

codecov Bot commented Aug 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.77%. Comparing base (786c0d4) to head (123ddc8).
⚠️ Report is 1 commits behind head on forks/amsterdam.

Additional details and impacted files
@@                 Coverage Diff                 @@
##           forks/amsterdam    #3441      +/-   ##
===================================================
+ Coverage            93.54%   93.77%   +0.22%     
===================================================
  Files                  624      624              
  Lines                37074    36905     -169     
  Branches              3394     3326      -68     
===================================================
- Hits                 34681    34606      -75     
+ Misses                1644     1597      -47     
+ Partials               749      702      -47     
Flag Coverage Δ
unittests 93.77% <ø> (+0.22%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant