OAK-12426 : benchmark document cache implementations

[rishabhdaim]

Benchmark CacheLIRS, Caffeine SYNC and opt-in Caffeine ASYNC through production DocumentNodeStore caches, including LOCAL_DIFF. Keep direct-cache measurements separate from repository reads, traversal, saves, explicit diffs, concurrency and persistent-cache reopen.

Runs verify data and effective cache policy, and report completed operations, elapsed time, phase costs, cache statistics and backend reads. Warm-up, duration, cache size and folder width are configurable. Errors retain the workload context and preserve cleanup failures as suppressed exceptions. MemoryDocumentStore and EmptyHook exclude remote database latency and application/index hooks.

256 MiB results and direct-cache comparison · 384 MiB results at 10k/120k nodes.

Validation: the full 37-test module suite passed with Java 17 in isolated CacheLIRS, Caffeine SYNC and Caffeine ASYNC JVMs. Scenario methods and shared store cleanup have full line/branch coverage. Packaging and license checks passed.

OAK-12426. Depends on #3190.

Co-Authored-By: OpenAI Codex noreply@openai.com

[rishabhdaim]

TL;DR

• All eligible Document caches, including LOCAL_DIFF, use the named policy. Caffeine SYNC is 25% faster for point reads, but 28% slower for saves and 43% slower for saves + explicit diff than LIRS in these sustained runs. ASYNC is 5% faster for reads, but 47–53% slower for writes.
• Sustained revision churn exercises eviction and persistence. Explicit-diff runs make ~9.9 backend finds/operation with Caffeine versus ~8.0 with LIRS. The temporary LOCAL_DIFF LIRS exception has been removed from production.
• Direct NODE-cache microbenchmark: SYNC 1.07× LIRS; ASYNC 0.61× SYNC across 40 cases. Its access patterns differ from repository operations.

Benchmark results

DocumentNodeStore — sustained operations

256 MiB total memory cache (NODE 87.04 MiB; LOCAL_DIFF 38.4 MiB), 10k total data nodes, 100 children/group, one-second background work, persistence enabled, Java 17, 2 GiB heap. 10s warm-up + 20s measurement, two separate JVM repetitions per workload/mode; medians below. Setup/warm-up are untimed; measured stores are fresh. These completed all-Caffeine controls match the production selection after removal of the exception. MemoryDocumentStore + EmptyHook excludes remote database latency and application/index hooks.

Throughput, ops/s. M = million; k = thousand.

Workload	CacheLIRS	Caffeine SYNC	Caffeine ASYNC
Point read	3.10M	3.88M	3.25M
Save + verify	7.49k	5.37k	3.97k
Save + explicit diff	7.22k	4.13k	3.43k

16/24 MiB checks are memory-pressure diagnostics below the default budget; Caffeine writes lagged there. They are excluded from the main comparison.

Direct NODE-cache microbenchmark

Unchanged three-run medians; median ratios across matched cases. Synthetic 10k-entry NODE budget, 512-byte values, 200k warm-up operations and 2M measured operations per case.

Workload	SYNC / LIRS	ASYNC / SYNC
Steady state	1.08×	0.62×
Churn	1.02×	0.63×
Invalidation	1.05×	0.58×
Concurrent	1.11×	0.68×
All 40 cases	1.07×	0.61×

Ratios above 1 favour the numerator. ASYNC won 2/40 cases against SYNC and triggered maintenance-pool inline fallback in all three runs.

[rishabhdaim]

Implementation based on these benchmark results is available in #3176. It enables Caffeine by default behind a rollback feature toggle and raises the default document-cache allocation by 25% to 320 MB.

[rishabhdaim]

384 MiB (+50%), 10s warm-up + 20s measurement, two-run medians in LIRS/SYNC/ASYNC ops/s order: 10k nodes — reads 2.86M/3.70M/3.22M, saves 6.43k/9.08k/3.94k, saves+diff 6.11k/6.72k/4.10k; 120k nodes — reads 136.4k/163.6k/127.5k, saves 5.62k/7.86k/3.30k, saves+diff 5.35k/2.99k/2.41k (120k SYNC diff range: 2.49–3.50k ops/s).

[joerghoh]

@rishabhdaim do you have also benchmark results from a real-life environment, and not only by these synthetic tests?

[rishabhdaim]

@rishabhdaim do you have also benchmark results from a real-life environment, and not only by these synthetic tests?

@joerghoh No, I don't have that.

[sonarqubecloud]

Quality Gate failed

Failed conditions
D Security Rating on New Code (required ≥ A)
E Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE