Repository navigation
OAK-12452: Reduce allocation and contention in the segment ReaderCache - #3207
Open
lweitzendorf wants to merge 4 commits into
Open
lweitzendorf wants to merge 4 commits into
lweitzendorf wants to merge 4 commits into
Conversation
added 4 commits
October 8, 2026 14:30
ReaderCache.get() allocated a FastCacheEntry on every fast-cache miss, including first-time loads. Compaction's sequential tree scan reads most keys exactly once, producing a large volume of single-use FastCacheEntry allocations. An entry is now promoted to the fast cache only after a slow-cache hit proves reuse. JMH (JDK 23, -prof gc), read-once access pattern against a CacheLIRS slow tier: eager (old): 112 B/op, ~124 ns/op deferred (new): 72 B/op, ~109 ns/op On reuse-heavy patterns both strategies converge.
When the slow-tier weight is non-positive, get() now bypasses the slow cache entirely instead of creating a key, probing, inserting and immediately evicting on every fast-cache miss. JMH, ReaderCache.get fast-cache-miss path, 48 threads, -prof gc: before: 24.3 ops/us, 171.6 B/op after: 178.3 ops/us, 110.4 B/op
Builds the ReaderCache slow tier with the Caffeine-backed Oak CacheBuilder instead of CacheLIRS (stats recording always on). CacheLIRS and its other usages are unchanged. Drops the unused averageWeight constructor parameter and updates StringCache/TemplateCache accordingly. ReaderCacheTest's LIRS-specific hit-rate assertion is made implementation-neutral. In a multi-threaded read-heavy workload (48 threads) end-to-end throughput was at parity with CacheLIRS.
- **`CacheKey` and `FastCacheEntry` are now records** — plain immutable value holders. The redundant cached `hash` field is removed from both: `CacheKey.hashCode()` recomputes `getEntryHash` (a few int ops, only on a slow-tier probe), and `FastCacheEntry` never read its stored hash. - **`FastCache` now owns its hash computation and entry allocation**: `get`/`put` derive the array index from `(msb, lsb, offset)` internally instead of the caller precomputing a hash and threading it through both calls, and `put` builds the `FastCacheEntry` itself. Isolated and end-to-end microbenchmarks confirm recomputing the hash is free (within measurement noise) — nothing was worth manually caching.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://issues.apache.org/jira/browse/OAK-12452
Summary
Reduces allocation and contention in the segment
ReaderCache(used byStringCacheandTemplateCache). The PR has four commits that can be reviewed one at a time:get()used to allocate aFastCacheEntryon every fast-tier miss, including first-time loads. Compaction's sequential scan reads most keys exactly once, so this produced many single-use allocations. Entries are now promoted only on a slow-tier hit.CacheBuilderinstead ofCacheLIRS, as was done forSegmentCacheandRecordCachein OAK-12157.CacheLIRSand its other uses are unchanged apart from two diamond-operator cleanups. Also drops the unusedaverageWeightconstructor parameter.CacheKeyandFastCacheEntrybecome records, the cached hash fields are removed, andFastCachecomputes its own index.Benchmarks (JMH 1.37, JDK 23,
-prof gc)For change 3, end-to-end throughput in a 48-thread read-heavy workload was at parity with
CacheLIRS.There is no feature toggle, following the precedent of the
SegmentCachemigration. Happy to add one for changes 1 and 2 if reviewers prefer.Tests
ReaderCacheTestis updated: its LIRS-specific hit-rate assertion is now implementation-neutral,largeEntriesis replaced byfastOnlyLargeValueReDecoded(weight-0 path) andlargeEntryServedFromSlowCache. Existing tests cover the deferred promotion.