Repository navigation
[6.x] Don't keep two copies of every entry in memory while building the Stache - #15795
Open
DanielDarrenJones wants to merge 1 commit into
Open
DanielDarrenJones wants to merge 1 commit into
DanielDarrenJones wants to merge 1 commit into
Conversation
…he Stache While indexes are built, makeItemFromFile() keeps every entry it makes in Blink's `structure-entries`, and getCachedItem() then unserialises a second full copy of each one from the cache. On a cold warm of a large site this holds every entry twice. While an index is loading, hand out a shallow clone of the entry already in memory instead, clearing the same per-object caches that Entry::__sleep() drops. Callers still get their own object, and its data is only copied if they change it. Outside index loading nothing changes.
DanielDarrenJones
force-pushed
the
perf/6x-reuse-loaded-entry
branch
from
November 5, 2026 17:18
24d0e5b to
7fc9ceb
Compare
DanielDarrenJones
marked this pull request as ready for review
November 5, 2026 17:25
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While indexes are built,
CollectionEntriesStore::makeItemFromFile()keeps every entry it makes in Blink'sstructure-entries(so URI building can find entries that aren't in the Stache yet).getCachedItem()then unserialises a second, full copy of each entry from the cache. On a cold warm, a terms index that loads every collection's entries ends up holding all of them twice.This changes
getCachedItem()so that, only while an index is loading (Index::isLoading()), it hands out a shallow clone of the entry already in memory instead of unserialising another copy. The clone has the same per-object caches cleared thatEntry::__sleep()leaves out of the cached copy (cachedKeys,computedCallbackCache,siteCache,augmentationReferenceKey), so callers get the same object they would have got from the cache, and its data arrays are only copied if they change it. Outside index loading nothing changes.Cold
Stache::warm()on 6.40.0, using a production site's content (4,111 entries across 11 sites, 113 MB of YAML; median of 3 runs, Apple M1 Pro, PHP 8.4, file cache store):%%{init: {"xyChart": {"width": 700, "height": 220}}}%% xychart-beta horizontal title "Peak memory, cold Stache warm (MB)" x-axis ["6.40.0", "This PR"] y-axis "MB" 0 --> 1800 bar [1710, 841]%%{init: {"xyChart": {"width": 700, "height": 220}}}%% xychart-beta horizontal title "Time, cold Stache warm (seconds)" x-axis ["6.40.0", "This PR"] y-axis "Seconds" 0 --> 55 bar [49.5, 41.3]Every entry's
uri()andurl()are identical with and without the change (all 4,111). Handing out the clone at all times (not just while loading) gets to 754 MB, but limiting it to index loading keeps normal requests exactly as they are.How this differs from #3984
#3984 (3.4, closed in 2024 without being merged) also used Blink and clones, but for a different problem:
BasicStore, so every store, on every request. This only touchesCollectionEntriesStore, and only while an index is loading.makeItemFromFile()already keeps instructure-entries, instead of unserialising a second one.clone, which keeps the object's memoised computed values and site cache. This clears exactly whatEntry::__sleep()drops, so the clone behaves like a copy read from the cache.assertSametoassertEquals. Here, normal requests are unchanged and one strict Blink mock needed to allow the extraget().On a cache miss,
BasicStore::getItem()already returns the very instance it put in Blink, so sharing in-memory entries isn't new; this change hands out clones rather than sharing.Tests: three new tests in
EntriesStoreTest(two fail without the change; the third checks that reads outside index loading still come from the cache).EntryTest::it_clears_blink_caches_when_savingmocks Blink strictly, so it now also allows theget()that indexes loaded during the save make; it still requires the entry to be forgotten and put again. The full 6.x suite passes (11,464 passed, 12 skipped, as on 6.x).