Skip to content

[6.x] Don't keep two copies of every entry in memory while building the Stache - #15795

Open
DanielDarrenJones wants to merge 1 commit into
statamic:6.xfrom
DanielDarrenJones:perf/6x-reuse-loaded-entry
Open

DanielDarrenJones wants to merge 1 commit into
statamic:6.xfrom
DanielDarrenJones:perf/6x-reuse-loaded-entry

Conversation

@DanielDarrenJones

@DanielDarrenJones DanielDarrenJones commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

While indexes are built, CollectionEntriesStore::makeItemFromFile() keeps every entry it makes in Blink's structure-entries (so URI building can find entries that aren't in the Stache yet). getCachedItem() then unserialises a second, full copy of each entry from the cache. On a cold warm, a terms index that loads every collection's entries ends up holding all of them twice.

This changes getCachedItem() so that, only while an index is loading (Index::isLoading()), it hands out a shallow clone of the entry already in memory instead of unserialising another copy. The clone has the same per-object caches cleared that Entry::__sleep() leaves out of the cached copy (cachedKeys, computedCallbackCache, siteCache, augmentationReferenceKey), so callers get the same object they would have got from the cache, and its data arrays are only copied if they change it. Outside index loading nothing changes.

Cold Stache::warm() on 6.40.0, using a production site's content (4,111 entries across 11 sites, 113 MB of YAML; median of 3 runs, Apple M1 Pro, PHP 8.4, file cache store):

time peak memory
6.40.0 49.5 s 1,710 MB
this PR 41.3 s (−17%) 841 MB (−51%)
%%{init: {"xyChart": {"width": 700, "height": 220}}}%%
xychart-beta horizontal
    title "Peak memory, cold Stache warm (MB)"
    x-axis ["6.40.0", "This PR"]
    y-axis "MB" 0 --> 1800
    bar [1710, 841]
Loading
%%{init: {"xyChart": {"width": 700, "height": 220}}}%%
xychart-beta horizontal
    title "Time, cold Stache warm (seconds)"
    x-axis ["6.40.0", "This PR"]
    y-axis "Seconds" 0 --> 55
    bar [49.5, 41.3]
Loading

Every entry's uri() and url() are identical with and without the change (all 4,111). Handing out the clone at all times (not just while loading) gets to 754 MB, but limiting it to index loading keeps normal requests exactly as they are.

How this differs from #3984

#3984 (3.4, closed in 2024 without being merged) also used Blink and clones, but for a different problem:

On a cache miss, BasicStore::getItem() already returns the very instance it put in Blink, so sharing in-memory entries isn't new; this change hands out clones rather than sharing.

Tests: three new tests in EntriesStoreTest (two fail without the change; the third checks that reads outside index loading still come from the cache). EntryTest::it_clears_blink_caches_when_saving mocks Blink strictly, so it now also allows the get() that indexes loaded during the save make; it still requires the entry to be forgotten and put again. The full 6.x suite passes (11,464 passed, 12 skipped, as on 6.x).

…he Stache

While indexes are built, makeItemFromFile() keeps every entry it makes in Blink's
`structure-entries`, and getCachedItem() then unserialises a second full copy of
each one from the cache. On a cold warm of a large site this holds every entry
twice.

While an index is loading, hand out a shallow clone of the entry already in
memory instead, clearing the same per-object caches that Entry::__sleep() drops.
Callers still get their own object, and its data is only copied if they change
it. Outside index loading nothing changes.
@DanielDarrenJones
DanielDarrenJones force-pushed the perf/6x-reuse-loaded-entry branch from 24d0e5b to 7fc9ceb Compare November 5, 2026 17:18
@DanielDarrenJones
DanielDarrenJones marked this pull request as ready for review November 5, 2026 17:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant