Skip to content

Add opt-in generator compile cache - #9266

Open
alexreinking wants to merge 2 commits into
mainfrom
alexreinking/generator-cache
Open

Add opt-in generator compile cache#9266
alexreinking wants to merge 2 commits into
mainfrom
alexreinking/generator-cache

Conversation

@alexreinking

@alexreinking alexreinking commented Jul 31, 2026

Copy link
Copy Markdown
Member

When HL_CACHE_DIR is set, execute_generator caches the artifacts a generator emits and restores them instead of recompiling when the inputs are unchanged. The cache key combines:

  • a compiler-identity fingerprint: the linker-assigned build id, read directly from the already-loaded libHalide image (Mach-O LC_UUID on macOS, ELF .note.gnu.build-id via dl_iterate_phdr on Linux), so it changes on any relink without hashing the ~32 MB binary; falls back to a full-binary hash when no build id is available (e.g. --build-id=none, or Windows, not yet implemented)
  • the target(s), output types, and generator-param settings
  • the serialized pipeline for each target, which is what makes the key sound: it captures the actual algorithm and schedule, so editing a generator's source invalidates the cache even when its command-line params are unchanged
  • the contents of any loaded -p plugins (e.g. autoschedulers), since they affect codegen but live outside libHalide

Both the generator (compile_multitarget) and standalone-runtime (compile_standalone_runtime) paths are cached. Entries are installed atomically (temp dir + rename) so concurrent builds never see a partial entry. The cache is pruned to honor HL_CACHE_MAX_SIZE (default 1 GiB) and optional HL_CACHE_MAX_AGE, debounced to at most once per 60s via a .last_prune stamp so a large parallel build's many generator processes don't each walk the whole cache; the size limit is thus a soft cap. The whole feature is gated on WITH_SERIALIZATION and is completely inert when HL_CACHE_DIR is unset.

The shipped CMake helpers gain a Halide_CACHE_DIR cache variable (defaulting to $ENV{HL_CACHE_DIR}) and wrap every generator/GenRT invocation so the value reaches the generator process. When it is empty the emitted commands are unchanged.

Measured on the app suite (~309 generator processes): using the build id instead of a full-binary hash halves the cache-hit generator time (~46s -> ~22s of summed edge time), taking the generator-phase speedup from ~2.2x to ~5.3x versus a full recompile.

Tests: test/correctness/generator_cache.cpp re-execs itself per generation and tampers a cached blob to prove artifacts are restored from the cache (a real hit only happens across separate processes, since Halide's global unique-name counter advances within a process). test/integration/cache verifies the shipped helpers route HL_CACHE_DIR to the generator and that a build sharing the cache still links and runs.

Breaking changes

None.

Checklist

  • Tests added or updated (not required for docs, CI config, or typo fixes)
  • Documentation updated (if public API changed)
  • Commits include AI attribution where applicable (see Code of Conduct)

alexreinking and others added 2 commits July 31, 2026 17:46
When HL_CACHE_DIR is set, execute_generator caches the artifacts a
generator emits and restores them instead of recompiling when the
inputs are unchanged. The cache key combines:

- a compiler-identity fingerprint: the linker-assigned build id, read
  directly from the already-loaded libHalide image (Mach-O LC_UUID on
  macOS, ELF .note.gnu.build-id via dl_iterate_phdr on Linux), so it
  changes on any relink without hashing the ~32 MB binary; falls back
  to a full-binary hash when no build id is available (e.g.
  --build-id=none, or Windows, not yet implemented)
- the target(s), output types, and generator-param settings
- the serialized pipeline for each target, which is what makes the
  key sound: it captures the actual algorithm and schedule, so editing
  a generator's source invalidates the cache even when its
  command-line params are unchanged
- the contents of any loaded -p plugins (e.g. autoschedulers), since
  they affect codegen but live outside libHalide

Both the generator (compile_multitarget) and standalone-runtime
(compile_standalone_runtime) paths are cached. Entries are installed
atomically (temp dir + rename) so concurrent builds never see a
partial entry. The cache is pruned to honor HL_CACHE_MAX_SIZE (default
1 GiB) and optional HL_CACHE_MAX_AGE, debounced to at most once per
60s via a .last_prune stamp so a large parallel build's many generator
processes don't each walk the whole cache; the size limit is thus a
soft cap. The whole feature is gated on WITH_SERIALIZATION and is
completely inert when HL_CACHE_DIR is unset.

The shipped CMake helpers gain a Halide_CACHE_DIR cache variable
(defaulting to $ENV{HL_CACHE_DIR}) and wrap every generator/GenRT
invocation so the value reaches the generator process. When it is
empty the emitted commands are unchanged.

Measured on the app suite (~309 generator processes): using the build
id instead of a full-binary hash halves the cache-hit generator time
(~46s -> ~22s of summed edge time), taking the generator-phase speedup
from ~2.2x to ~5.3x versus a full recompile.

Tests: test/correctness/generator_cache.cpp re-execs itself per
generation and tampers a cached blob to prove artifacts are restored
from the cache (a real hit only happens across separate processes,
since Halide's global unique-name counter advances within a process).
test/integration/cache verifies the shipped helpers route HL_CACHE_DIR
to the generator and that a build sharing the cache still links and
runs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add doc/GeneratorCache.md covering the cache key composition, CLI and
CMake usage, and pruning/maintenance behavior, and link it from
README.md, HalideCMakePackage.md, and BuildingHalideWithCMake.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@alexreinking
alexreinking requested a review from abadams July 31, 2026 21:53
@codecov

codecov Bot commented Aug 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 56.63265% with 170 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.22%. Comparing base (72dd07b) to head (286006f).
⚠️ Report is 8 commits behind head on main.

Files with missing lines Patch % Lines
src/GeneratorCache.cpp 54.04% 107 Missing and 35 partials ⚠️
src/Generator.cpp 66.26% 19 Missing and 9 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9266      +/-   ##
==========================================
- Coverage   70.27%   70.22%   -0.05%     
==========================================
  Files         255      256       +1     
  Lines       78727    79117     +390     
  Branches    18857    18947      +90     
==========================================
+ Hits        55323    55563     +240     
- Misses      17805    17930     +125     
- Partials     5599     5624      +25     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alexreinking alexreinking added the release_notes For changes that may warrant a note in README for official releases. label Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release_notes For changes that may warrant a note in README for official releases.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant