Skip to content

Collect a module's functions once and stop walking it per function - #79

Merged
SizzleUnrlsd merged 2 commits into
mainfrom
perf/module-functions
Sep 5, 2026
Merged

SizzleUnrlsd merged 2 commits into
mainfrom
perf/module-functions

Conversation

@SizzleUnrlsd

Copy link
Copy Markdown
Contributor

Summary

Part of #69: measurements on large projects and the hot spot they revealed.

  • Measured (single process, cold, bundled plugins): healthchecks, 45k lines, 9.5 s and 107 MB; rich, 52k lines, 7 s and 114 MB; wagtail, 273k lines, 123 s and 470 MB. A warm --cache brings healthchecks to 1.1 s; --jobs 4 to 8 s.
  • Profiling healthchecks showed the taint engine re-collecting every analysable function of the module for each function analysed: the enclosing-function search returned None both for "top-level" and "not found", so every named function fell through to the lambda fallback, and the collection itself was never memoised. About a quarter of the cold time.
  • Acceptance tests first (test: define the module function collection as computed once), then the fix: analyzable_functions is memoised per module (identity-keyed, bounded), and _enclosing distinguishes a top-level definition from an absent one. healthchecks drops from 9.5 s to 7 s, rich from 7.2 s to 5.8 s, wagtail from 123 s to 85 s, same findings.
  • healthchecks (pinned) joins the regression corpus, so a real 45k-line project is analysed on every change; the usage guide gains a Performance section with the measurements.
  • The measurements also surfaced precision and coverage gaps (walrus, yield from and starred targets rejecting whole files; hex digests flooding high-entropy-string; credentials in test fixtures; Django redirect("name", arg)); they are recorded as tasks on Precision: nonlocal write-back and a broader regression corpus #71.

Part of #69.

@SizzleUnrlsd SizzleUnrlsd self-assigned this Sep 5, 2026
@SizzleUnrlsd
SizzleUnrlsd merged commit 7ee50ab into main Sep 5, 2026
10 checks passed
@SizzleUnrlsd
SizzleUnrlsd deleted the perf/module-functions branch September 5, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant