Skip to content

Defer the imports no module-level statement reads - #555

Closed
jaraco wants to merge 1 commit into
mainfrom
feature/lazy-imports
Closed

jaraco wants to merge 1 commit into
mainfrom
feature/lazy-imports

Conversation

@jaraco

@jaraco jaraco commented Oct 31, 2026

Copy link
Copy Markdown
Member

Closes #554.

Declares __lazy_modules__ (PEP 810) and moves two imports into the functions that read them.

Approach

PEP 810 landed in 3.15 as a module global, not as syntax: a module that declares __lazy_modules__ gets lazy semantics for plain import statements. Being just a global, it is inert on 3.11–3.14 and needs no guard — so this costs nothing in the stdlib copy and nothing at sync time.

The declaration lists only what is observably deferred, so the test can assert all of it:

  • contextlib, email, ._collections, .compat — read only inside functions.
  • collections and collections.abc qualify by the same rule but are left off, because typing imports them eagerly through TypeVar; declaring them would promise a deferral that does not happen. The declaration's docstring says so, since that is the trap for the next person adding to the list.

platform (read only by _compat.pypy_partial) and compat.py312 (read only by two _compat.localize functions) need no lazy import at all, just a function-local one in the idiom this codebase already uses for importlib.metadata, _adapters, and inspect. That part helps on every supported Python.

Not done here

re, textwrap, and pathlib are the expensive ones, and each is held eager by a single module-level statement — EntryPoint.pattern = re.compile(...), Sectioned._sample = textwrap.dedent(...), and class PackagePath(pathlib.PurePosixPath). Deferring the first two is worth roughly another 14%, but EntryPoint.pattern is a documented class attribute, so it needs an API decision (and CPython's agreement); #554 has the detail. PackagePath's base class makes pathlib permanently eager.

Why not the backport

@pganssle's lazy-importer gives these semantics on 3.11+, and I measured it before settling on the native-only approach: importing lazy_importer costs ~3.7 ms on 3.11, more than the ~3.1 ms available to save, so it made import slower on every Python it applied to (+11% in the deferrable-today configuration). It would also re-indent every lazy import into a with block, which each sync would have to undo. #554 has the numbers. The declaration is in place if a cheaper backport appears.

Verification

tox -e py,py311,py312,py313,py314 — 262 passed on each, mypy clean, coverage 99% with no new gaps; tox -e docs clean. The project's own pytest-perf exercise, per env:

import time
3.11 −8%
3.12 −3%
3.13 −4%
3.14 −8%
3.15 −30%

The new test asserts that nothing in __lazy_modules__ is in sys.modules after a fresh import importlib_metadata, comparing against a snapshot taken before the import so a local sitecustomize cannot mask a regression. It skips below 3.15, where the declaration is inert.

🤖 Generated with Claude Code

Python 3.15 honors __lazy_modules__ (PEP 810), so declaring it defers
contextlib, email, and two of the package's own modules; earlier Pythons
ignore the declaration. platform and compat.py312 move into the functions
that read them, which helps on every Python.

Cuts import time ~30% on 3.15 and 3-8% on 3.11 through 3.14.

Closes #554

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jaraco

jaraco commented Oct 31, 2026

Copy link
Copy Markdown
Member Author

Closing: the scope was the wrong way round. The aim is not to cut import time but to restore these imports to module scope for readability, using lazy imports to hold performance roughly where it is. Reworking against that in a fresh PR.

@jaraco jaraco closed this Oct 31, 2026
@jaraco
jaraco deleted the feature/lazy-imports branch October 31, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Restore the deferred imports to module scope with PEP 810 lazy imports

1 participant