Repository navigation
perf: import clientwright and capabilities_matrix() without loading asyncio - #63
Merged
Merged
Conversation
…syncio core.policy.concurrency imported asyncio at module level and sits on the import path of the package root, so every `import clientwright` paid for asyncio, ssl, socket and subprocess. AsyncOriginLimiter now imports asyncio where it creates a semaphore, inside a running loop. capabilities_matrix() loaded the httpx and httpx2 records through _httpx_shared, which imports asyncio, ssl and both engines. The family's capability record now lives in _httpx_capabilities, which needs only the core's capability and model types. Closes #62
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
import clientwrightloaded asyncio, and ssl, socket and subprocess with it, although nothing at import time needs them. Two modules caused it:core/policy/concurrency.pyimported asyncio at module level, and it is on the import path ofcore.planand the package root.AsyncOriginLimiteruses asyncio only in_semaphore(), which runs inside a running loop, so the import moved there. The annotations get asyncio underTYPE_CHECKING. The constructor does not touch asyncio, soClientRuntimestill builds both limiters eagerly and its construction is unchanged.capabilities_matrix()loaded the httpx and httpx2 records fromadapters/{httpx,httpx2}/capabilities.py. Those get the record fromcapabilities_for()in_httpx_shared, which imports asyncio, ssl, both engines andcore.plan. The function moved verbatim (diffed against master) to a new privateadapters/_httpx_capabilities.pythat imports onlycore.capabilitiesandcore.model. Like_httpx_shared, it sits outside the independence contract. Nothing importedcapabilities_forfrom_httpx_sharedany more, so it is gone from that module's__all__. The module is private.No public name, signature or behaviour changes. The root
__all__is untouched.Numbers
Median of 15 interleaved runs of master and this branch. "importtime" is the cumulative
clientwrightline ofpython -X importtime -c "import clientwright". "scenario" is the wall time ofimport clientwright+ClientConfig(...)+capabilities_matrix(), the configuration-only job from the issue.python:3.13-slimcontainer, bare (no extras), bytecode precompiledThe machine is faster than the one in the issue, but the share asyncio took is the same: about a third of the import. With this change,
asyncio,ssl,socketandsubprocessare not insys.modulesafter the import, and they stay out aftercapabilities_matrix().Rejected
__getattr__forClientRuntime,ClientHandleand the rest). It hurts static typing and readability, and those classes are not what costs time.AsyncOriginLimiterintocore/engine/aio.pyand building it only for async clients. That changes howClientRuntimeis constructed, which is more churn for the same result.sys.modulescheck is deterministic and catches exactly this regression.Tests
Two probes in
tests/unit/adapters/test_bare_install.pyrun in a subprocess with every extra blocked, like the existing bare-install probes:test__import_clientwright__does_not_import_asynciotest__config_and_capabilities_matrix__do_not_import_asyncioBoth fail on master (
asyncio loaded) and pass here. The async limiter tests intests/unit/core/policy/test_concurrency.pyand the adapter suites cover the deferred import at runtime. I also ran a manual check: 5 concurrent requests throughbuild("httpx")andbuild("aiohttp")withmax_connections_per_host=1againstOriginServerall returned 200.Local results:
make check: ruff, format, mypy (100 files) and lint-imports (3 contracts kept) all clean.make test: 875 passed, 42 skipped, coverage 99.61%.uv sync --group test,pytest -m unit): 417 passed, 45 skipped.Docs
docs/learn/install.md: the note on the bare install now says that the import and the matrix do not load asyncio.docs/advanced/writing-an-adapter.md: a third-party capabilities module should stay light, becausecapabilities_matrix()imports every registered record.docs/advanced/architecture.md, thepyproject.tomlcomment and.agents/project.mdnow mention the new private module next to_httpx_shared.Type of change
Checklist
make check && make testpass locallydocs/agents.mdincluded. The public API did not change, sodocs/agents.mdis untouched.Related issues
Closes #62