Skip to content

perf: import clientwright and capabilities_matrix() without loading asyncio - #63

Merged
AlexeyShalaev merged 1 commit into
masterfrom
perf/62-lazy-asyncio
Oct 16, 2026
Merged

AlexeyShalaev merged 1 commit into
masterfrom
perf/62-lazy-asyncio

Conversation

@AlexeyShalaev

Copy link
Copy Markdown
Member

Summary

import clientwright loaded asyncio, and ssl, socket and subprocess with it, although nothing at import time needs them. Two modules caused it:

  • core/policy/concurrency.py imported asyncio at module level, and it is on the import path of core.plan and the package root. AsyncOriginLimiter uses asyncio only in _semaphore(), which runs inside a running loop, so the import moved there. The annotations get asyncio under TYPE_CHECKING. The constructor does not touch asyncio, so ClientRuntime still builds both limiters eagerly and its construction is unchanged.
  • capabilities_matrix() loaded the httpx and httpx2 records from adapters/{httpx,httpx2}/capabilities.py. Those get the record from capabilities_for() in _httpx_shared, which imports asyncio, ssl, both engines and core.plan. The function moved verbatim (diffed against master) to a new private adapters/_httpx_capabilities.py that imports only core.capabilities and core.model. Like _httpx_shared, it sits outside the independence contract. Nothing imported capabilities_for from _httpx_shared any more, so it is gone from that module's __all__. The module is private.

No public name, signature or behaviour changes. The root __all__ is untouched.

Numbers

Median of 15 interleaved runs of master and this branch. "importtime" is the cumulative clientwright line of python -X importtime -c "import clientwright". "scenario" is the wall time of import clientwright + ClientConfig(...) + capabilities_matrix(), the configuration-only job from the issue.

Where master this PR
python:3.13-slim container, bare (no extras), bytecode precompiled importtime 30.4 ms 19.3 ms
scenario 30.6 ms 19.4 ms
macOS arm64, Python 3.13.14, dev venv importtime 24.7 ms 17.0 ms
scenario 26.9 ms 17.4 ms

The machine is faster than the one in the issue, but the share asyncio took is the same: about a third of the import. With this change, asyncio, ssl, socket and subprocess are not in sys.modules after the import, and they stay out after capabilities_matrix().

Rejected

  • Lazy attributes at the package root (module __getattr__ for ClientRuntime, ClientHandle and the rest). It hurts static typing and readability, and those classes are not what costs time.
  • Moving AsyncOriginLimiter into core/engine/aio.py and building it only for async clients. That changes how ClientRuntime is constructed, which is more churn for the same result.
  • An import-time threshold in CI. Shared runners are too noisy to gate on. A sys.modules check is deterministic and catches exactly this regression.

Tests

Two probes in tests/unit/adapters/test_bare_install.py run in a subprocess with every extra blocked, like the existing bare-install probes:

  • test__import_clientwright__does_not_import_asyncio
  • test__config_and_capabilities_matrix__do_not_import_asyncio

Both fail on master (asyncio loaded) and pass here. The async limiter tests in tests/unit/core/policy/test_concurrency.py and the adapter suites cover the deferred import at runtime. I also ran a manual check: 5 concurrent requests through build("httpx") and build("aiohttp") with max_connections_per_host=1 against OriginServer all returned 200.

Local results:

  • make check: ruff, format, mypy (100 files) and lint-imports (3 contracts kept) all clean.
  • make test: 875 passed, 42 skipped, coverage 99.61%.
  • No extras (uv sync --group test, pytest -m unit): 417 passed, 45 skipped.

Docs

  • docs/learn/install.md: the note on the bare install now says that the import and the matrix do not load asyncio.
  • docs/advanced/writing-an-adapter.md: a third-party capabilities module should stay light, because capabilities_matrix() imports every registered record.
  • docs/advanced/architecture.md, the pyproject.toml comment and .agents/project.md now mention the new private module next to _httpx_shared.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Refactoring / internal (performance)

Checklist

  • Tests added or updated
  • make check && make test pass locally
  • Conventional-commit title (the changelog is generated by Release Please — do not edit it by hand)
  • Documentation updated (if the public API changed), docs/agents.md included. The public API did not change, so docs/agents.md is untouched.

Related issues

Closes #62

…syncio

core.policy.concurrency imported asyncio at module level and sits on the
import path of the package root, so every `import clientwright` paid for
asyncio, ssl, socket and subprocess. AsyncOriginLimiter now imports asyncio
where it creates a semaphore, inside a running loop.

capabilities_matrix() loaded the httpx and httpx2 records through
_httpx_shared, which imports asyncio, ssl and both engines. The family's
capability record now lives in _httpx_capabilities, which needs only the
core's capability and model types.

Closes #62
@AlexeyShalaev
AlexeyShalaev merged commit 02a2827 into master Oct 16, 2026
9 checks passed
@AlexeyShalaev
AlexeyShalaev deleted the perf/62-lazy-asyncio branch October 16, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

import clientwright takes ~190 ms on Linux; ~80 ms of it is asyncio, pulled in eagerly by core.policy.concurrency

1 participant