Skip to content

Deferred loading of NumPy and SciPy dependencies#1521

Open
josemmo wants to merge 3 commits into
compas-dev:mainfrom
josemmo:performance/deferred-loading
Open

Deferred loading of NumPy and SciPy dependencies#1521
josemmo wants to merge 3 commits into
compas-dev:mainfrom
josemmo:performance/deferred-loading

Conversation

@josemmo

@josemmo josemmo commented Jun 16, 2026

Copy link
Copy Markdown

Problem

import compas.geometry takes several seconds due to top-level NumPy and SciPy imports in compas.geometry and compas.data. This triggers a cascade of around 470 transitive imports at startup, even when those libraries are not used.

While the COMPAS codebase already defers heavy imports in several places123, some modules don't follow this practice, which defeats the effort made elsewhere.

Impact of the PR

Import time
Before ~2.21s
After ~0.14s
Speedup ~15x

How to reproduce

  1. Setup a blank project:
uv init
uv add compas==2.15.1 snakeviz
  1. Profile with SnakeViz:
# Clear cache first for a cold-start measurement
Get-ChildItem -Directory -Recurse -Force -Filter "__pycache__" -ErrorAction SilentlyContinue | Remove-Item -Recurse -Force -ErrorAction SilentlyContinue

uv run python -c 'import cProfile; cProfile.run("import compas.geometry", "before.prof")'
uv run snakeviz before.prof

The profile will show _io.open_code dominating runtime across 500+ file reads. This is Python pulling in source/bytecode files from NumPy/SciPy from disk.

For my particular device, it took 2.21 seconds to run import compas.geometry. Note this time may vary significant depending on the machine hardware and cache status.

  1. Capture an import time log:
uv run python -X importtime -c "import compas.geometry" 2>&1 | Out-File -FilePath before.txt -Encoding utf8

The log will contain around 470 NumPy/SciPy imports with high cumulative times. For example:

import time: self [us] | cumulative | imported package
import time:      1271 |      63580 | numpy
import time:     27278 |      39595 | numpy.testing._private.utils
import time:       507 |      49062 | numpy.testing
import time:       727 |     160679 | scipy.linalg
import time:       694 |      94077 | scipy.optimize
import time:       439 |      20286 | scipy.spatial.transform
import time:       539 |      79419 | scipy.spatial

How to verify fix

Repeat the steps above after switching to this branch:

uv add "git+https://github.com/josemmo/compas@performance/deferred-loading"

The profile should be significantly shorter (0.14 seconds in my case) and the import log should contain no NumPy or SciPy entries.

Alternatives considered

  • lazy_loader (SPEC 1): The approach used by NumPy, scikit-image, NetworkX, and other libraries. Compatible with Python 3.9+, but would require restructuring large parts of the codebase.

  • lazy keyword (PEP 810): Native language support, but requires Python 3.15+, which is outside COMPAS's current support range.

Footnotes

  1. https://github.com/compas-dev/compas/blob/e03e40975ebec56ab98b4cfaead1eedf70241cb4/src/compas/geometry/polyhedron.py#L395

  2. https://github.com/compas-dev/compas/blob/e03e40975ebec56ab98b4cfaead1eedf70241cb4/src/compas/geometry/_core/distance.py#L595

  3. https://github.com/compas-dev/compas/blob/e03e40975ebec56ab98b4cfaead1eedf70241cb4/src/compas/datastructures/mesh/mesh.py#L404

@josemmo josemmo marked this pull request as ready for review June 16, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant