Skip to content

Use FindPython3 consistently for source builds - #2830

Open
cdeil wants to merge 1 commit into
apple:mainfrom
cdeil:codex/cmake-python-discovery
Open

Use FindPython3 consistently for source builds#2830
cdeil wants to merge 1 commit into
apple:mainfrom
cdeil:codex/cmake-python-discovery

Conversation

@cdeil

@cdeil cdeil commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Modernize source-build Python discovery so CMake uses one explicitly selected interpreter and its matching development and NumPy artifacts throughout the build.

  • replace the deprecated split FindPythonInterp / FindPythonLibs lookup with FindPython3
  • request the interpreter, development, and NumPy components together
  • run kmeans, wheel, pip, and pytest commands through Python3_EXECUTABLE
  • update the regular and conda build scripts to pass the modern executable variable and let FindPython3 derive matching artifacts

This addresses the Python-discovery and virtual-environment portions of #2376.

Problem

The old build can select different Python installations for the interpreter and development artifacts. On a machine with a uv-managed Python 3.13.11 environment and Homebrew Python 3.13.15, passing only PYTHON_EXECUTABLE selected:

interpreter: .../.venv/bin/python (3.13.11)
headers:     /opt/homebrew/.../python@3.13/3.13.15/.../include/python3.13
library:     /opt/homebrew/.../python@3.13/3.13.15/.../libpython3.13.dylib

The kmeans configure step also invoked the literal python3 from PATH, independently of the selected interpreter. That can use another Python ABI or fail because that unrelated interpreter does not have the build dependencies.

Fix

CMake 3.14 provides the FindPython3 NumPy component, so this raises the minimum to 3.14 and performs one component-aware lookup:

set(Python3_FIND_VIRTUALENV FIRST)
find_package(Python3 REQUIRED COMPONENTS Interpreter Development NumPy)

All native includes and all Python build/test subprocesses then use results from that lookup. The build scripts pass only Python3_EXECUTABLE; this avoids independently supplied header and library paths drifting to another installation.

Red/green validation

Red: current main

  1. Put a python3 executable first on PATH that prints an error and exits 86.
  2. Configure with an explicitly selected virtual-environment interpreter and matching legacy include/library variables.
  3. Configuration still invoked the poisoned PATH executable from the kmeans step and failed:
unexpected PATH python3 invoked
CMake Error at CMakeLists.txt:204 (message):
  Could not build kmeans1d dependency

A second clean configure that supplied only the virtual-environment PYTHON_EXECUTABLE completed but paired its Python 3.13.11 interpreter with Homebrew Python 3.13.15 headers and library.

Green: this branch

Repeated the poisoned-PATH configure while passing only Python3_EXECUTABLE:

  • Python 3.13.11: configure succeeded and resolved the executable, headers, library, and NumPy include directory from the selected uv environment
  • Python 3.10.18: the same configure succeeded with every artifact resolved from the selected Python 3.10 environment
  • the poisoned python3 was never invoked in either case
  • cmake --build <dir> --target dist -j4 succeeded for both interpreters
  • produced correctly tagged cp313 and cp310 macOS arm64 wheels
  • each wheel installed in a fresh isolated environment and imported coremltools plus _MLModelProxy successfully

The existing build_wheel_macos_py310 CI job runs scripts/build.sh --python=3.10 --dist, so it exercises the updated discovery and complete wheel-build path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant