Skip to content

Don't touch Python state from destructors that outlive the interpreter - #601

Open
alexander-beedie wants to merge 1 commit into
duckdb:mainfrom
alexander-beedie:fix/import-cache-shutdown-abort
Open

Don't touch Python state from destructors that outlive the interpreter#601
alexander-beedie wants to merge 1 commit into
duckdb:mainfrom
alexander-beedie:fix/import-cache-shutdown-abort

Conversation

@alexander-beedie

@alexander-beedie alexander-beedie commented Aug 21, 2026

Copy link
Copy Markdown

Fixes #531.

(I have a very similar issue: getting "gilstate_tss_set" crashes after running xdist pytests that use DuckDB and register custom UDFs, so was actively looking into this).

Problem

DuckDBPyModuleState is a function-local static, so its members can be destroyed after the interpreter is unusable; two destructors touch Python state from there:

  • DefaultConnectionHolder::~DefaultConnectionHolder drops a shared_ptr<DuckDBPyConnection>, whose destructor enters nb::gil_scoped_releasePyEval_SaveThread.

  • PythonImportCache::~PythonImportCache acquires the GIL to clear its refs.

Both abort or segfault at shutdown (depending how the process ends).

Fix

Neither destructor should touch the interpreter once it is unreachable. So, the default connection is now leaked deliberately, and the import cache releases its references without freeing them. This cleanly resolves the crashes.

Tests

Added three new test cases in tests/fast/test_module.py that can exercise each of the previously-crashing paths; each gets run by a subprocess.

  • Deleting _clean_default_connection forces teardown from static destruction after Py_Finalize.
  • Exiting via exit(0) / ExitProcess(0) with no GIL held and no Py_Finalize.
  • A normal exit still clears the import cache, so the nanobind leak check stays quiet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Crash when terminating process if Py_Finalize isn't called

1 participant