Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` option (cmake, C++ define) - #4216

Merged

rwgk merged 34 commits into

pybind:masterfrom

Chekov2k:pypy_gil

Oct 30, 2022

Commits on Oct 30, 2022

Add option to force the use of the PYPY GIL scoped acquire/release logic to support nested gil access, see https://github.com/pybind/pybind11/issues/1276 and https://github.com/pytorch/pytorch/issu…

SeeChange-CI
authored and
rwgk
committed
75acc11
Apply suggestions from code review

henryiii
authored and
rwgk
committed
1cd4cff
Update CMakeLists.txt

henryiii
authored and
rwgk
committed
b855af2
docs: update upgrade guide

henryiii
authored and
rwgk
committed
fb233f0
Update docs/upgrade.rst

henryiii
authored and
rwgk
committed
7c4f8fe
All bells & whistles.
rwgk
committed
6f7317e
Add Reminder to common.h, so that we will not forget to purge `!WITH_THREAD` branches when dropping Python 3.6
rwgk
committed
cf1bb86
New sentence instead of semicolon.
rwgk
committed
8885e79
Temporarily pull in snapshot of PR #4246
rwgk
committed
f2538ba
Add `test_release_acquire`
rwgk
committed
52629ac
Add more unit tests for nested gil locking

SeeChange-CI
authored and
rwgk
committed
4062387
Add test_report_builtins_internals_keys
rwgk
committed
31b3868
Very minor enhancement: sort list only after filtering.
rwgk
committed
87e50b0
Revert change in docs/upgrade.rst
rwgk
committed
fe7470d
Add test_multi_acquire_release_cross_module, while also forcing unique PYBIND11_INTERNALS_VERSION for cross_module_gil_utils.cpp
rwgk
committed
897c9dd
Hopefully fix apparently new ICC error.
rwgk
committed
620cc70
clang-tidy fixes
rwgk
committed
a8ac941
Workaround for PYPY WIN exitcode None
rwgk
committed
e25bcd9
Revert "Temporarily pull in snapshot of PR #4246"
rwgk
committed
1e72313
Another workaround for PYPY WIN exitcode None
rwgk
committed
af1d9ee
Clean up how the tests are run "run in process" Part 1: uniformity
rwgk
committed
83c099d
Clean up how the tests are run "run in process" Part 2: use `@pytest.mark.parametrize` and clean up the naming.
rwgk
committed
ee8353e
Skip some tests `#if defined(THREAD_SANITIZER)` (tested with TSAN using the Google-internal toolchain).
rwgk
committed
5019457
Run all tests again but ignore ThreadSanitizer exitcode 66 (this is less likely to mask unrelated ThreadSanitizer issues in the future).
rwgk
committed
ab45337
bug fix: missing common.h include before using `PYBIND11_SIMPLE_GIL_MANAGEMENT`
rwgk
committed
857ee1a
if process.exitcode is None: assert t_delta > 9.9
rwgk
committed
10b0334
More sophisiticated `_run_in_process()` implementation, clearly reporting `DEADLOCK`, additionally exercised via added `intentional_deadlock()`
rwgk
committed
ea8b132
Wrap m.intentional_deadlock in a Python function, for `ForkingPickler` compatibility.
rwgk
committed
4c19a9a
Add link to potential solution for WOULD-BE-NICE-TO-HAVE feature.
rwgk
committed
2c5f864
Add `SKIP_IF_DEADLOCK = True` option, to not pollute the CI results with expected `DEADLOCK` failures while we figure out what to do about them.
rwgk
committed
19513d4
Add COPY-PASTE-THIS: gdb ... command (to be used for debugging the detected deadlock)
rwgk
committed
5ce396e
style: pre-commit fixes
pre-commit-ci[bot]
committed
8c9bce0
Do better than automatic pre-commit fixes.
rwgk
committed
065ac50
Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` to `pytest_report_header()` (so that we can easily know when harvesting deadlock information from the CI logs).
rwgk
committed
6984cab

Navigation Menu

Uh oh!

Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` option (cmake, C++ define) - #4216

Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` option (cmake, C++ define)#4216
rwgk merged 34 commits into
pybind:masterfrom
Chekov2k:pypy_gil

Commits on Oct 30, 2022

Add option to force the use of the PYPY GIL scoped acquire/release logic to support nested gil access, see https://github.com/pybind/pybind11/issues/1276 and https://github.com/pytorch/pytorch/issu…

Apply suggestions from code review

Update CMakeLists.txt

docs: update upgrade guide

Update docs/upgrade.rst

All bells & whistles.

Add Reminder to common.h, so that we will not forget to purge `!WITH_THREAD` branches when dropping Python 3.6

New sentence instead of semicolon.

Temporarily pull in snapshot of PR #4246

Add `test_release_acquire`

Add more unit tests for nested gil locking

Add test_report_builtins_internals_keys

Very minor enhancement: sort list only after filtering.

Revert change in docs/upgrade.rst

Add test_multi_acquire_release_cross_module, while also forcing unique PYBIND11_INTERNALS_VERSION for cross_module_gil_utils.cpp

Hopefully fix apparently new ICC error.

clang-tidy fixes

Workaround for PYPY WIN exitcode None

Revert "Temporarily pull in snapshot of PR #4246"

Another workaround for PYPY WIN exitcode None

Clean up how the tests are run "run in process" Part 1: uniformity

Clean up how the tests are run "run in process" Part 2: use `@pytest.mark.parametrize` and clean up the naming.

Skip some tests `#if defined(THREAD_SANITIZER)` (tested with TSAN using the Google-internal toolchain).

Run all tests again but ignore ThreadSanitizer exitcode 66 (this is less likely to mask unrelated ThreadSanitizer issues in the future).

bug fix: missing common.h include before using `PYBIND11_SIMPLE_GIL_MANAGEMENT`

if process.exitcode is None: assert t_delta > 9.9

More sophisiticated `_run_in_process()` implementation, clearly reporting `DEADLOCK`, additionally exercised via added `intentional_deadlock()`

Wrap m.intentional_deadlock in a Python function, for `ForkingPickler` compatibility.

Add link to potential solution for WOULD-BE-NICE-TO-HAVE feature.

Add `SKIP_IF_DEADLOCK = True` option, to not pollute the CI results with expected `DEADLOCK` failures while we figure out what to do about them.

Add COPY-PASTE-THIS: gdb ... command (to be used for debugging the detected deadlock)

style: pre-commit fixes

Do better than automatic pre-commit fixes.

Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` to `pytest_report_header()` (so that we can easily know when harvesting deadlock information from the CI logs).