[tae898]

Closes #237

DRAFT, and it ships only if its A/B passes. On the arcadedb (native) build, insert_many would pause Python's cyclic garbage collector for the duration of a call when rows is a list or a tuple, and restore the previous state in finally (re-enabled only if it was enabled on entry). A generator, an iterator, a DataFrame or any other lazy input is not paused. This is the native counterpart of the JVM change in #236 (issue #235), with the same guard and the same semantics.

Why a separate decision: the JVM change was measured on the JVM build only (insert wall 0.819 to 0.903 of the default, 95% CI, #236). The untimed probe behind it found 19% to 28% of the JVM insert_many wall in the collector and zero collections inside the native insert_many, so the expected gain on native is nothing. That is a prediction from single untimed runs on earlier wheels, so the timed A/B on the current main wheel decides: arm A is the native CI wheel of main, arm B is the same wheel with only _native/database.py from this PR's head, arm Z is A again, 1,000,000 rows in slices of 100,000, ab.py --noise, with a control load whose caller has already disabled the collector. The rule (the thresholds of the JVM study), the round plan and the policy for a GRAY result are fixed in the study README before the first timed run. If the A/B does not pass, this PR is closed unmerged with the measurements on issue #237 and the native default stays.

Code: 14 added lines in NativeDatabase.insert_many (src/arcadedb/_native/database.py), the same structure as the JVM change in #236. insert_many keeps its signature, __defaults__ and docstring, checks that the database is open, and for a list or tuple while the collector is on calls gc.disable(), runs the body and calls gc.enable() in finally. The body is moved unchanged into a private _insert_many_nogc(self, type_name, rows, commit_every, parallel), which a lazy input, or a collector the caller already disabled, reaches directly, so nothing is restored that this call did not disable. insert_many does not call itself, so a spy or a subclass override of it sees one call per call, and it passes commit_every explicitly, so the patched default of tests/test_native_columnar_insert.py still reaches the body. Two earlier heads are withdrawn: cedf3a8 (a decorator, which broke that test because it patches insert_many.__defaults__) and 3b2eaaa (re-entry through the public method). Tests: tests/test_native_insert_many_gc_pause.py (native only, ignored on the JVM build in conftest.py) observes gc.isenabled() from inside the call for a list and a tuple, several chunks, the parallel mode, an exception, a collector the caller had disabled, a generator, an iterator and a generator expression, rows given by keyword, and a call without rows, asserts that insert_many.__defaults__ is (1000, False) and that nothing wraps it, and that an instance-level spy on insert_many sees exactly one call for a list.

Overlap with other open PRs on NativeDatabase.insert_many: #218 (head ab207e667e) changes the imports, the docstring and the body of insert_many; #232 (head f91cf142aa) changes one docstring line of it. This PR adds 14 lines: import gc at the top of the file, the block after _check_not_closed() and a new def _insert_many_nogc( line above the unchanged body, so it changes neither the body, the docstring nor the signature. git merge-tree of this branch with the heads of #218, #232 and #236 reports no conflict for each pair. #236 (JVM) edits core.py, tests/conftest.py (a line in NATIVE_NOT_APPLICABLE) and the bulk-loads page; this PR adds a line in a different part of conftest.py.

Docs: none yet, on purpose. TODO if the A/B passes: extend the bulk-loads section that #236 adds so that it covers the native build (today it says the native package does not change the collector), and add the same sentence to the insert_many docs; if it does not pass, no docs change and the sentence in #236 stays true.

Static checks run on head 3bc5f63: compile() of the three files, pyflakes (only the three existing numpy and pyarrow unused-import notes, none on a line of this change), black 26.10.1 and isort 9.0.2 unchanged. The tests run in CI (Test native, dispatched on the branch because the pull_request run skips a draft).

Edited 2026-11-04: the code paragraph and the overlap paragraph now describe head 3bc5f63 (body in _insert_many_nogc); they described the withdrawn heads cedf3a8 and 3b2eaaa before.

🤖 Generated with Claude Code

[tae898]

Head moved to 3b2eaaa: the first head (cedf3a8, a decorator) failed the parity legs of Test native because tests/test_native_columnar_insert.py patches insert_many.__defaults__, which a wrapper does not have. The pause is now inside the body of insert_many (ten lines, signature and defaults untouched), and the new tests assert insert_many.__defaults__ == (1000, False) and that nothing wraps it. The PR body is edited to match. Test native is dispatched on the new head.

[tae898]

Head moved to 3bc5f63: the body of insert_many is now in a private _insert_many_nogc and insert_many calls it (as in #236), instead of re-entering the public insert_many, which made an instance-level spy or a subclass override run twice per call. Signature, __defaults__ and docstring are unchanged, and a new test asserts a spy on insert_many sees one call for a list. The PR body is edited to match. Test native is dispatched on the new head.

[tae898]

Test runs on head 3bc5f63, read on 2026-11-04:

Run	Branch	Code	Result
41734004056	perf/native-insert-many-gc-pause (head 3bc5f63090)	this PR	every job green
41734008464	perf/native-gc-pause-tests-only (head 08daa74cc5)	main plus only the new test file	test_native_insert_many_gc_pause.py: 6 failed, 14 passed; every other file passes (linux-x86_64 py3.12 parity: 2012 passed, 6 failed, 0 errors)

The six that fail on main are the pause tests: test_the_collector_is_paused_during_a_call_with_a_sized_input[list] and [tuple], test_a_call_that_uses_several_chunks_is_paused_for_all_of_them, test_a_parallel_call_with_a_list_is_paused_too, test_the_collector_is_enabled_again_after_an_exception, test_rows_given_by_keyword_are_paused_too. The ones that pass on main check what must not change (generators are not paused, a collector the caller disabled stays as it was, the signature and defaults of insert_many, one call for an instance-level spy). The tests-only branch is deleted. The timed A/B on this head is queued.

[tae898]

Timed A/B on head 3bc5f63 (native wheel; arm A = main's CI native wheel, arm B = the same tree with only this PR's database.py, A/A control; laptop timed lane, tools/ab.py --noise, ABBA, round 0 dropped). B/A median with 95% CI:

figure	role	B/A	A/A
insert_many of the documents (list input)	gating	1.000 [0.997, 1.005]	1.001 [1.000, 1.003]
load wall (insert plus commit)	gating	1.001 [0.998, 1.004]	1.001 [0.998, 1.002]
peak RSS	guard	0.998 [0.995, 1.000]	1.000 [0.999, 1.001]
control insert	control	1.003 [0.993, 1.007]	0.997 [0.993, 1.001]

Clean pairs per figure: 11, 12. By the rule written before the run this is GRAY, no effect: the insert and load intervals lie inside the equivalence band [0.97, 1.03]. The study also counted the cyclic collections that run inside insert_many on main: 0 [0-0] (median [min-max]), so on the native build there is no collection for a pause to remove. The JVM build's pause (PR 236) did measure a gain; on native the call runs no collection to begin with.

Speed was the only motive, so by the rule this PR does not ship. Closing it, and its issue, with this result; the tests that check the collector is restored after a call stay in this branch if the question comes back.