Repository navigation
Native glue: graph batches go through the engine's single-batch guard (P-04), parallel insert_many into a vertex type creates vertices (P-01b) - #538
Closed
tae898 wants to merge 3 commits into
Conversation
…tOnNativeYet, never from failure text
The report hook turned a failed native test into a skip when its exception text contained one of ~15 generic markers
("assert (None is not None)", "KeyError: 'flush'", "DID NOT RAISE KeyboardInterrupt", "cause is <class 'NoneType'>", "getClass", ...)
or when its name was on a list. Those texts are also what genuine bugs look like, and the hook hid three real regressions in PR #522.
Now only an explicit signal skips a test: NotOnNativeYet raised by the native code (NOTYET), or an entry in NATIVE_NA_JVM_TESTS keyed
by file::[Class::]function with a reason (NA-JVM, skipped at collection). Of the 154 items the text and name matching hid, 73 now run
and pass on native (60 resultset read errors, 7 graph batch, 6 sparse vector), 81 are listed as NA-JVM with a reason, and one real bug
is pinned as xfail(strict): BACKLOG-bindings.md P-04.
Also found and fixed: to_java_int_array and to_java_byte_array had no native branch and raised JVMNotRunning on the native wheel
(to_java_float_array has one); six tests that were listed as NA-JVM by name now run on native.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…, and parallel insert_many into a vertex type creates vertices P-04 (data loss): arcadedb_n_gb_create built its GraphBatch with GraphBatch.builder(db), which takes no part in LocalDatabase.batchStarted, so a second graph_batch() opened while one was open and two concurrent batches on one hub vertex lost edges (300 of 600 on the pre-fix image). It now opens the batch with db.batch(), as the JPype path does, so build() takes the guard and close() gives it back. The Python API is unchanged. test_a_second_graph_batch_is_refused_while_one_is_open is no longer xfail, the native carve-out in the close-refused test is gone, and test_two_concurrent_batches_on_one_hub_lose_no_edge counts the edges exactly on both wheels. P-01b: the async insert_parallel helper called newDocument for every row, so insert_many(parallel=True) and insert_columns(parallel=True) into a vertex type raised IllegalArgumentException. It now looks the type up once per call (the rule of humemai #519 / DocumentBatcher): a vertex type gets newVertex, a document type newDocument, an edge type is refused with a message naming it. TestInsertManyVertexType is #519's test class, copied unchanged so the two PRs merge cleanly. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898
added a commit
to ArcadeData/arcadedb-python
that referenced
this pull request
Oct 25, 2026
…ert_many into a vertex type creates vertices (P-01b) Port of humemai/arcadedb-embedded-python#538. gbCreate called GraphBatch.builder(db), which skips the engine's single-batch guard, so two concurrent batches on one database lost edges. It now calls db.batch(), as the JPype path does. insert_parallel looked at no schema and always called newDocument, which the JVM build refuses for a vertex type; it now looks the type up once per call: a vertex type gets vertices, an edge type is refused. Tests: TestInsertManyVertexType, the second-batch refusal and the two-writers-one-hub test (the last two run on both builds). Needs a rebuilt native library. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898
marked this pull request as ready for review
October 25, 2026 11:54
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
|
Closing as superseded. The native build now ships from ArcadeData/arcadedb-python, and this change is ported as ArcadeData/arcadedb-python@99a9dc2 in ArcadeData/arcadedb-python#66. That port does not take effect yet (the parallel vertex insert and graph batch guard tests still fail on native there), so it is being fixed there. This branch stays until that fix lands, because it is the reference. The JVM legs here fail TestInsertManyVertexType because native-preview lacks main's JVM vertex insert fix (#519). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #535 (branch
fix/native-explicit-skip-markers): this branch contains #535's commit, so until #535 merges the diff againstnative-previewshows it too. The change to review is the top commit317c92b7dd. It turns thexfail(strict=True)that #535 added for P-04 into a passing test.Two native-glue Java fixes that share one native-image rebuild.
P-04: a second graph batch while one is open (data loss)
arcadedb_n_gb_createbuilt its batch withGraphBatch.builder(db), which takes no part in the engine's single-batch guard (LocalDatabase.batchStarted, ArcadeData#5666). The JPype path goes throughdb.batch(), which does. On native a secondgraph_batch()therefore opened while one was open, and two concurrent batches on one hub vertex lost edges (300 of 600 on the image from before this change). The glue now opens the batch withdb.batch():build()takes the guard andclose()gives it back. The Python API is unchanged.test_a_second_graph_batch_is_refused_while_one_is_openis no longer xfail.test_graph_batch_close_refused_in_the_callers_transaction_can_be_retriedis gone, so that test also checks the refusal on native.test_two_concurrent_batches_on_one_hub_lose_no_edge, on both wheels: two writers load edges onto the same hub vertex, the second tries to open a batch while the first holds one, and the edge counts (total and the hub'sout('E').size()) must be exact.P-01b:
insert_many(..., parallel=True)into a vertex typeThe async
insert_parallelhelper callednewDocumentfor every row, soinsert_many(parallel=True)andinsert_columns(parallel=True)into a vertex type raisedIllegalArgumentException. It now applies the rule of #519 (DocumentBatcher): the type is looked up once per call, a vertex type getsnewVertex, a document typenewDocument, an edge type is refused with a message naming the type, and a missing type raises the schema's own error. The sync native paths already did this.TestInsertManyVertexTypeintest_bulk_insert.pyis #519's test class, copied unchanged so the two PRs merge cleanly in either order.Local checks
./mvnw -f native-spike/pom.xml -Darcadedb.version=26.10.2 compilethroughquiet_build.sh, exit 0.preview-build.sh, XMX=8g, engine 26.10.2, compatibility march) into~/build-scratch/native-lib/nbatch, 2m 21s.ARCADEDB_BACKEND=native): the focused run (the two P-04 tests, the close-refused test,TestInsertManyVertexType) is 11 passed.test_graph_batch.py,test_resultset_read_errors.py,test_sparse_quantization_compact.py,test_vector_sql.py(the files Native tests: skips come from explicit markers and NotOnNativeYet, never from failure text #535 touched) plustest_bulk_insert.py,test_insert_columns.pyandtest_async_executor.pyis 234 passed, 10 skipped (all explicitNA-JVMorNOTYETentries from Native tests: skips come from explicit markers and NotOnNativeYet, never from failure text #535).test_a_second_graph_batch_is_refused_while_one_is_openfailed (DID NOT RAISE), the new concurrency test failed with300 == 600, and the parallel cases of the vertex-type class raisedCannot create a document of type 'Vtx' because is not a document type.pre-commit run --fileson the three touched files: passed (black reformatted the new test once, then clean).🤖 Generated with Claude Code