Repository navigation
Conversation
…hrough the JSON bulk path (#243) create_vertices and new_edges refused a whole call when one property was a list, dict, date-time, date, Decimal or bytes, and went per element through JPype. They now use json_bulk_dumps_tagged (insert_many's format from #82) and VertexBatcher/EdgeBatcher decode with DocumentBatcher.untag after toMap(). A chunk (vertices) or call (edges) holding a value the text cannot carry still takes the old path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…243) Step-0 probe (untimed): a list of 128 doubles cost 117 us per row through the text against 31 us on the property matrix (one double[]), and a scalar row 6.18 against 5.22 us. A list or tuple of only int/float now keeps main's path, and an input whose values are all scalars runs main's untagged json.dumps and gate (no shape test, no tagged writer); Java reads scalars as before (DocumentBatcher.value adds two type tests). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the NaN (CI run 41731925082) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Test runs for this PR, read on 2026-11-04:
Every test that fails on main is one of the new shape tests (the shape gate, the tagged writer and the bridge decode do not exist there). The tests-only branch is deleted after this read. The timed A/B (one-library overlay on main's CI JVM wheel, rule written before the run) is queued on the laptop's timed lane. |
|
Timed A/B on head
Clean pairs per figure: 10, 11, 12; every A/A median between 0.968 and 1.043. By the rule written before the run (revised after the untimed probe and before any timed run, as the study README records) this is a SHIP: every gating interval is below 0.90, the list-of-floats guard shows no harm, and the scalar control is flat. The merge waits for the end of the current batch of timed studies that pin main (merging now would move their pins); the PR leaves draft at that point, with its docs. |
… fallback rule, the laptop A/B figures (#243) Docs and docstring only. Figures copied by script from the study ab.txt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes #243
On the JVM wheel one list, tuple, dict, date-time, date, Decimal or bytes property sent the whole
GraphBatch.create_verticescall, and the wholeGraphBatch.new_edgescall with properties, down per-element Python (_to_java_property_matrix, ornew_edgeper edge). This carries those shapes throughjson_bulk_dumps_tagged, the tagged-value JSON #82 added forinsert_many.What changes:
json_bulk_shape_ok(type_conversion.py) is the cheap per-value gate for whatjson_bulk_scalar_okrefuses: the exact typeslist,tuple,dict,datetime,date,Decimal,bytes,bytearray. A non-empty list or tuple of onlyintandfloatis refused and keeps main's path: the untimed step-0 probe measured 117 us per row through the text against 31 us on the property matrix for a list of 128 doubles (onedouble[]). Subclasses (pandasTimestamp), numpy values, sets and pandas NA stay refused as before.json.dumps, and in Java the same per-key loop (DocumentBatcher.valueadds two type tests per value). The tagged writer and the shape gate are only reached when a row holds a shape._create_vertices_json_bulkandnew_edgeswrite the text withjson_bulk_dumps_tagged. A chunk of vertices that holds a value the text cannot carry (a NaN, an integer beyond 64 bits or a date-time inside a list or map, a subnormal, a non-str dict key) goes through the property-matrix path on its own. For edges every chunk is written before the first one crosses, so the same values start the per-edge path over with nothing buffered. A declaredARRAY_OF_FLOATSproperty whose list the float matrix cannot carry still sends its chunk to the matrix path, as in #496.VertexBatcherandEdgeBatcherdecode throughDocumentBatcher.value: a nested object or array becomes a plainMaporList(asDocumentBatcher.filldoes) and a tag is decoded byDocumentBatcher.untag(now package-private).Tests (
tests/test_graph_batch_tagged_shapes.py, JVM only, listed inNATIVE_NOT_APPLICABLE): every shape, declared and schemaless, is stored bycreate_verticesandnew_edgesas the property-matrix path andnew_edgeper edge store it (Python type andreprof every property compared); spies on_to_java_property_matrixandnew_edgeprove which path ran; a list of only numbers keeps the matrix path (alone and beside a dict); an input of scalars never reaches the shape gate or the tagged writer; values the text cannot carry keep the old path and store the same; chunk-level fallback with_BULK_CHUNKlowered; declared vector next to list, dict and date-time properties; the gates.tests/test_graph_batch.pyused adatetimeto reach the property-matrix path in two tests, and now uses a NaN.Known difference inside a nested container: a small integer inside a list or map arrives from the JSON text as a Java
Integerwhere the per-element path builds aLong. It reads back as the same Pythonint(the tests compare it); scalar integers already behaved this way on the bulk path.Overlap with #221 (
perf/jvm-create-vertices-jarray): #221 changes onlyis_vector_candidateintype_conversion.py; this PR adds functions elsewhere in that file and editsgraph_batch.pyand the bridge classes, so the two merge without a conflict. A JPypefloat[]ordouble[]on an undeclared property is not a shape and keeps the old path here; on a declaredARRAY_OF_FLOATSproperty #221 sends it to the float matrix.Status: draft. The step-0 probe (untimed, one run each, per row, PR/main) for the first head showed list_strings 0.46 and 0.43 (vertices, edges), dict 0.33 and 0.33, datetime 0.74 and 0.54, and the two regressions this head removes (list of doubles 3.8 times slower, scalar rows 1.18 and 1.12). Nothing is timed yet. A timed wheel-against-wheel A/B (pre-registered, DECISIONS #240; list_strings, dict and datetime gate it, list of doubles and scalar rows are no-harm guards) decides whether the PR leaves draft.
Docs updated in 4d6c9c0: a section in
how-to/bulk-loads.adoc(shapes on the bulk path, numeric lists keep the older path, the fallback rule, the laptop A/B figures with 95% intervals) and thenew_edgesdocstring. Edited 2026-11-04 to add this line.🤖 Generated with Claude Code