Repository navigation
Conversation
…l k (B * k lookups -> k) A type with B buckets has B vector indexes; the search loaded and wrapped every hit of every bucket and kept k. GraphCalls.topHits merges the hit lists by score (stable, as the Python sort was), cuts to k, and loads those records in one crossing. Same records, order and scores; a missing record is still named, but now only when it is among the final k. New test file pins the result against a merge of the raw per-bucket lists and counts k wraps and k lookups (the no-bridge-jar fallback too). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Author
|
Closing here: the bindings moved to ArcadeData/arcadedb-python. The JVM change is carried there by the open PR ArcadeData/arcadedb-python#51, a port of this one. The branch stays. |
tae898
added a commit
to ArcadeData/arcadedb-python
that referenced
this pull request
Oct 29, 2026
…l k (B * k lookups -> k) A type with B buckets has B vector indexes; the search loaded and wrapped every hit of every bucket and kept k. GraphCalls.topHits merges the hit lists by score (stable, as the Python sort was), cuts to k, and loads those records in one crossing. Same records, order and scores; a missing record is still named, but now only when it is among the final k. New test file pins the result against a merge of the raw per-bucket lists and counts k wraps and k lookups (the no-bridge-jar fallback too). Port of humemai/arcadedb-embedded-python#534. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
VectorIndex.find_neareston a type with B buckets loaded and wrapped every hit of every bucket index (one record lookup and one Python wrapper per hit, B * k of them) and then sorted and kept k. This merges the per-bucket hit lists by score first and loads only the final k: a new bridge callGraphCalls.topHits(db, pairLists, k)takes the Java hit lists, sorts the candidates by score with a stable sort (equal scores keep the bucket order, as the Python sort of the concatenated hits did), cuts to k, and returns the k records, scores and RIDs in one crossing. Without the bridge jar the Python fallback does the same, reading each candidate's score and looking up only the k winners. This is backlog row B-01 (J-01), JVM half; the native half is #533.Results are identical: the same k records in the same order with the same scores. Checked with a fixed-seed dump of 120
find_nearestcalls (k = 1, 5, 10, 50; 600 vectors, 16 dimensions, 4 buckets) taken onorigin/mainand on this branch, byte-identical (rid andrepr(score)per hit, plus the hit shape). One edge differs on purpose: a hit whose record is missing now raises only if it is among the final k (before: if it was anywhere in any bucket's list); the message is unchanged.What it saves, untimed: record lookups and wrappers per search are k instead of B * k (40 to 10 at B = 4 and k = 10, counted by the new tests). With B = 1 nothing changes, every hit is a final hit; the 1077 us of record lookups and wrappers that jpype-crossings-20261022 measured at k = 10 on one bucket is not touched by this change. No timing was run; the A/B study for the multi-bucket gain is a separate step.
Local checks (all on the committed bytes, JVM wheel, bridge jar rebuilt from this branch with
wt-bridge-jar.sh):tests/test_vector_top_k_first.py(7 new tests: result equals a stable merge of the raw per-bucket lists for k = 1, 5, 10, 50, 400; k wraps; k lookups on the no-jar path) 7 passed, and its two count tests fail on the oldvector.py(40 vs 10) while the result tests pass on both;tests/test_vector.py,test_vector_second_pass.py,test_vector_params_verification.py,test_vector_delta_visibility.py,test_vector_sql.py,test_docs_examples.pypass; the one existing test that countedGraphCalls.hitscalls (test_lsm_vector_search_looks_up_all_hits_in_one_bridge_call) now countstopHits; pre-commit (black, isort, the A/B table check) passes. Not run: the full suite, CI.Docs:
docs/development/bridge.md(theGraphCallsrow). Overlap:git merge-treeagainst #524 head is clean. Whenmainis merged intonative-preview,tests/test_vector_top_k_first.pyreads Java hit lists, so it needs aNATIVE_NOT_APPLICABLEentry in the native-previewconftest.py.🤖 Generated with Claude Code