Repository navigation
Conversation
…a Python list NativeVectorIndex._run_search turned every query into a list with tolist(). The codec then checked the list for float32-exactness (_exact_float_list: a pass of numpy over the list, tag BL) or, for a float64 query whose values float32 rounds, printed JSON number text (about 611 us at 1536 dimensions, #107). A numeric 1-d query (an ndarray of any numeric dtype, or a list/tuple of numbers, which numpy walks in C) now goes to the codec as a float32 array and travels through the float32 side buffer (tag B) directly. The engine reads the vector as float32 whatever it is given, so the rounding is unchanged. Anything else (text or None entries, a generator, ragged input, numpy missing) stays a list and reaches the engine as before. Port of humemai/arcadedb-embedded-python#523 (backlog row N-07) onto main after #107: the per-element _prep calls it removed are gone since #107, what remains is tolist() plus the exactness check, and the float64 text path. Untimed; a timed A/B decides it. Tests: test_find_nearest_query_forms.py (both backends: every form answers the same neighbours at the same distances, refusals unchanged; passes before and after) and test_native_find_nearest_params.py (native: where the floats travel and that the list exactness check is not run; 5 of its 6 tests fail on the code before). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No textual conflicts: main's #202 (record ids as a :rid parameter, src/arcadedb/_native/records.py and database.py) and #201 (run_in_transaction witness) touch other hunks than this branch. The record-id rule of #202 is kept: this branch pastes no record id into SQL text (the lazy per-hit vector load, if present, goes through lookup_by_rid, which keeps its literal id by design, see tests/test_native_statement_texts.py). Static checks after the merge: compile() and pyflakes of the changed files (the unused typing.Optional in records.py is on main too), pre-commit black and isort pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NativeVectorIndex._run_search(nativefind_nearestandfind_nearest_by_key) turns the query into a Python list withtolist(). Since #107 the codec then either checks that list for float32-exactness (_exact_float_list:set(map(type, ...)),np.array,astype,array_equal, then the side-buffer tagBL), or, for a float64 query whose values float32 rounds, prints one float repr per element as JSON text (about 611 us at 1536 dimensions, the figure in the codec's own docstring). This change hands the codec a float32 ndarray instead: a numeric 1-d query (an ndarray of any numeric dtype, or a list or tuple of numbers, which numpy walks in C) goes through the float32 side buffer (tagB) directly. The engine reads the vector as float32 whatever it is given, so the rounding does not change. Text orNoneentries, a generator, ragged input and a missing numpy stay a list and reach the engine as before.This is the port of humemai/arcadedb-embedded-python#523 (backlog row N-07) onto main. It was written before #107; what #107 already removed is the per-element
_prepcalls that #523 counted (768 to 0). What is left to save istolist(), the exactness pass for a float32 query, and the JSON text for a float64 query. Whether that shows in a search of 0.3 to 0.75 ms is for the timed A/B to say; no timing was run here.Results are identical: every form of the query (float32 and float64 array, tuple, list of numpy floats, generator, non-contiguous view, integer array,
find_nearest_by_key, a float64 query that float32 rounds) answers the same neighbours at the same distances (exact equality),allowed_ridsandef_searchstill apply, and a query of the wrong dimension or withNoneor text entries is still refused withArcadeDBError.Tests:
tests/test_find_nearest_query_forms.pyruns on both backends and passes before and after the change (it is the contract the change keeps).tests/test_native_find_nearest_params.pyis native only (not collected on the JVM backend): where the floats travel (tagBwithdimfloats beside the text, neverBL) and that_exact_float_listis not run on a query; 5 of its 6 tests fail on the code before. Local check on the committed bytes: the 14 tests pass against the unmodified native wheel26.11.1.dev0(matched dev pair dd45b53 + e353330c5c) with only this PR'sschema_objects.pycopied over it (not a wheel built from this repo); the vector, SQL-vector, numpy and query-vector-parameter suites show the same failing set with and without the change (the wheel predates main, andtest_vector.pyneeds JPype).black,isort(profile black),pyflakesandbanditare clean on the changed files. Not run: the full suite,scripts/build.sh, CI (a draft skips the matrix).This stays a draft until a timed A/B decides it: native wheels of
mainand of this head,find_nearestk = 10 p50 at 768 and 1536 dimensions for a float32 array, a float64 array and a list, with the answers compared pair by pair. If the interval does not exclude 1.00 it stays a cleanup or is closed. It touches the same function as the lazy-vector change of the same series (a separate PR); the two edit adjacent lines of_run_search, so whichever merges second needs a trivial rebase.Refs #19
🤖 Generated with Claude Code