[tae898]

find_nearest keeps its default: every hit carries its indexed vector, and the default search is the code of #203 (query as one float32 buffer), unchanged. This PR adds a keyword-only include_vector=False for callers that need only the ids, the distances or other properties of the hits.

On the native build, find_nearest(q, k, include_vector=False) runs SELECT *, !<vector property>, !record FROM (SELECT expand(vectorNeighbors(...))) and builds each hit without the vector (768 or 1536 floats per hit as JSON text, most of it read by nobody). A hit loads its record once, by RID, the first time the vector is read (get, has_property) or the whole record is needed (to_dict, get_property_names, modify, save). Values, property order and types are those of lookup_by_rid, and a hit deleted since the search raises ArcadeDBError on that first read. A caller that opts in and then reads the vector of every hit pays one extra lookup per hit, so the keyword is for searches that do not read it. The JVM build accepts the keyword and ignores it: it loads whole records either way, so the call costs the same (stated in the docstring and in docs/modules/ROOT/pages/how-to/vectors.adoc, which gets one short paragraph).

Why an opt-in and not the default. The first version of this PR made every hit lazy. Its timed A/B (native wheel of main 7e64c5e, engine 26.10.2, find_nearest k = 10, ab.py 95% CI, 12 clean pairs) was good for callers that do not read the vector and bad for callers that do:

unit	p50 B/A at 1536-d	p50 B/A at 256-d
hits not read	0.66 [0.657, 0.663]	0.805 [0.803, 0.815]
id read	0.663 [0.659, 0.665]	0.808 [0.807, 0.812]
id and vector read	1.011 [1.008, 1.016]	1.336 [1.333, 1.346]

At 256 dimensions the lookup per hit costs more than shipping the vector, which is above the 1.10 cap for a default. The statement control was flat. So the default stays as it is, and a caller who knows it does not need the vector says so.

Stacked on #203: the base of this PR is perf/native-find-nearest-ndarray, and the diff is the three files of the change (src/arcadedb/_native/records.py, src/arcadedb/_native/schema_objects.py, src/arcadedb/vector.py) plus docs and tests. Retarget to main after #203 merges.

Tests. tests/test_vector_hit_records.py runs on both backends: the default call and the opt-in call equal lookup_by_rid in type, property names and order, every value and the vector, for a four-bucket type with a subtype and in every first-access order; a hit can be modified and saved; ef_search and allowed_rids work with the keyword; the keyword is keyword-only and changes no answer; an opt-in hit of a type with properties called record and distance (the names of the search row's own columns) equals the lookup. tests/test_native_vector_hit_lazy.py is native only: the default search ships the vector, runs a statement that drops nothing and looks up nothing; the opt-in search ships no vector and looks up nothing, then one lookup per record however the vector is reached, the loaded vector equals the default's, and a closed database does a lookup for none. Not run locally (no workload on this machine); CI runs them. black 26.10.1, isort 9.0.2 (profile black), bandit and compile are clean on the changed files.

Known and left alone: on the default path (this PR does not touch it), a type with a property called record or distance still gets a wrong or failing hit on native find_nearest, because those names are the search row's own columns. The opt-in path handles them. A fix of the default path would be a separate change.

The timing of this design (the opt-in against the default, and the default path against #203's head, both dims) is a pre-registered study, not run yet; this PR makes no speed claim until it is. It stays a draft.

Refs #19

Edited 2026-11-04: reworked from "lazy by default" to an explicit opt-in (include_vector=False), stacked on #203; the first body described the lazy-by-default design and its side fix for properties named record or distance on the default path.

🤖 Generated with Claude Code

[tae898]

Timed A/B (native wheel; arm A = main's CI native wheel, arm B = the same tree with only this PR's files, A/A control; laptop timed lane, tools/ab.py --noise, ABBA, round 0 dropped). find_nearest p50 latency, B/A median with 95% CI (below 1 = this PR is faster):

caller	1536-d	256-d
hits not read	0.66 [0.657, 0.663]	0.805 [0.803, 0.815]
each hit's id read	0.663 [0.659, 0.665]	0.808 [0.807, 0.812]
each hit's id and vector read	1.011 [1.008, 1.016]	1.336 [1.333, 1.346]
statement path (control, not changed by this PR)	1.001 [0.994, 1.006]	1.007 [0.994, 1.012]

Clean pairs per figure: 12; answers equal on both arms in every round. By the rule written before the run this is a DROP as the default: a caller who reads the vector of each hit pays one extra lookup per hit, 1.336 [1.333, 1.346] at 256 dimensions, above the rule's 1.10 regression cap, while callers who do not read the vector gain clearly.

So the cut is real but only for callers who do not need the vectors, which makes it an API question rather than a default. The options as we see them, with no choice made here: (1) an explicit opt-in on find_nearest that returns hits without their vector (other vector stores expose the same choice as an include-vector flag); (2) leave find_nearest as it is and close this PR. This PR stays draft until that is decided; an opt-in version would get its own timed check of both the opt-in and the default path.

[tae898]

Change of design, 2026-11-04. The first version of this PR made every find_nearest hit lazy. Its timed A/B (main's native wheel 7e64c5e, engine 26.10.2, 12 clean pairs, ab.py 95% CI) showed that it helps callers that do not read the vector (hits not read p50 B/A 0.66 [0.657, 0.663] at 1536-d and 0.805 [0.803, 0.815] at 256-d) and hurts callers that do (id and vector read 1.011 [1.008, 1.016] and 1.336 [1.333, 1.346], above the 1.10 cap for a default). So the default is back to what it is today: hits carry their vector, and the default search is #203's code. The lazy hits are now an explicit opt-in, find_nearest(..., include_vector=False), for callers who need only ids or distances; the JVM build accepts the keyword and ignores it.

The PR is rebased onto #203 (head 02103b5) and its base is now perf/native-find-nearest-ndarray; new head c91a828. The body is rewritten to match. A new pre-registered timed study will measure both paths (the default must stay non-inferior to #203, the opt-in must win when hits are not read or only the id is read); until it runs there is no speed claim and the PR stays a draft.