Repository navigation
vectors: remove the stale per-bucket PRODUCT guard; the engine searches small buckets exactly - #177
Merged
Merged
Conversation
_ensure_product_quantization_ready (added in fc44025 for ArcadeData/arcadedb#3723) raised "requires at least pq_clusters indexed vectors per bucket" on the JVM build, and on the native build refused a whole type with fewer than pq_clusters vectors. Since ArcadeData/arcadedb#5417 (b37d610cd0) the engine skips PQ for such a bucket and searches it exactly, so the guard only blocks a working path. Measured on both 26.11.1.dev0 wheels with the guard bypassed: below pq_clusters per bucket the build succeeds and find_nearest_approximate returns the exact neighbours with exact scores; from pq_clusters up PQ is trained. Remove the method, its cache flag and its callers, fix the two docstrings and the vectors how-to page, drop the test that pinned the raise, and add a shared test of the engine behaviour (4 buckets, 5 and 15 vectors each, pq_clusters 16). Against the main 26.11.1.dev0 wheels the new small-bucket test fails on the JVM wheel (the guard raises) and passes on the native wheel, for both parameters. Closes #173 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bandit flagged the f-string SQL of the small-bucket test; the bucket number is a loop counter, not input. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #173
Removes
VectorIndex._ensure_product_quantization_ready()and its calls (find_nearest,find_nearest_approximate,build_graph_now, and the nativefind_nearest_approximate), and the_pq_readyflag that only it used. Since ArcadeData/arcadedb#5417 the engine builds and searches a bucket with fewer vectors thanpq_clustersexactly, without PQ; the guard refused that case on the JVM build and could refuse a small type on the native build.core.pydocstrings (quantization,pq_clusters) andhow-to/vectors.adocnow state the engine behaviour: a bucket belowpq_clustersbuilds, skips PQ and is searched exactly; at or above it, PQ is trained and the search is approximate.tests/test_vector_product_quantization.py::TestProductQuantizationSmallBuckets::test_buckets_below_pq_clusters_build_and_search_exactly[5|15](4 buckets,pq_clusters16, every bucket below it):build_graph_now()succeeds andfind_nearest_approximateandfind_nearestboth return the exact top-5 of a plain-Python cosine ranking, with matching scores.Evidence (the measurement in #173): with the guard bypassed, the JVM build gives exactly what the native build gives at 5, 15, 16, 17 and 40 vectors per bucket. Against the 26.11.1.dev0 wheels of main the new test fails on the JVM wheel (the guard's "requires at least pq_clusters" error, both parameters) and passes on the native wheel.
🤖 Generated with Claude Code