Optimize recall calculation and enhance CAGRA search features - #2502
Open
sherylll wants to merge 4 commits into
Open
Optimize recall calculation and enhance CAGRA search features#2502sherylll wants to merge 4 commits into
sherylll wants to merge 4 commits into
Conversation
Decoupes work assignment in recall calculatoin loop from batch size (n_queries) used for search. This prevents each worker from having very little to do when batch size is small.
Author
|
A sample run with the new defaults, varying itopk and n_queries: With the new defaults we shouldn't observe recall drop across multi- to single-CTA algorithm change, as observed in #1187 . In contrary, the recalls even increase mildly... If we want a very smooth transition we could play with |
Contributor
|
Hi @sherylll, please also make sure any relevant configured parameters/options are also being proposed through the Python scripts where necessary (users always go through the Python scripts). I haven't looked at this PR, just pointing this out, since I know you're onboarding to cuVS. |
sherylll
marked this pull request as draft
August 26, 2026 02:09
Allow search_width=0 to scale exploration with itopk and graph degree, avoiding overly narrow SINGLE_CTA searches at large itopk while preserving MULTI_CTA's existing CTA floor.
At low batch sizes, increasing itopk adds parallel CTAs cheaply, so reduce local iterations as itopk grows. Larger candidate pools tolerate shallower traversal while retaining a fixed quality baseline.
Expose the hashmap bit-length floor to benchmarks and select the automatic default from the GPU's shared-memory capacity for SINGLE_CTA searches while preserving the MULTI_CTA default.
sherylll
force-pushed
the
improve-cagra-heuristic
branch
from
August 27, 2026 02:11
3ef607a to
885c056
Compare
sherylll
marked this pull request as ready for review
August 27, 2026 03:09
Author
|
Hi @cjnolet thanks for the reminder. I undid the quality knob for now, which we might add in a later PR, so that we can keep the scope of this PR to fixing the defaults. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
current default at 1 can be too weak for large itopk, e.g. itopk=512, width=1, graph_degree=64, and does not introduce enough diversity by each expansion. Support search_width=0 now, which select width to be
ceildiv(itopk,graph_degree)The current default (min_bitlen=8) can be too conservative, and often limits the hash reset interval to just 1 (reset every iteration). For later GPU architectures SMEM is usually larger and we can afford a larger hashmap size. I did experiments on a B300 with 228KB SMEM and min_bitlen=11 is consistently best performing for itopk <=256 (for itopk=512 12 bits wins).
base_iteration=32is too conservative and the QPS-recall curve is way below the actual pareto curve. Added asearch_qualityparameter to adjust the search budget. This parameter is fixed to 3 for the moment.