Search_At8735By1024RealisticScale_CompletesWithinAGenerousBound failed on a docs-only PR (#31), then passed on a re-run of the identical commit.
VectorSearcher.Search (bounded heap) over 8735 x 1024 vectors: best of 5 runs took 21.404 ms.
Reference full-sort search over the same data: best of 5 runs took 10.136 ms.
Heap/full-sort ratio: 2.112 (lower is better).
Ceiling is 1.5; the comment records ~0.65 locally.
Why this is worth a look rather than just widening the bound
The test is well built — relative ratio rather than an absolute threshold, 8 warmup runs, best-of-5 minimum. A best-of-N minimum is specifically robust against GC pauses and scheduler noise, which is what makes a 2.1x reading hard to dismiss as ordinary jitter.
More to the point: both paths compute all 8,735 dot products over 1,024 dimensions. That shared work should dominate, and selection over the results — bounded heap versus full sort at topK=20 — should be a small fraction of the total. The expected ratio is therefore close to 1.0, slightly under. Observing 2.1 means the measurement is capturing something other than the selection strategy it claims to compare.
Candidates worth checking before touching the threshold:
- Does
FullSortReferenceSearch share the same dot-product implementation as VectorSearcher.Search, or does it have a simpler loop the JIT vectorises differently?
- Does
Search allocate per call in a way the reference does not, making it sensitive to GC state that the best-of-5 minimum cannot see through?
- Is the ubuntu runner's CPU/SIMD support hitting a different code path in one of the two?
If it turns out the two paths genuinely differ by that much on some hardware, then the assertion's premise — that the bounded heap is the faster choice, which is the entire justification for its extra complexity — is weaker than the comment claims, and that is worth knowing.
Raising the ceiling to make the failure go away would hide exactly the thing the test exists to detect.
Search_At8735By1024RealisticScale_CompletesWithinAGenerousBoundfailed on a docs-only PR (#31), then passed on a re-run of the identical commit.Ceiling is 1.5; the comment records ~0.65 locally.
Why this is worth a look rather than just widening the bound
The test is well built — relative ratio rather than an absolute threshold, 8 warmup runs, best-of-5 minimum. A best-of-N minimum is specifically robust against GC pauses and scheduler noise, which is what makes a 2.1x reading hard to dismiss as ordinary jitter.
More to the point: both paths compute all 8,735 dot products over 1,024 dimensions. That shared work should dominate, and selection over the results — bounded heap versus full sort at topK=20 — should be a small fraction of the total. The expected ratio is therefore close to 1.0, slightly under. Observing 2.1 means the measurement is capturing something other than the selection strategy it claims to compare.
Candidates worth checking before touching the threshold:
FullSortReferenceSearchshare the same dot-product implementation asVectorSearcher.Search, or does it have a simpler loop the JIT vectorises differently?Searchallocate per call in a way the reference does not, making it sensitive to GC state that the best-of-5 minimum cannot see through?If it turns out the two paths genuinely differ by that much on some hardware, then the assertion's premise — that the bounded heap is the faster choice, which is the entire justification for its extra complexity — is weaker than the comment claims, and that is worth knowing.
Raising the ceiling to make the failure go away would hide exactly the thing the test exists to detect.