research(nightly): semantic-query-cache — cosine-similarity ANN cache for agent-memory workloads - #817
Draft
ruvnet wants to merge 1 commit into
Draft
Conversation
Pass 1–3 research loop: agent-memory workloads repeat semantically similar queries (35% repeat rate measured); no major vector DB provides cosine- similarity-aware query result reuse as a first-class primitive. feat: add ruvector-query-cache Rust proof of concept Three variants: NoCache (ground truth), ExactCache (bitwise hash, 0% hit rate), SemanticCache (cosine scan, 34.8% hit rate at θ=0.85, 27.2% mean latency reduction). CachedAnn trait composes over any ANN backend. test: add 21 numeric acceptance tests for semantic-query-cache All pass: NoCache recall=1.0, ExactCache recall≥0.99, SemanticCache@0.90 hit_rate>ExactCache, recall≥0.70, Semantic@0.85 latency<90% of NoCache, monotone quality (higher threshold → higher recall). bench: capture semantic-query-cache benchmark results Linux x86_64, release build (LTO=fat, opt-level=3), n=5000×128-dim, 500 queries, repeat_rate=35%: Semantic@0.90 → 638µs mean, 0.871 recall, 1564 QPS vs 827µs / 1.0 / 1205 QPS for NoCache. All acceptance tests PASS. docs: add ADR-298 for semantic-query-cache Covers decision, consequences, failure modes, security considerations, migration path, open questions, and full benchmark evidence table. docs: add SEO gist for semantic-query-cache Public technical article with feature table, Mermaid architecture diagram, full benchmark results, competitor comparison, practical and exotic applications, usage guide, and optimisation roadmap. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_014jLrMrLrKoe3e8uSWs8Dib
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ruvector-query-cachecrate: a composable semantic query cache that wraps any ANN backend and returns stored results when an incoming query vector has cosine similarity ≥ threshold with a recently-answered query.NoCache(ground truth),ExactCache(bitwise-exact hash, 0% hit rate in practice),SemanticCache(θ)(cosine scan over stored queries).Benchmark Evidence (Linux x86_64, release build, n=5000×128-dim, 500 queries, repeat_rate=35%)
Key result:
Semantic@0.90achieves 31% hit rate, 22.9% mean latency reduction, 87.1% recall fidelity.Test plan
cargo build --release -p ruvector-query-cache— greencargo test -p ruvector-query-cache— 21/21 passcargo run --release -p ruvector-query-cache --bin benchmark— all 6 acceptance tests PASSdocs/research/nightly/2026-08-12-semantic-query-cache/README.mddocs/research/nightly/2026-08-12-semantic-query-cache/gist.mdDeliverables
crates/ruvector-query-cache/docs/adr/ADR-298-semantic-query-cache.mddocs/research/nightly/2026-08-12-semantic-query-cache/README.mddocs/research/nightly/2026-08-12-semantic-query-cache/gist.mdGenerated by Claude Code