diff --git a/README.md b/README.md index af8ecb1..f3b2075 100644 --- a/README.md +++ b/README.md @@ -803,6 +803,23 @@ the wrapper regardless of case, spacing, or hidden Unicode tricks. - **Markdown can outrank the code it documents for doc-shaped queries.** See [Narrowing to code vs. documentation](#narrowing-to-code-vs-documentation) for what was measured and how to filter it out with `kind` when it matters. +- **Code that participates in a feature only by *calling into* it is close to invisible here.** + Similarity is computed against a chunk's own text, so a method that takes part in a feature + without naming it has nothing to match on. Measured case: `wallet`'s Windows lifecycle handler + `OnAppWindowChanged` calls `OnAppSleep`/`OnAppResume` on minimise/restore and is therefore part + of the auto-lock mechanism, but its body never says "lock", "auto" or "timeout", and neither + does its path. Against the query *"what fires when the auto-lock timeout elapses"* it scores + **0.4184** — well inside the band of genuinely irrelevant results, not marginally below the + floor. `Grep` finds it easily, because once you know the symbol `OnAppResume` a literal search + reaches every caller; embedding similarity has no equivalent of that second hop. + This is a capability boundary, not a tuning problem — lowering the relevance floor far enough + to admit 0.4184 would re-admit measured off-topic matches too (see `MinCosineSimilarity`'s + remarks and issue #35). It is also the concrete mechanism behind the coverage gap in + [Cost and accuracy](#cost-and-accuracy-what-four-measurements-actually-showed): platform-guarded + variants, alternative implementations and parallel code paths are exactly the shapes that + participate without naming. **If completeness is what you need — "find every caller", "is this + handled on all platforms" — use `Grep`, or use `code_search` to find the symbol and `Grep` to + find its callers.** - **Chunk ids do not survive a reindex — but reusing a stale one is now detected, not silently wrong.** Each id (e.g. `"myproject:3:4137"`) is an ordinal position in the specific index snapshot of *that project* a `code_search` call ran against, plus the generation number of that diff --git a/src/CodeIndex.Core/Embedding/EmbeddingOptions.cs b/src/CodeIndex.Core/Embedding/EmbeddingOptions.cs index 60d587d..4eafe8b 100644 --- a/src/CodeIndex.Core/Embedding/EmbeddingOptions.cs +++ b/src/CodeIndex.Core/Embedding/EmbeddingOptions.cs @@ -123,6 +123,28 @@ public sealed class EmbeddingOptions /// (rank-1 cosine for genuine vs. off-topic queries against a real index) before trusting this /// default on a different configuration. /// + /// + /// Issue #35 case study — a genuine miss is not always a threshold problem. The + /// wallet corpus's Windows-only lifecycle wiring — Platforms/Windows/App.xaml.cs's + /// OnAppWindowChanged method, which calls OnAppSleep/OnAppResume when the + /// window is minimized/restored — never surfaces for the query "what fires when the auto-lock + /// timeout elapses": measured directly against Ollama (bypassing this floor), that chunk's + /// cosine similarity for that query is 0.4184 — 0.13 below this floor, and squarely + /// inside the noise band above (0.3321–0.5402), not a marginal miss sitting at the boundary. + /// The method body never mentions "lock", "auto", or "timeout"; it only calls two + /// unrelated-looking methods on another type. already + /// prepends the file's own path as a lexical signal ("File: {path}\n" — see + /// BuildEmbedText), but that does not help here because the path itself + /// ("Platforms/Windows/App.xaml.cs") carries no lock/timeout vocabulary either. Tellingly, a + /// different, path-adjacent phrasing — "how does the app lock on resume" — scores 0.5880 + /// for the exact same chunk, comfortably above the floor: this is a phrasing-sensitive content + /// gap in one file's embedding, not a systemic threshold miscalibration. Lowering the floor far + /// enough to admit 0.4184 would also re-admit several of the genuinely off-topic queries this + /// constant was measured against (the noise band tops out at 0.5402), trading one known miss + /// for unmeasured false positives elsewhere — so the floor was left unchanged after this + /// investigation. See issue #35 for the full measurement and the alternatives considered (and + /// rejected, for lack of measurement) before reaching that conclusion. + /// /// public double MinCosineSimilarity { get; set; } = DefaultMinCosineSimilarity;