Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -803,6 +803,23 @@ the wrapper regardless of case, spacing, or hidden Unicode tricks.
- **Markdown can outrank the code it documents for doc-shaped queries.** See
[Narrowing to code vs. documentation](#narrowing-to-code-vs-documentation) for what was measured
and how to filter it out with `kind` when it matters.
- **Code that participates in a feature only by *calling into* it is close to invisible here.**
Similarity is computed against a chunk's own text, so a method that takes part in a feature
without naming it has nothing to match on. Measured case: `wallet`'s Windows lifecycle handler
`OnAppWindowChanged` calls `OnAppSleep`/`OnAppResume` on minimise/restore and is therefore part
of the auto-lock mechanism, but its body never says "lock", "auto" or "timeout", and neither
does its path. Against the query *"what fires when the auto-lock timeout elapses"* it scores
**0.4184** — well inside the band of genuinely irrelevant results, not marginally below the
floor. `Grep` finds it easily, because once you know the symbol `OnAppResume` a literal search
reaches every caller; embedding similarity has no equivalent of that second hop.
This is a capability boundary, not a tuning problem — lowering the relevance floor far enough
to admit 0.4184 would re-admit measured off-topic matches too (see `MinCosineSimilarity`'s
remarks and issue #35). It is also the concrete mechanism behind the coverage gap in
[Cost and accuracy](#cost-and-accuracy-what-four-measurements-actually-showed): platform-guarded
variants, alternative implementations and parallel code paths are exactly the shapes that
participate without naming. **If completeness is what you need — "find every caller", "is this
handled on all platforms" — use `Grep`, or use `code_search` to find the symbol and `Grep` to
find its callers.**
- **Chunk ids do not survive a reindex — but reusing a stale one is now detected, not silently
wrong.** Each id (e.g. `"myproject:3:4137"`) is an ordinal position in the specific index
snapshot of *that project* a `code_search` call ran against, plus the generation number of that
Expand Down
22 changes: 22 additions & 0 deletions src/CodeIndex.Core/Embedding/EmbeddingOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,28 @@ public sealed class EmbeddingOptions
/// (rank-1 cosine for genuine vs. off-topic queries against a real index) before trusting this
/// default on a different configuration.
/// </para>
/// <para>
/// <b>Issue #35 case study — a genuine miss is not always a threshold problem.</b> The
/// <c>wallet</c> corpus's Windows-only lifecycle wiring — <c>Platforms/Windows/App.xaml.cs</c>'s
/// <c>OnAppWindowChanged</c> method, which calls <c>OnAppSleep</c>/<c>OnAppResume</c> when the
/// window is minimized/restored — never surfaces for the query "what fires when the auto-lock
/// timeout elapses": measured directly against Ollama (bypassing this floor), that chunk's
/// cosine similarity for that query is <b>0.4184</b> — 0.13 below this floor, and squarely
/// inside the noise band above (0.3321–0.5402), not a marginal miss sitting at the boundary.
/// The method body never mentions "lock", "auto", or "timeout"; it only calls two
/// unrelated-looking methods on another type. <see cref="Chunking.RoslynChunker"/> already
/// prepends the file's own path as a lexical signal (<c>"File: {path}\n"</c> — see
/// <c>BuildEmbedText</c>), but that does not help here because the path itself
/// ("Platforms/Windows/App.xaml.cs") carries no lock/timeout vocabulary either. Tellingly, a
/// different, path-adjacent phrasing — "how does the app lock on resume" — scores <b>0.5880</b>
/// for the exact same chunk, comfortably above the floor: this is a phrasing-sensitive content
/// gap in one file's embedding, not a systemic threshold miscalibration. Lowering the floor far
/// enough to admit 0.4184 would also re-admit several of the genuinely off-topic queries this
/// constant was measured against (the noise band tops out at 0.5402), trading one known miss
/// for unmeasured false positives elsewhere — so the floor was left unchanged after this
/// investigation. See issue #35 for the full measurement and the alternatives considered (and
/// rejected, for lack of measurement) before reaching that conclusion.
/// </para>
/// </remarks>
public double MinCosineSimilarity { get; set; } = DefaultMinCosineSimilarity;

Expand Down