find: an exact-text tool that replaces grep over the index - #43
find: an exact-text tool that replaces grep over the index#43ekechinwokah wants to merge 13 commits into
Conversation
The MCP surface had no answer to grep's question. The guard pointed agents at bm25_search for exact identifiers, but that is ranked and cut at k: 'every place Foo appears' came back as the ten chunks most about Foo. Agents kept a reason to reach for grep. Add a fourth tool, find: every line containing an exact string, cited path:line like grep -n, complete and unranked. The engine's token match narrows to the chunks that contain every token of the query (an inverted-list intersection, no scoring), then the literal is verified line by line inside those chunks. The analyzer splits identifiers (parse_config indexes as parse and config), so the first step alone would over-match; the second makes every hit a real occurrence, and grep's line-based, case-sensitive semantics fall out of it (ignoreCase opts out). Results are capped (default 100, max 500) and always carry the repo-wide total, so a cut list still says how many matches exist. The same door serves the CLI as cx find, prints path:line: text, and records a receipt and ledger entry like search and sql. Docs, the skill, and the tool descriptions now describe four tools, one per question: find is unranked and complete where search is ranked and top-k, so it is not the near-duplicate the three-tool rule guards against.
ashishmishra26
left a comment
ashishmishra26
left a comment
There was a problem hiding this comment.
Reviewed the implementation, the tool surface, the CLI path, tests and the doc rewrite, and ran it head to head against main on the same repo, questions, model and judge (details in the report I shared separately).
Design. This argues its way past the three-tool rule correctly. find is unranked and complete where search is ranked and top-k, so it is not a near-duplicate, and the rewritten rule ("a new tool must answer a question none of these four does") is a better rule than the one it replaces. The two-step shape, token intersection to pick candidate chunks and then a literal check per line, is right: no file is scanned and every hit is a real occurrence. The overlap dedupe by path:line is the subtle part and it is handled.
Measured. Quality is level: blind judge 29/22/13 (main/find/tie) over 64 pairs, unsupported claims 159 vs 143, no out-of-bounds citation in 128 answers. Cost falls where the tool applies: -35% tokens, -17% dollars, -38% tool calls on eight exact-lookup questions, flat on the shipped question set. The agent opened with find in 12 of 16 lookup runs against Grep-first in 8 of 16 on main, so the description steers as intended. Two costs of a fourth tool showed up: about a thousand tokens per turn of prompt (a uniform 17k to 18k on single-call questions) and mis-selection on a read-one-known-file question, where the agent ran find three times and then read the file anyway.
Should fix before merge (each anchored inline):
- Expose per-file counts. Every match is already in memory before the limit is applied, so a
{path, count}breakdown is free. Without it the one grep-shaped questionfindcannot answer isgrep -c, and in testing the agent recognised that and went to Grep for it. - Window the cited
textaround the match. Head-truncation at 240 characters can return a hit whose text does not contain the query. - Validate
--limiton the CLI path.Number("abc")is NaN and the clamp passes it through, so the command prints nothing and reports nothing. - Say what
findis not for in the tool description itself. The skill has the "known file: Read" row; the MCP description does not, and that is where the mis-selection came from.
Worth a look: candidate volume on very common tokens (fn, self) walks most of the table synchronously in the server process; a test that analyzerTokens agrees with the engine on a fixed corpus, since the mirror will break silently if the analyzer changes; a dedupe test for a line that lives in two overlapping chunks; partial propagation under the file cap; and the README/tradeoffs rewrite could cite the measurement rather than assert, and "matches grep's cost rather than beating it" undersells it.
Sequencing. #40's denial message still routes agents to bm25_search via sql. If this merges first, #40 needs a rebase and that message rewritten to name find.
Overall: this is a real improvement to the surface and the measurement supports the claim it makes. Item 1 is the one I would insist on, because it is the difference between "grep is optional" and "grep is still needed for one thing".
| query, | ||
| ignoreCase, | ||
| matches: all.slice(0, limit), | ||
| total: all.length, |
There was a problem hiding this comment.
Every match is already in all before the limit is applied, so a per-file breakdown is free here: something like byFile: [{path, count}] alongside total and files. Without it the one grep-shaped question this tool cannot answer is grep -c; in testing the agent recognised that and went to Grep for it every time. This is the one item I would insist on.
There was a problem hiding this comment.
taken. byFile on the result — {path, count}, most matches first with the tie on path — computed over every match before the slice, so it stays whole when matches is cut, and files is now its length rather than a second set. the cli gets -c / --count and prints path: count like grep -c. the description names it as the grep -c answer, so that last reason to leave for Grep is gone. 126 tests green on the head.
| all.push({ | ||
| path, | ||
| line: m.line, | ||
| text: m.text.length > FIND_LINE_CAP ? m.text.slice(0, FIND_LINE_CAP) + "..." : m.text, |
There was a problem hiding this comment.
Head-truncation at 240 characters means a match past column 240 returns a hit whose text does not contain the query. An agent reading that will conclude the tool is wrong. Slice a window around the first occurrence instead (and mark both ends when cut).
There was a problem hiding this comment.
taken. matchLines now returns the column of the first occurrence and a new excerpt cuts a window of the same cap around it, with ... on whichever ends were cut — sixty characters of lead so the excerpt shows what the match sits in rather than opening on it. pinned in unit tests and on a 900-character fixture line with the marker at column 600.
| const handle = openIndex(opts.path); | ||
| const result = find(handle, text, { | ||
| ignoreCase: opts.ignoreCase, | ||
| limit: opts.limit === undefined ? undefined : Number(opts.limit), |
There was a problem hiding this comment.
Number("abc") is NaN; Math.max(1, NaN) is NaN; matches.slice(0, NaN) is [] and all.length > NaN is false, so cx find x --limit abc prints nothing and sets no truncated. The MCP schema catches this, the CLI does not. Parse and reject non-integers here.
There was a problem hiding this comment.
taken, in find itself rather than the cli: a non-integer or sub-1 limit throws limit must be a positive integer, got NaN, so cx find x --limit abc dies with that instead of printing nothing, and the mcp path keeps the same rule behind its schema. over the cap still clamps — a big number is a valid wish, not a mistake.
| title: "Find exact text (every occurrence, like grep -n)", | ||
| description: | ||
| "Every line in the repository that contains an exact string - the grep replacement. Use it " + | ||
| "where you would reach for grep or rg: every use or definition of an identifier, an error " + |
There was a problem hiding this comment.
Worth one more sentence on what find is not for. The skill has the "a file you already know the path of: Read" row, but this description is what the agent actually sees. In testing it ran find three times to pull version lines out of Cargo.toml and then read the file anyway, in every run of that question.
There was a problem hiding this comment.
taken. the description now says it outright: not for a file you already know — Read that file — find locates occurrences across the repo, and Read path:line for the code around a hit. the skill's find section carries the same line so both surfaces steer the same way.
also flagging so it doesn't get conflated later: #40's deny message still routes to bm25_search via sql. when this lands, #40 rebases and that message names find.
| const limit = Math.min(Math.max(1, opts.limit ?? DEFAULT_FIND_LIMIT), MAX_FIND_LIMIT); | ||
|
|
||
| const table = handle.db.openTable(TABLE); | ||
| const candidates = table.tokenMatch("content", tokens.join(" "), { mode: "and", projection: FIND_PROJECTION }); |
There was a problem hiding this comment.
Non-blocking: with mode: "and" on a very common token (fn, self) this returns most of the table and the loop below splits every chunk synchronously in the server process, blocking auto-sync and other requests while it runs. Fine at ~5k chunks; worth measuring at the 20k-file cap, or capping candidates and degrading total to a lower bound.
There was a problem hiding this comment.
not taken here, and agreed it's the open cost. the work is bounded by the chunks that contain every query token, so it only degenerates when the whole query is one ubiquitous token — then the candidate set is the table, read synchronously in the server. a candidate cap would turn total into a lower bound and quietly break the completeness that is the tool's whole point, so i'd rather measure it at the file cap first and decide with a number. recording it as the follow-up.
| * text is dropped whole. Mirrored here so the candidate lookup asks the index | ||
| * for exactly the tokens it holds - a different split would miss chunks that | ||
| * do contain the literal. Duplicates are dropped; the intersection is the same. */ | ||
| export function analyzerTokens(text: string): string[] { |
There was a problem hiding this comment.
Non-blocking: this mirrors the engine analyzer client-side and will break silently if the content column analyzer ever changes. A test asserting analyzerTokens agrees with the engine on a fixed corpus would catch that; the current tests only check the JS against itself. A dedupe test for a line that lives in two overlapping chunks would also be worth having, since that is the subtle correctness property.
There was a problem hiding this comment.
taken, both. the integration suite now runs five edge-case strings — underscore and punctuation splits, digits, a dropped non-ascii run — through tokenMatch twice, raw text to the engine and the mirror's tokens, and asserts the same candidate chunks come back, so a change in the content analyzer fails the build instead of shrinking recall silently. the dedupe case is a 130-line text fixture whose line 55 sits in two 60-line windows; the test asserts the overlap is real in the table before asserting find reports the line once. partial propagation is pinned in the same block.
| Naming the one file a known symbol lives in is a single grep's job. `find` | ||
| does that job from the index - every matching line as `path:line`, no file | ||
| scanned - and returns the same one-line-per-match shape grep does, so it | ||
| matches grep's cost rather than beating it. Ranked `search` is the wrong tool |
There was a problem hiding this comment.
Undersells it. On eight exact-lookup questions it beat the grep path by about a third in tokens and a sixth in dollars, because grep answers need follow-up reads to turn a match into a cited line and find does not. The README/AGENTS rewrite could cite the measurement rather than assert the four-tool design the way the old text asserted three.
There was a problem hiding this comment.
taken in part. the section now says what the saving is — a grep hit still needs a read to become a cited line and a find hit already is one — and drops the 'matches grep's cost' line. the numbers stay out of the prose until the run lands in docs/benchmark.md with the rest of the measurements; a figure quoted from a review comment ages badly. the readme rule text is the design statement, the benchmark is where the evidence goes.
…iption
Review follow-ups on the find tool.
byFile on the result: matching lines per file over every match, most
first, computed before the limit is applied so it stays whole when the
line list is cut. It is the grep -c answer, the one grep-shaped question
the tool could not answer; cx find gets -c/--count for the same view.
A long line is now cut to a window around the match instead of its head:
head-truncation at the cap could return a hit whose text did not contain
the query, which reads as the tool being wrong. Both cut ends are marked.
A malformed limit is rejected: Number("abc") is NaN, and the old clamp
passed it through so cx find printed nothing and reported nothing. The
check lives in find itself so the CLI and MCP paths share it.
The tool description says what find is not for - a file you already know
is a Read - which is where a mis-selection showed up in review testing.
Tests: excerpt windows; per-file counts on a cut list; a line in two
overlapping fixed-window chunks is reported once (and the overlap is
asserted to exist); the client-side analyzer mirror agrees with the
engine on which chunks are candidates for five edge-case strings; the
partial-index marker propagates; malformed limits throw. The tradeoffs
page names the saving on exact lookups instead of calling it a wash.
The review asked the docs to cite the measurement rather than assert the four-tool design. docs/benchmark.md now carries the 2026-09-04 run of feat/find-tool against main: same repo (infino @ ed4e020, 402 files, 5,528 chunks), questions, model (claude-sonnet-4-6) and a blind judge (claude-opus-5), 128 agent runs over 64 judged pairs. Answer quality is level (29 / 22 / 13, no out-of-bounds citation); exact lookups are -35% tokens, -17% dollars, -38% tool calls; the shipped set is flat; a fourth tool costs about a thousand prompt tokens per turn. The section also records what the run found and the commit after it changed (byFile for the per-file count gap, the known-file steer), and the discarded first run whose index had drifted from the tree. README and tradeoffs cite the numbers instead of asserting. The eight pinpoint questions ship as bench/questions/infino-pinpoint.json so the comparison can be re-run. The default find limit moves to config beside DEFAULT_SEARCH_K and reads CX_FIND_LIMIT, so the fallback for an agent that passes no limit is a deployment setting rather than a constant; the hard cap stays the context-flood guard.
The default number of matching lines find returns moves from 100 to 500, the hard cap, so the cut only ever lands on a flood - a ubiquitous term returning tens of thousands of lines - and never on a real answer: the largest lookup measured needed about 300 lines and the old default cut it. At roughly fifty tokens per returned line the cap is about 25k tokens of tool result, large but survivable. A caller that wants fewer passes a smaller limit; total and byFile are complete either way.
The hook tally counted how many prompts used code-context; it could not say which tool, or what the agent reached for first. Both are the selection signal the tool-surface measurement needs: whether a grep-shaped prompt opens with find or with Grep, and which door each call goes through. PromptStats gains cxCallsByTool and firstToolByPrompt. Every PostToolUse event the hook forwards counts toward the first-tool tally (code-context tools by their short name, anything else by the name delivered); only code-context's own tools count as invocations, as before. With the documented matcher only code-context tools are forwarded; the README says how to widen it. Older stats files load and grow the fields on the next event. cx usage prints both lines. Two question sets for the surface comparison: infino-known-file.json, where the right first call is Read, and infino-by-meaning.json, where the question names no identifier.
The lanes ran this checkout's dist/cli.js only, so two builds of the server could not be compared from one harness. CX_BENCH_CLI names another build and CX_BENCH_BUILD labels it; both land on every result row.
Three readers of the multi-build results file and one judge, so a tool- surface comparison is a report rather than a spreadsheet: - compare-builds.mjs: per set and build, the sum over questions of the median tokens, cost and calls, how often the first call was a code-context tool, and every run's first tool; CX_MD=1 for markdown, CX_DETAIL=1 for the per-question view with the tool mix. - cite-check.mjs: every path:line an answer cites must exist and be in bounds, and an identifier written beside it must appear within five lines; bare file names resolve against the repo or the answer's own full paths. - judge.mjs: the blind pairwise judge from the find run, as a script - claude-opus-5 with Read/Grep/Glob on the repo, both answers in random order, a winner, a confidence and unsupported-claim counts per pair, appended to .work/results/judge.jsonl. JUDGE_LIMIT caps a smoke run. - judge-report.mjs: wins, ties, unsupported claims and median confidence per set for each judged pair, latest verdict per pair. Builds are selected by their CX_BENCH_BUILD label or a since..until window for rows recorded before the label existed.
Plan 101's matrix over the code-context MCP surface: V0 (the surface after find) through V8, one lever at a time, on Sonnet 4.6 with 108 runs per build, second passes of V0 and V3 for the spread, Haiku 4.5 two passes per build, and the Opus 5 blind judge against V0. What it found: the routing prose was not doing the steering (sql 30/30, search 18/18 on every build); the receipt sentence was - dropping it flipped two lookups from find to Grep (12/12 vs 15/15) because it said what the result reports, and naming the field without the request brings them back; the rename to context moved nothing; the tiered search result buys dollars with round trips and answers no better; Haiku needs the answer-from-a-hit sentence in the search description; reindex was called only by Haiku and only where it hurt. V8 is the shipping candidate: 17-20% fewer tokens than V0 on Sonnet at the same round trips, code-context first on 97 of 108 runs. The V8 judge row lands when its run finishes.
The blind judge finished after the section landed: V8 wins 51 pairs to V0's 33 with 23 ties and 236 unsupported claims against 302, ahead on every set but comprehension (7 to 7), and 6 to 3 on the known-file set where its extra find-first runs were the one soft spot.
The surface plan 101 measured (docs/benchmark.md, "The tool surface: names, shapes, and prose"), as one change: - descriptions cut to question shape, not-for, and result shape, and the server instructions to a routing table: ~680 tokens of tool text plus ~165 of instructions per turn against ~1,280 plus ~545, with selection unchanged (sql first on 30/30 aggregation runs, search first on 18/18 comprehension and by-meaning runs, on every build measured); - three sentences the trim had cut and the runs showed load-bearing, kept: answer from a hit without re-opening the file (in the search description - Haiku doubled its Reads without it), what the usage receipt reports (dropping it flipped two lookups from find to Grep 12/12 vs 15/15; naming the field brings them back), and that lang is the file extension (every trimmed build paid a three-call detour without it); - the request to show the usage line to the user is gone; the field stays and cx usage keeps the ledger; - the reindex tool is gone. No Sonnet run called it in 216; Haiku called it in 2 of 3 runs of one aggregation question, once as its first call. The first query builds the index, every query re-syncs it, and cx index --full rebuilds from a shell. The noIndex error, the status hook line, the CLI help, the skill, README, FAQ, llms.txt, context7 and wiki notes say three tools. On Sonnet 4.6 against the previous surface: 17-20% fewer tokens over the 36-question suite at the same round trips, a code-context tool first on 97 of 108 runs against 94, judged 51 to 33 with 23 ties and a fifth fewer unsupported claims. Two Haiku passes each: aggregation 40% cheaper, comprehension at or under the previous surface in three of four passes.
A tool leaving the MCP tool list is a breaking change for clients that called it; package.json, the plugin manifest, the .mcp.json pin, the registry manifest and the CLI version move together.
The message told the agent to reach for bm25_search through the sql tool for exact identifiers, which was the workaround before find existed. It now names the three tools by question: find for every line containing an exact string (what grep did), search for meaning, sql for counts and rankings. Comments in both the source and the shipped CJS copy say the same. Merge after #43, which adds find.
The PNG is rendered from docs/architecture.svg (sharp, at the shipped 1984x1344), so the tool list in the picture matches the surface.
The heading read as a 10x; the number is 10 to 12% fewer tokens.
ashishmishra26
left a comment
ashishmishra26
left a comment
There was a problem hiding this comment.
Thanks for the fast turnaround on the threads. I checked each fix in the head: byFile plus cx find -c, the excerpt window with both cut ends marked, the limit check shared by CLI and MCP, the known-file steer in the description and the skill, the analyzer-mirror test against the engine, the overlap-dedupe and partial tests, and the tradeoffs rewording. All good, and I agree with deferring the candidate-volume question: a cap would break the completeness the tool exists for. Measure at the file cap first.
The branch has grown since the review, though, and some of what it now carries should not merge in this PR.
1. Split the PR. Everything up to and including the bench scripts and the measurement docs is the find change and its review follow-ups, and I would merge that today. The trimmed descriptions, dropping the "show the usage line" request, removing reindex and the version bump are a second change with a different risk profile: one of them is breaking for MCP clients. They deserve their own diff, their own review and their own release note.
2. Drop the release commit. Release on merge only tags when a release-* PR opened by the train merges. Merging this PR with package.json at 0.5.0 tags nothing and publishes nothing. The next daily Release prep run classifies the src/ changes since v0.4.0 as minor and runs npm version minor on top, which yields 0.6.0. In between, main's plugin manifest and .mcp.json point at @infino-ai/code-context@0.5.0, which will not exist on npm, so a plugin install in that window fails. Let the train cut the version and stamp the pins; that is what it does.
3. reindex is being removed on the wrong evidence. The bench freezes the index (auto-sync off, tree pinned), so no agent in it ever has a reason to call reindex. "Nothing called it in 216 runs" measures the bench, not the need. The tool is for a session where the tree changes under the agent, or where auto-sync is off, and at least one downstream client (Lantern) allowlists it. Removing it may still be right, but it should be decided on sessions where files change, land in its own PR, and carry a breaking-change note.
4. Retiring the "show the usage line" request is a product decision, not a cleanup. It changes what every user sees after every answer. The study makes a good case that the steering came from the field description, and the field stays, but the visible receipt was a deliberate choice and the call should be made explicitly, not inside a description trim.
5. Before anything here merges, please reword the messages of 63fe538 and dacab7e, and line 224 of docs/benchmark.md. They cite a document that is not in this repository. History and docs here are public; the design rationale belongs in the docs page itself or nowhere.
Suggested order: rebase this branch down to the find work plus the bench and measurement docs with item 5 fixed, merge it, let the train release it, then open the surface PR on top.
2 participants
find, the measured tool surface, and 0.5.0
Against
main(eec2fe7:search,sql,reindex), this branch addsfind, measures the whole tool surface lever by lever, and ships the surface that won: three tools, trimmed prose,reindexremoved. Everything below is measured on the same pinned repo (infino @ ed4e020), model (claude-sonnet-4-6), harness and question sets.main vs shipped, Sonnet
Sums over questions of the per-question median.
mainfrom thefindcomparison run (pinpoint 2 repeats, shipped set 3); shipped build (V8) from the surface run (3 repeats). Known-file and by-meaning sets were added for the surface run and have nomainpass.Where it comes from, in order of size:
findon lookups. Definitions, call sites and repo-wide inventories go from grep-then-read to one to threefindcalls; the env-var inventory's worstmainrun was 23 calls and 380k tokens.findreturns every matching line aspath:line, complete and unranked, with per-file counts, and every hit is a real occurrence (token match picks candidate chunks, each line is checked for the literal, no file is scanned).reindexgone. Selection did not depend on the prose:sqlfirst on 30/30 aggregation runs andsearchfirst on 18/18 comprehension runs on every build measured, long or short.searchdescription; Haiku doubled its Reads without it), what theusagereceipt reports (dropping it flipped two lookups fromfindto Grep, 12/12 vs 15/15 across all runs), and thatlangis the file extension (every trimmed build paid a three-call detour without it).Quality
Blind pairwise judge (
claude-opus-5, Read/Grep/Glob on the clone, both answers in random order, unsupported-claim counts):findbuild vsmain: 22 to 29 with 13 ties over 64 pairs, unsupported claims 143 vs 159. Level.findanswers carried 111 line-ranged citations on the lookup set againstmain's 22.findbuild: 51 to 33 with 23 ties over 108 pairs, unsupported claims 236 vs 302, ahead on every set but comprehension (7 to 7). The aggregation gain is a secondsearchaftersqlon the "with reasons" questions: the answers cite their reasons instead of inventing them.Not shipped, measured and rejected
searchtocontext: 18/18 first calls either way, by-meaning tokens higher. Branchexp/surface-v4a.searchresult (content on the top 3 hits, one-line excerpts below): −11% dollars, +35% round trips, judge level. Branchexp/surface-v5.Breaking
reindexMCP tool is removed. No Sonnet run called it in 216; Haiku called it in 2 of 3 runs of one aggregation question, once as its first call. The first query builds the index, every query re-syncs it,cx index --fullrebuilds from a shell. 0.5.0;package.json, plugin manifest,.mcp.jsonpin,server.jsonand the CLI version move together (the last two had drifted at 0.1.4).llms.txt,context7.json, wiki notes and the architecture diagram say three tools.Verification
npm run build,npm test: 133 tests green at the tip.find3/3 under one build, Grep 3/3 under a neighbour differing by another tool's name). Set totals are the unit; one question is not.Full tables, the Haiku passes, and the per-lever ablation:
docs/benchmark.md, sections "find, the grep replacement" and "The tool surface: names, shapes, and prose". Harness:bench/.Follow-ups
hook/block-grep) now points its denial message atfind; merge it after this..mcp.jsonpin resolves.