Put -n on the ranker the list already uses - #52
Open
devcodes9 wants to merge 1 commit into
Open
Conversation
-n was a second search path: an AND of raw substrings over message rows, sorted by date. No BM25, no stemming, no typo tier, no demotion of SDK-spawned runs, and one session repeated once per matching message. `agsearch -n "heldout evluation"` returned "No matches" while the same query put the right session first in the TUI, so the typo tolerance the README demos did not exist on the surface it demos it from. It matters more than an -n flag sounds like. `uvx agsearch -n "stripe tax id"` is the README's zero-install first command, and print_matches is also what runs when fzf is missing. The path most new users met was the unranked one, and every ranking change we measure in the eval never reached it. So -n calls rank_sessions now, and rank_matches/fmt_row are gone rather than left for the next caller to pick up by accident. Output is one entry per session led by its session id, with the matching line indented under it. The id is the point: a hit you cannot open is a hit you cannot use. Also here, because they are the same seam: - `agsearch read <sid>` prints a whole conversation without resuming it. The code was already there as the TUI's ctrl-o, reachable only as an internal subcommand. - Colour is chosen at the edge, in _emit. A terminal gets it; a pipe or NO_COLOR gets text, including for escapes quoted inside a transcript. The fzf paths pass color=True explicitly, since their pager is a pipe but their reader is human. Ranking is untouched: gold 0.733/0.800/0.795 and held-out 0.490/0.618, identical before and after. -n costs ~0.7s per run against 400 sessions, up from ~0.25s, because it now builds the same per-session index the list does. 103 tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug behind the bug
Step 2 started as "add a session id to `-n` and stop printing ANSI into a pipe". Both are real, but they are symptoms.
-nand the interactive list were two different search paths:-n(before)rank_sessions: BM25 over title / first prompt / body, recency and resume-count boostrank_matches:all(t in line.lower()), sorted by datemigration≈migrate)Demonstrating it on my own corpus:
The typo tolerance the README's demo GIF sells does not exist on
-n.This is not a corner.
uvx agsearch -n "stripe tax id"is the README's quick start, the first command a new user runs.print_matchesis also the fallback when fzf is missing. So the surface most people meet first was the unranked one, and every ranking improvement the eval measures never reached it.A test pinned the old behaviour as intentional (
test_n_mode_keeps_raw_substring_and), so this was a deliberate simplification once, not drift. It stopped being defensible when-nbecame the surface scripts and coding agents use.Change
-ncallsrank_sessions.rank_matchesandfmt_roware deleted rather than left for the next caller to pick up by accident.Each result is one session, led by its id, with the matching line indented under it:
The id leads because it is the only field another program needs. A hit you cannot open is a hit you cannot use, which is why the two additions below come with it:
agsearch read <session-id>prints a whole conversation without resuming it. The code was already there as the TUI's Ctrl-O, reachable only as an internal subcommand._emitseam. A terminal gets colour; a pipe, orNO_COLOR, gets text. That includes escape sequences quoted inside a transcript, and ones a snippet cut in half. The fzf paths passcolor=Trueexplicitly, because their pager is a pipe but their reader is human.Cost
-nnow builds the same per-session index the list does: ~0.7s per run on a 727-session corpus, up from ~0.25s. Accepted rather than optimised. The TUI pays the same cost at startup, it is a one-shot CLI call, and cachingsessions.tsvagainst the index would be cache-invalidation work this change does not need.Verification
Ranking is untouched, and the eval confirms it:
103 tests pass. Also checked by hand:
_filterstill emits its 4-field fzf rows,_transcriptkeeps colour through the pager,NO_COLORis honoured on a tty, andreadexits 1 on an unknown id.New tests in
tests/test_noninteractive_rows.pypin the contract: one entry per session however many messages match, the id leads the line, no ANSI in a pipe, a typo still finds the session.Why now
Groundwork for #46. A skill built on the old
-nwould have inherited the weaker ranker and had no way to open what it found.