Skip to content

fix(step): resolve search rows structurally instead of re-parsing rendered text - #278

Open
li-xiu-qi wants to merge 1 commit into
stepfun-ai:mainfrom
li-xiu-qi:fix/search-files-greedy-line-parse
Open

li-xiu-qi wants to merge 1 commit into
stepfun-ai:mainfrom
li-xiu-qi:fix/search-files-greedy-line-parse

Conversation

@li-xiu-qi

@li-xiu-qi li-xiu-qi commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #186

Problem

search_files can display the wrong source line when a matched line contains a
:<digits>: shape. That shape is not an artificial edge case: a line that
echoes a compiler error (src/app.ts:42: error), a log prefix (12:30:45: retrying), or a line that quotes grep output all contain it, so a single build
log is enough to hit this.

What the layers know. The grep backends resolve every match exactly — the
file, the line number, and the line text come out of the backend as separate
values. What reaches the Step adapter is not that tuple but a rendered string,
path:line: text. parseNativeSearchRows in step/tool-profile.ts then
re-derives the tuple from that string with a greedy ^(.*):(\d+): (.*)$.

What goes wrong. Take a 45-line log named many.log whose line 1 is
literally src/example.ts:42: build failed, and search for build failed.
Line 1 is the only match, so the only correct output is line 1's own text. The
rendered row is:

many.log:1: src/example.ts:42: build failed

A greedy (.*) takes as much as it can, so the split lands on the last
:<digits>: in the row, not the real one:

group value note
path many.log:1: src/example.ts not a real file
line 42 the real line number is 1
text build failed correct, by luck

renderSearchBlocks then loads line 42 of many.log and prints that text as if
line 1 had matched — the reader gets content that never matched. When the false
line number is past EOF, there is nothing to load, the row is dropped, and the
tool reports (no matches) for a match the backend did find. Those are two
different failures: wrong text shown, or the match lost entirely.

Why the string round-trip is the defect. Tightening the regex is not a fix.
The path and the matched text can both contain the delimiter, so any
regex-based split only decides which of the two gets misread; changing the
greediness moves the bug from one field to the other. The tuple is already known
one layer below. Flattening it to text and parsing it back is the step that can
fail, and it is the step that should not exist.

Changes

The row tuple stops being serialized and parsed. It crosses the tool boundary as
data, and the grep backends stop guessing where the path ends. Three places, and
each one is required.

packages/coding-agent/src/core/tools/grep.ts — the answer leaves as data.

  • GrepToolDetails gains matchRows?: GrepMatchRow[] (path / line /
    text), filled in the same loop that renders the text, so the two views
    cannot disagree. details is local extension metadata and is not sent to the
    model, so model-visible output is unchanged.
  • git grep gets -z and the POSIX backend gets -Z where GNU grep supports
    it, so the backend's own field boundary is a NUL byte instead of a guess.
    parseGrepLine decodes path\0line\0text and path\0line:text, and reverses
    the private use area escaping git grep -z applies to ambiguous path bytes
    (a : in a path arrives as \uF03A). The colon form is still accepted for
    BSD grep and custom backends.

packages/coding-agent/src/step/tool-profile.ts — the answer is read, not
re-derived.

  • searchRowsFromDetails reads details.matchRows; executeSearchFiles uses it
    first and falls back to parseNativeSearchRows only when a search operation
    reports no structured rows (custom / remote backends that emit plain text).
  • details keeps matches as the rendered match count (existing contract) and
    adds matchRows in the same shape, so a consumer has one format to read.

packages/coding-agent/test/step-tool-profile.test.ts

  • Matched text containing :<digits>: renders the real line (the reported case).
  • A path containing :<digits>: renders the real line (skipped on Windows, where
    : cannot appear in a filename).
  • details.matchRows carries the resolved row, so consumers never parse text.

Why all three. Structured rows alone would not help while the git/POSIX
backends still split path:line:text on the colon — a file named
report:7:notes.txt is already wrong before the row reaches search_files.
NUL output alone would not help the Step adapter, which would still parse the
rendered string. A regex change alone only moves the ambiguity. Together, no
step between the backend and the renderer has to guess.

Behavior and compatibility

  • Rendered output is unchanged for rows that parse unambiguously.
  • parseNativeSearchRows is module-private, and the fallback keeps working for
    backends that only emit text.
  • GrepToolDetails.matchRows is a new optional field; matches, backend,
    truncation, filesMatched and truncated keep their meaning.
  • No changes to public interfaces, configuration keys, CLI arguments, persisted
    sessions, or provider responses.
  • grep -Z is skipped on platforms whose grep does not support it (BSD /
    macOS), where the colon form is used as before.

Validation

Repro. The two logs from Problem: many.log (45 lines, line 1 is
src/example.ts:42: build failed) and short.log (2 lines, line 1 is
src/example.ts:99: build failed). Line 1 is the only match in both, so the
expected output is line 1's own text.

Before (main @ be88ee7) — one search, both symptoms:

many.log:1: src/example.ts:42: UNRELATED_LINE_42   <- line 42's text shown as line 1's match
(no matches)                                       <- short.log: match dropped, not shown wrong

After (this branch) — both rows carry the text that matched, and the past-EOF
case returns a row instead of an empty result:

many.log:1: src/example.ts:42: build failed
short.log:1: src/example.ts:99: build failed

No rendered row is parsed on this path any more, so nothing is left to mis-split.

Checks.

  • New tests cover the reported case, the mirrored path case, and the structured
    row payload.
  • vitest run test/step-tool-profile.test.ts test/grep-fallback.test.ts —
    38/38 pass.
  • npm run check — green (biome, tsgo with zero errors, repo check scripts).
  • Backend output checked byte for byte: git grep -n -z emits three
    NUL-terminated fields — many.log, 1, src/example.ts:42: build failed —
    and grep -r -n -Z emits ./weird:42:.txt NUL 2:hit here; both parse to the
    right row, with the colon inside the path or the text intact.
  • Verified on Windows 11 x64; the path case runs on the CI platforms that allow
    : in filenames.

Limitations

  • None known for the reported class: both ambiguous shapes (matched text and
    path) resolve correctly.
  • BSD grep (macOS) has no -Z, so that backend keeps the colon form there;
    the git backend covers the same case on those platforms via -z.

…dered text

search_files could return the wrong source text when a matched line itself
contains a :<digits>: shape. The native grep tool resolves path, line and text
as structured values, then the tool boundary flattens them into
"path:line: text" and the Step adapter parses that string back into a row. Its
greedy path group re-read the :<digits>: shape inside the matched text as the
line number and reloaded that line, or reported "(no matches)" when the false
line number was past EOF.

Tuning that regex only moves the ambiguity to paths, since both fields may
contain the delimiter. So the re-parse is removed instead:

- grep reports the rows it already resolved in details.matchRows (details are
  extension metadata and are not sent to the model).
- search_files reads those rows and only falls back to parsing rendered text
  for custom/remote search operations that emit plain text.
- the git and POSIX grep backends request NUL-separated output (-z / -Z) so
  their own parse carries the field boundary. git grep -z escapes ambiguous
  path bytes into the private use area (0xF000 + byte), decoded back here;
  grep -Z is GNU-only, so it is gated and the colon form stays supported.

Both directions are now correct: matched text containing :<digits>:, and a path
containing it.
@li-xiu-qi
li-xiu-qi force-pushed the fix/search-files-greedy-line-parse branch from 34515bc to 570e018 Compare October 20, 2026 17:19
@li-xiu-qi li-xiu-qi changed the title fix(step): keep the real line number when search text contains :<digits>: fix(step): resolve search rows structurally instead of re-parsing rendered text Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

search_files can substitute the wrong line when matched text contains :number:

1 participant