Skip to content

Cut the url where the failure message is written, not where it is read - #123

Merged
webmatze merged 3 commits into
mainfrom
issue-112-tool-result-url-leak
Sep 10, 2026
Merged

Cut the url where the failure message is written, not where it is read#123
webmatze merged 3 commits into
mainfrom
issue-112-tool-result-url-leak

Conversation

@webmatze

Copy link
Copy Markdown
Owner

Closes #112

The url from mcp.json reached the model's context, transcript.jsonl and every smith sessions export of the session, through the result of a tool call that failed against a running HTTP server.

The path

HttpTransport#die! composes each of its five reasons around @url whole — userinfo, path, query, fragment — and stores it as failure_hint. That one value has two readers, and both put it somewhere it must not be:

  • send (http_transport.cr:50) raises it as a ConnectionError at the next write
  • Client#abandon_pending (client.cr:348) hands it to every caller still waiting as RpcError.new(0, reason, safe_message: reason)both fields the same string

McpTool (mcp_tool.cr:61/63/65) prints whichever arrives into the tool result. Measured before the change, with a fake server that 500s every call:

Error: the MCP server at http://LEAKUSER:LEAKPASS@127.0.0.1:62727/mcp/LEAKPATH?token=LEAKQUERY#LEAKFRAG answered HTTP 500 Its tools are no longer available.

Why safe_message did not already cover it

It answers "are these a server's words?" — and the answer here is genuinely no, which is why the comment at client.cr:352 is right on its own terms. scrub_urls answers a second question: "does this quote the configuration file?" A message smith composed itself passes the first and fails the second. That was the distinction #110 established; this is the surface it had not yet reached.

Where the filter went

The issue proposed abandon_pending. It went one seam earlier, into die!, for a reason the issue could not have known: abandon_pending is not the only reader. send raises the same stored string, and that ConnectionError reaches the same rescue in McpTool. Filtering at composition covers both, leaves nothing raw in the ivar, and means a third reader added later is covered by construction rather than by remembering.

failure_hint has no other readers — checked — and nothing wants the unfiltered form: both uses are messages.

failure_body is untouched. It is the server's own words, it stays apart, and it still reaches no tool result — only smith doctor/mcp list with with_server_output, which is #109's territory.

What survives

The reason word for word (answered HTTP 500, could not reach, a timeout), plus scheme, host and port. Which server it was is @handle.name, which mcp_tool.cr:63 already prints beside it. Nothing diagnostic is lost — this half had no conflict to trade against, unlike #109.

Specs

Both were run with the fix commented out and both fail there:

  1. Through the tool layer — the acceptance criterion the issue asked for: fake server starts, handshake succeeds, tools/call 500s, and the string McpTool returns is asserted against all four secret positions. Not just the start-up path, which smith mcp list gibt die URL aus mcp.json ungekuerzt in der Fehlerzeile aus #110 already covers.
  2. On the stored value — a refused port, then failure_hint itself and the ConnectionError the next send raises. That is the reader the issue did not name.

crystal spec: 1424 examples, 0 failures. crystal tool format --check: clean.

🤖 Generated with Claude Code

webmatze and others added 2 commits September 10, 2026 23:28
`HttpTransport#die!` composed every one of its reasons around `@url` whole
and stored it as `failure_hint`. Both readers of that value put it somewhere
it must not be: `send` raises it at the next write, and `abandon_pending`
hands it to every waiting caller as an `RpcError` whose `safe_message` was
the same string — which `McpTool` prints straight into the tool result.

A tool result is not a line someone reads and discards. It goes into the
model's context, is appended to `transcript.jsonl`, and travels with every
`smith sessions export` of the session. #110 fixed the url in a line a human
reads; this is the same url in a line that outlives the run.

Filtered at composition rather than at either reader: `scrub_urls` runs in
`die!`, so the ivar never holds the raw url and a reader added later is
covered by construction. `safe_message` answers whether a line is a server's
words; `scrub_urls` answers whether it quotes the configuration file. A
message smith composed itself passes the first and needed the second.

The reason survives verbatim, as do scheme, host and port. The server's own
body stays apart in `failure_body` and still reaches no tool result.

Two specs, both shown to fail without the change: one drives a running
server through a failing call to the tool layer, one asserts the stored hint
itself and the `ConnectionError` the next write raises.

Closes #112

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… down

Review found the first version still leaking, and it was right: `scrub_urls`
ends a match at whitespace, a quote or an angle bracket, so a query holding
one of those was cut at that character and everything after it — the token —
left standing. Reproduced end to end:

  mcp.json    : http://127.0.0.1:8080/mcp?filter=<all>&token=LEAKQUERY
  tool result : ...at http://127.0.0.1:8080<all>&token=LEAKQUERY answered HTTP 500

The location was right and the technique was not: scanning a finished
sentence for something url-shaped is a read-site tool, and it had been moved
to the compose site without being replaced. `@url` is a `URI` and the class
already refuses one without a host, so the safe form is available
structurally: `@safe_url` is built once in `initialize` and is what the five
`die!` callers interpolate. No message ever holds the whole url, so no regex
has to find it.

`scrub_urls` stays in `die!` for the other source of a url in these lines —
an exception message from the HTTP client — where a filter is the only tool
there is, and where the url was composed by the stdlib rather than written by
hand, so the shapes that defeat it do not arise.

Also from the review: `Transport#failure_hint` now carries the obligation it
imposes, since a second transport overriding it the obvious way would leak
again with no spec failing; and its "read once" was already untrue, there are
two readers. The poll in the transport spec asserts before `not_nil!`, so a
host that drops rather than refuses fails legibly.

Third spec, failing without the change: a query with an angle bracket and a
token, driven through a running server to the tool layer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… raw url

Review: the ivar's comment claimed "the only form ever written into a
message", and twelve lines below it the no-host refusal writes the raw one.
Scoped to what is true — after construction — with the exception named, plus
where that one line goes and why it stops there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@webmatze

Copy link
Copy Markdown
Owner Author

Review durch einen Reviewer-Agenten, zwei Runden. Erste Runde REQUEST CHANGES mit einem reproduzierbaren Leck, zweite APPROVE WITH NITS.

Was der Review gefunden hat

Die erste Fassung leckte weiter. ServerSpec.scrub_urls beendet einen Match an Whitespace, ', ", < oder > — bewusst, damit die Satzzeichen um eine URL herum nicht mitgeschluckt werden. Die Folge: eine Query mit einem dieser Zeichen wird dort abgeschnitten, und alles danach bleibt wörtlich stehen.

mcp.json    : http://127.0.0.1:8080/mcp?filter=<all>&token=LEAKQUERY
tool result : ...at http://127.0.0.1:8080<all>&token=LEAKQUERY answered HTTP 500

Unabhängig nachgestellt, ebenso mit " und mit einem Leerzeichen. Das Loch stammt aus #110/#111 — neu war nur, dass diese Fassung die Regex zur alleinigen Absicherung einer persistierten Senke gemacht hat.

Die Korrektur

Der Ort war richtig, die Technik nicht: einen fertigen Satz nach etwas URL-Förmigem abzusuchen ist ein Werkzeug für die Lesestelle, und es war an die Schreibstelle verschoben worden, ohne ersetzt zu werden.

@url ist ein URI, und die Klasse verweigert bereits eine ohne Host — die sichere Form ist also strukturell verfügbar. @safe_url wird einmal in initialize gebaut, und die fünf die!-Aufrufer interpolieren sie. Die vollständige URL wird nie in einen Satz geschrieben, also muss sie auch keine Regex darin wiederfinden.

scrub_urls bleibt in die! für die andere Quelle einer URL in diesen Zeilen — die Meldung einer Exception des HTTP-Clients. Dort ist ein Filter das einzige verfügbare Werkzeug, und er reicht: diese URL hat die stdlib komponiert, nicht ein Mensch in mcp.json geschrieben.

Was der Review sonst noch abgeräumt hat

  • Transport#failure_hint trägt jetzt die Verpflichtung, die es auferlegt. Ein zweiter Transport, der es auf die naheliegende Art überschreibt, hätte erneut geleckt, ohne dass ein einziger Spec bricht — beide neuen Specs sind HttpTransport-spezifisch. Sein „read once" war zudem schon vor diesem Branch falsch: es gibt zwei Leser.
  • Der Kommentar an die! beschreibt jetzt, was der Code tut, statt was die erste Fassung vorhatte.
  • Der Poll im Transport-Spec assertiert vor not_nil! — ein Host, der DROPt statt zu REFUSEn, scheitert damit lesbar statt mit blankem NilAssertionError.
  • Der Kommentar an @safe_url behauptete „the only form ever written into a message"; die No-Host-Absage zwölf Zeilen darunter schreibt die rohe. Auf das eingegrenzt, was stimmt, mit der Ausnahme benannt.

Was der Review gegengeprüft und sauber gefunden hat

26 URL-Formen durch URI.parse → to_s → safe_url: IPv6 mit und ohne Port, mit Zone-ID, Default-Port, Userinfo, Trailing-Dot-Host, IDN, %-kodierter Host, Port 0 und 99999, ein programmatisch gebautes URI. Der rescue "(url)" in safe_url feuert in keinem davon, sobald der Host-Guard passiert ist. Dazu alle fünf Meldungs-Vorlagen gegen sechs Host-Formen — dreißig Zeilen, keine vom verbliebenen scrub_urls verändert.

Ebenfalls geprüft: alle drei rescue-Zweige in mcp_tool.cr bis zur Quelle, der Restart-Pfad, registry.crs Catch-all, die stdio-Seite (failure_hint bleibt dort nil), und dass kein Aufrufer die volle URL braucht.

Specs

Drei, jede einzeln als fallend nachgewiesen — die dritte scheitert, wenn nur :118 auf @url zurückgedreht wird, während die anderen sechzehn grün bleiben. Sie ist also nicht redundant.

crystal spec: 1425 Beispiele, 0 Fehler. crystal tool format --check: sauber.

Nicht in diesem PR

ServerSpec#description (server_config.cr:47) gibt die rohe URL zurück und hat null Aufrufer in src/ — drei in den Specs. Ein geladener Lauf, auf den nichts zielt. Eigener Cleanup, nicht dieser.

Der rote macOS-Job auf einem früheren Commit war #114, der bekannte Flake in manager_spec.cr:208 — er fasst nichts an, was dieser PR berührt. Neulauf grün, auf #114 dokumentiert.

@webmatze
webmatze merged commit d49ed2f into main Sep 10, 2026
2 checks passed
@webmatze
webmatze deleted the issue-112-tool-result-url-leak branch September 10, 2026 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Die URL aus mcp.json erreicht ueber ein gescheitertes Tool-Ergebnis den Modellkontext und den Export

1 participant