mcp: broaden reconnect retry, add per-server backoff, keep tools visible during reconnect

[Legoben]

Problem

• ReconnectingPeer retried a failed tool call only on ServiceError::TransportClosed. A failed streamable-HTTP POST surfaces as TransportSend (the worker keeps running), so the common failure mode was never retried.
• Nothing bounded how often a persistently failing server could be respawned.
• During a reconnect window the server left active_servers, so its tools vanished: tool calls failed with "MCP server for tool not found" and the UI tool list flickered empty.

Changes

• Broadened retry (flag-gated by McpSelfHeal): a failed call retries once after reconnect when the error classifies as Transient or AuthExpiredRecoverable and is_safe_to_resend proves the request never executed. Resend safety is deliberately conservative: pre-execution rejections (401/403/408/429), connection failures, and broken pipes retry; 5xx-after-accept and response timeouts never do, so non-idempotent tools can't double-run.
• Per-server circuit breaker: repeated reconnect failures back off with the shared jittered-exponential schedule (500ms → ~32s cap); while blocked, reconnect requests fail fast with an actionable message naming the server. Cleared on the next successful connection.
• known_servers index: tools/resources of every successfully connected server stay cached, so during reconnect windows (server retained in spawn_configs but not active) the peer factories, tools()/resources() views, tool_input_schema, and server_from_tool/resource keep working — the reconnect is triggered on use.

Tests

Retry-decision tests (retry on re-mintable expiry, never on 5xx/timeout/user-fixable auth, flag-off = old behavior), breaker bookkeeping and fail-fast, known-server visibility lifecycle and flag-off inertness, plus new is_safe_to_resend coverage in cargo test -p mcp.

[Legoben]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• mcp: detect dead transports, add Reconnecting state, background reconnect #15540
• mcp: proactively re-mint managed proxy tokens before expiry during agent runs #15539
• mcp: re-mint expired managed proxy tokens on demand instead of failing or misrouting to OAuth #15538
• mcp: broaden reconnect retry, add per-server backoff, keep tools visible during reconnect #15537 👈 (View in Graphite)
• mcp: classify auth failures (HTTP + SSE) and guard OAuth credential deletion #15536
• mcp: retain spawn configs so reconnect works for ephemeral/managed/builtin servers #15535
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.