Repository navigation
feat(dev-proxy): both protocol eras, backend era negotiation, backend start on first use (#1025 step 2, PR 5/5) - #1118
Conversation
…nd start the backend on first use (#1025) Step 2 of #1025, PR 5 of 5: the development loop's proxy. Measured first, as the plan required: a v2 client's `auto` negotiation over the SDK's own StdioClientTransport probes server/discover on a disposable SIBLING process spawned from the same parameters, then starts the real one (the SDK's negotiateStdioViaSibling; a subclass probes in place). The proxy ran main() in full on that sibling, including the unconditional backend.start(), so every probing connect started a backend twice and ran ensurePortFree twice against one port; ten dev-proxy processes had idled for four days under this Claude Code session with two backends between them. The client-facing server is now served through the SDK's stdio entry (tools/dev-proxy/front-server.mjs): one instance per connection in the era the client opens with, the relay target following the pinned instance, the handlers moved out of main(), and an ObservedStdioServerTransport so the proxy exits when the wire closes by itself. The backend starts on the first request that reaches a handler (BackendManager.ensureStarted): the SDK entry answers server/discover itself, so a probe sibling never reaches a handler and never spawns a backend. The first tools/list still waits for the start (#716), and a successful initial start still announces tools/list_changed. The backend Client negotiates its era for an http backend (legacy for a stdio one, where `auto` would probe on a sibling of the backend); dev_server_status reports protocolEra and protocolVersion. Resource subscriptions are translated across eras by backend-subscriptions.mjs: one resources/subscribe per URI to a 2025 backend, re-sent after a restart; ONE subscriptions/listen with the URIs plus the list-changed flags to a 2026 backend, re-opened on set change and after every (re)connect. The backend runs with MCP_DEBUG_SESSION_IDLE_MS=0 unless the environment or an override sets it. Measured after: the same connect lands on modern / 2026-07-28, the sibling spawns nothing, the backend starts on the first tools/list, and the proxy's backend connection is in the modern era. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…alize the relay, and say what a modern pin drops (#1025) Three findings from the review of PR #1118. A 2026-07-28 front client's subscriptions never reached the backend: its subscriptions/listen is served by the SDK entry and never reaches a handler, and relay.subscribe was only called from the resources/subscribe handler a 2026 client cannot send. The relay's set stayed empty, so no output pings flowed for exactly the client this PR exists for. The fixture hid it by pinging without a subscription. Now, while a modern instance is pinned, the relay tracks every resource the backend lists (trackAll), re-read on each relayed resources/list_changed (refreshKnown) and after every (re)connect; the front's own router delivers each client only what it listened for. The fixture backend keeps a subscribe set and refuses a touch for an unsubscribed URI, so the integration test now pins the chain. Every expression against the backend is serialized: two subscribes arriving back-to-back used to both open a listen and leak the first one (and a 2026 backend emits one copy per live subscription). A refused modern listen no longer remembers its URI. The relay only ever sees a non-stopped backend's client. README: a 2026 client's tools/list_changed is dropped unless its listen opted in (toolsListChanged: true), so after an inventory-changing restart such a client keeps its list until it lists again; the http backend lands on 2026-07-28 only when its dist serves it. Live, through the proxy against dist/index.js http: a pinned 2026 client hears the backend's resources/list_changed on create_debug_session and a resources/updated for the session's output once a Python debuggee prints. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…ckend that refuses a subscription, and say when the backend has not started yet (#1025) A second look at three rulings made while building step 2, on JF's review. A stdio backend was pinned to the 2025 era because the SDK's `auto` negotiation probes on a disposable sibling process, which here would be a second backend. The SDK documents the way around that: only its exact base stdio transport takes the sibling path, and a subclass probes on the connection itself. The proxy's backend transport is now such a subclass and negotiates on both transports, so the stdio-backend entries exercise the server's modern stdio leg, with the backend still started once. Measured through the proxy: the current build lands on 2026-07-28; the legacy fixture and the published 0.26.0 npx bundle and Docker image answer the probe with "method not found" and fall back to 2025-11-25 on the same pipe, tool calls working. The subscription relay kept a refused subscription forever and rethrew on it. A backend restart is the proxy's purpose and it ends every debug session, so each subscription a client named before it then points at nothing: the first refusal aborted the re-expression, and every later reopen failed at the same URI. Re-expression no longer stops at a refusal: a URI the client named is dropped and said once, a listed resource the backend will not subscribe (its proxy-log tails) is asked once and remembered until the backend stops listing it. Only a JSON-RPC error answer counts as a refusal; a failed connection still surfaces, and a client's own subscribe still gets its refusal. Smaller: - dev_server_status before the first start says the backend starts on the first request that needs it; "stopped" alone read like a crash. - A request that lands while the proxy is shutting down logs that it is, not "Initial backend start failed" with a hint to retry. - The proxy no longer forces MCP_DEBUG_SESSION_IDLE_MS=0 on its backend: the idle policy is off by default now, so the override was dead. - installShutdownHandlers loses its unused `server` parameter and the stale note that the SDK transport never notices EOF. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…e proxy shuts down; keep a subscription the backend merely failed on (#1025) Found by CI on Linux and by the review of the second-look pass. The previous commit made the stdio backend negotiate its era in place, through a subclassed transport. Dispatched CI then failed on Linux in the rebuild suite: "shuts down with work queued/starting without spawning queued replacements" found the backend process still alive after the proxy had gone. Measured on the SDK client 2.4.0: negotiating in place, the SDK does not watch the signal given to connect(). The connect stays pending past the abort and the child keeps running, where with its base transport the SDK closes the transport itself. So a shutdown that found a start still negotiating never cancelled it, the start's cleanup never ran, and the proxy exited without its backend. (Not a missing pid: transport.pid is set right after connect() in both modes.) The transport now takes the proxy's shutdown signal and closes itself when it aborts, which ends the child and fails the pending negotiation at once, so the start's own failure path cleans up. It lets go of the signal when it closes or when its backend has gone by itself, so restarts do not pile listeners onto a signal that lives as long as the proxy. It moves to its own module so a unit test can hold it on every platform: without the close, connect is still pending four seconds after the abort and the child is alive. From the review: - The relay treated any JSON-RPC error as "that resource is gone". Only InvalidParams (or the older resource-not-found code) says that; an internal error from a live backend now leaves the subscription in place, to be tried again, instead of silently dropping it. - dev_server_status showed its "not started yet" note beside state "running" when the first handled request was a dev restart; the note now follows whether any start has begun. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…either path, not the in-place probe as such (#1025) Comments only. The previous commit said the SDK ignores connect()'s signal when it negotiates in place and closes the transport itself for its base class. Measured with all four combinations on client 2.4.0, the variable is the mode: in legacy mode an abort rejects the connect and the transport is closed, base class or subclass; in auto mode the connect stays pending and the child keeps running, on the sibling path and in place alike. The fix is unchanged and right either way; the comments now name the cause correctly. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Check subscription state inside the operation queue so rapid subscribe/unsubscribe requests preserve arrival order. Add regressions for both orders across both protocol eras.
debugmcpdev
left a comment
There was a problem hiding this comment.
Review reproduced a subscription ordering race: subscribe(A) immediately followed by unsubscribe(A) could finish with A subscribed, and the reverse order could finish unsubscribed. Fixed in 0dc64d3 by moving all duplicate/state checks inside the existing serialized queue. Four regression cases cover both orders and both backend eras; they failed before and pass after the fix, and independent review confirmed the change. The reviewed #1115 idle-expiry correction is also included through the stack. Full CI on the updated branch remains required before merge.
debugmcpdev
left a comment
There was a problem hiding this comment.
Follow-up validation on the updated stack head 9e948b8: all 117 targeted Vitest tests pass (22 idle-policy, 71 child-session, 24 backend-subscription), including every new regression. typecheck:all passes with zero source errors and a 0/0 test baseline; ESLint passes for both changed production files. Frozen installation used the pinned pnpm 12.4.1 and left source/lockfile unchanged. Fresh current-base GitHub CI remains the merge gate.
Retain all merged dependency updates and reviewed fixes, including HTTP shutdown regression coverage, alongside the reviewed dev-proxy changes.
debugmcpdev
left a comment
There was a problem hiding this comment.
Final stack integration review: merged current main 8b80c52 after #1117. The resulting 25-file diff exactly matches the reviewed dev-proxy changes, including the subscription ordering fix and its four regression cases. All merged dependency updates, the idle-expiry fix, the logging security fix, and HTTP shutdown regression are retained. Fresh CI on this head must pass before the final normal merge.
Step 2 of #1025, PR 5 of 5 (stacked on #1117; base retargets as the stack merges). The dev proxy — the dogfood loop this project is developed through — speaks both MCP protocol eras to its client, negotiates the era with its backend, translates resource subscriptions across eras, and no longer starts a backend from a probe sibling.
Step 0: the measurement the plan asked for
The plan required measuring, before any code, whether Claude Code's
autonegotiation probes a stdio server in place or on a sibling process. The SDK source answers it (negotiateStdioViaSibling,readStdioServerParams): a v2 client whose transport is exactly the SDK'sStdioClientTransportrunsserver/discoveron a disposable sibling process spawned from the same parameters, then starts the real transport once the era is known; a subclass probes in place. Claude Code uses the base class.Measured against the pre-change proxy (
scratchpad/pr5/measure-sibling-probe.mjs, ledgered): oneautoconnect spawned twodev-proxy.mjsprocesses under the test's pid — the probe sibling and the real one — and landed onlegacy/ 2025-11-25 in 1439 ms. The sibling ranmain()in full, including the unconditionalbackend.start()at module top level, so every probing connect started a backend twice and ranensurePortFreetwice against the sameDEV_PROXY_PORT. In this Claude Code session tendev-proxy.mjsprocesses had been alive for four days under the Claude Code pid with two backends between them, and every mcp-debugger entry showed CONNECTION_CLOSED at session start.Measured after: the same connect lands on
modern/ 2026-07-28 in 798 ms, the sibling spawns nothing, the backend starts on the firsttools/list([dev-proxy] Starting backendfollows the request in the log), and the proxy's own backend client connects in the modern era.What changes
Front behind
serveStdio(tools/dev-proxy/front-server.mjs, new):createFront({ backend, relay, devTools, handleDevTool, discoveryWaitMs, log })returns the serveStdio factory — one freshServerper connection or probe, in the era the client opens with — pluscurrent()(the newest instance that has not closed; a discarded probe closing later never clears a newer pin),notify()(backend notifications relayed through the pinned instance's typed senders, so the SDK routes them in the client's era: a 2026 client's listen stream, a 2025 client's session) andsendToolListChanged(). The handlers are the onesmain()had, moved;ObservedStdioServerTransport(the PR 3 pattern) makes the proxy exit when the SDK transport closes the wire by itself, not only on stdin EOF.Deferred backend start (
BackendManager.ensureStarted()): the initial start runs on the first request that reaches a handler (tools/list,tools/call,resources/*), never at process start.server/discoveris answered by the SDK entry itself, so a probe sibling never reaches a handler and never spawns a backend. One start per process; a failed one is not retried here (dev_restart_debuggeris the retry); the firsttools/liststill waits for it underDEV_PROXY_DISCOVERY_WAIT_MS(#716 unchanged), and a successful initial start still announcestools/list_changed.Backend era negotiation: the backend
ClientusesversionNegotiation: { mode: 'auto' }for anhttpbackend (which speaks both eras since #1117) and'legacy'for a stdio backend (autoover stdio would probe on a sibling of the backend).dev_server_statusgainsprotocolEraandprotocolVersion(nullwhile stopped).Subscription relay across eras (
tools/dev-proxy/backend-subscriptions.mjs, new):createSubscriptionRelay({ getClient, log })keeps the set of URIs the client subscribed to and expresses it in the backend's era — oneresources/subscribeper URI to a 2025 backend (re-sent after a restart, which used to lose them), ONEsubscriptions/listenwith the URIs plus the list-changed flags to a 2026 backend, re-opened whenever the set changes and after every (re)connect, closed before the backend goes away. Notifications still arrive through the client'sfallbackNotificationHandler, as before.Backend defaults (
withBackendDefaults): the backend runs withMCP_DEBUG_SESSION_IDLE_MS=0unless the environment or an override sets it — one client, and the proxy's lifetime is the backend's.Tests
tests/unit/dev-proxy/front-server.test.ts(new, in process overInMemoryTransportbehindserveStdio): a 2026-07-28 client lands on a modern pin andtools/listmerges backend and dev tools afterensureStarted(connecting alone starts nothing); a 2025 client's subscribe/unsubscribe reach the relay; dev tools are answered locally and other calls start the backend and forward; a legacy client hears a relayedresources/updated; a modern client hears one on its listen stream and atools/list_changed; no pin → dropped, not thrown; the pin clears on close.tests/unit/dev-proxy/backend-subscriptions.test.ts(new): both eras, reopen after restart, duplicate subscribe, close keeps the set, no-client buffering, a refused URI is not remembered.tests/unit/dev-proxy/backend-env.test.ts: the idle default and its precedence.tests/integration/dev-proxy-eras.test.ts(new, the SDK's baseStdioClientTransportas Claude Code uses it): anautoclient lands on 2026-07-28 with zero backend starts after the probe and the connect,dev_server_statussaysstopped/protocolEra: null, the firsttools/liststarts exactly one backend; a 2025 client is unchanged; a pinned 2026 client'slistenreceives aresources/updatedthe 2025 fixture backend emitted for a URI the proxy subscribed on its behalf (fixture gainedresources/subscribeand atouchargument).dev-proxy-startup(dev-proxy: initial tools/list can expose only development tools while a healthy backend starts #716) anddev-proxy-rebuildsuites unchanged and green.Verification
measure-sibling-probe.mjs auto→era=modern version=2026-07-28, backend started on the firsttools/list, backend client in the modern eraDocs
tools/dev-proxy/README.md(eras, deferred start, the idle default), CLAUDE.md dev-proxy section, fragment1025-dev-proxy.changed.md.Final proof still to come
Claude Code restarted against this proxy: its
mcp-logs-*should record"protocolEra":"modern"for the dev-proxy entries. That needs a Claude Code restart, which ends the session that built this; it is the first thing to check after merge.Review fix pass (commit 2)
A fresh-context review of this PR found no Critical and three Important findings, all fixed here RED→GREEN:
subscriptions/listenis served by the SDK entry and never reaches a handler, and the relay was only fed by theresources/subscribehandler a 2026 client cannot send — so the relay's set stayed empty and no output pings flowed for exactly the client this PR exists for. The fixture hid it by pinging without a subscription. Now, while a modern instance is pinned, the relay tracks every resource the backend lists (trackAll), re-read on each relayedresources/list_changed(refreshKnown) and after every (re)connect; the front's own router delivers each client only what it listened for (a test pins that a modern client hears nothing for a URI it did not listen for). The fixture backend keeps a subscribe set and refuses atouchfor an unsubscribed URI, sodev-proxy-eras.test.tsnow proves the chain.express()was not serialized. Two subscribes arriving back-to-back both opened a listen and leaked the first (a 2026 backend emits one copy per live subscription). Every expression against the backend now runs through one promise tail; a test drives two subscribes before either listen settles and asserts one open listen and one close.tools/list_changeda 2026 client did not opt into (toolsListChanged: trueon its listen), so after an inventory-changing restart such a client keeps its tool list until it lists again. Documented, with the hedge that anhttpbackend lands on 2026-07-28 only when its dist serves it.Also fixed from the minors: the relay only ever sees a non-stopped backend's client (a crash left a closed client behind); a refused modern listen no longer remembers its URI;
ObservedStdioServerTransporthas a unit test; the eras test's firstbackendStartsread is annotated as the probabilistic one.Measured live (
scratchpad/pr5/live-modern-relay.mjs, through the proxy againstdist/index.js http): a pinned 2026-07-28 client hears the backend'sresources/list_changedoncreate_debug_sessionand then aresources/updatedfordebug://sessions/<id>/outputonce a Python debuggee prints (the mock adapter prints nothing, which is why the probe uses Python).dev_server_statusreportsprotocolEra: modern,protocolVersion: 2026-07-28.Deferred minors (ledgered): the typed-sender switch in
notify()is cosmetic;dev_server_statusbefore first use saysstoppedwith no "starts on first use" hint; a request landing during shutdown logs a misleading "use dev_restart_debugger". Ruled: the relay over-subscribes the backend to every listed resource while a 2026 client is pinned (the plan's design; the SDK entry exposes no view of the client's filter) — the extra pings are coalesced per URI and filtered at the proxy.Second look (commits
84d1bad8,627a3095,9132a2e0)A stdio backend negotiates in place. I had pinned it to the 2025 era because the SDK's
automode probes on a disposable sibling process — here, a second backend. The SDK documents the way around that: only its exact base stdio transport takes the sibling path; a subclass probes on the connection itself. The proxy's backend transport is now such a subclass (tools/dev-proxy/backend-stdio-transport.mjs), so the stdio-backend entries exercise the server's modern stdio leg with the backend still started once. Measured through the proxy: the current build lands on 2026-07-28; the legacy fixture, the published 0.26.0 npx bundle and the published Docker image answer the probe with "method not found" and fall back to 2025-11-25 on the same pipe, 32 tools listed and a tool call working (557 ms and 1024 ms to first status).What CI caught in that change. The first version passed locally on Windows and failed in dispatched CI on Linux: the rebuild suite's "shuts down with work queued/starting" cases found the backend still alive after the proxy had gone. Measured on the SDK client 2.4.0 with all four combinations of transport class and mode: under
autonegotiation the SDK does not watch the signal given toconnect()while its probe is unanswered — the connect stays pending past the abort and the child keeps running, on the sibling path and in place alike — where inlegacymode the abort rejects the connect and closes the transport. (My first commit message blamed the in-place path;9132a2e0corrects the comments.)627a3095gives the transport the proxy's shutdown signal: it closes itself on abort, which ends the child and fails the pending negotiation at once, so the start's own cleanup runs. A unit test holds it on every platform (without the close, connect is still pending four seconds after the abort and the child is alive).The relay survives a backend that refuses a subscription. It kept a refused subscription forever and rethrew on it. A restart ends every debug session, so each subscription a client named before it then points at nothing: the first refusal aborted the re-expression and every later reopen failed at the same URI. Now a named URI the backend says is gone is dropped and said once, a listed resource it will not subscribe (its proxy-log tails) is asked once, any other error answer leaves the subscription in place to be retried (review finding: I had treated every JSON-RPC error as "gone"), a failed connection still surfaces, and a client's own subscribe still gets its refusal.
Smaller:
dev_server_statusbefore any start says the backend starts on the first request that needs it (and stops saying so once any start has begun — review finding); a request during shutdown logs that the proxy is shutting down, without a retry hint; the dead idle override and the unusedserverparameter are gone.Verified:
typecheck:all, lint, check-docs, changelog; dev-proxy integration 3 files / 27 tests; pre-push hook unit 421 files / 8814 tests, integration 37 files / 182 tests.CI on this branch head (dispatched by hand: the workflow triggers on PRs to
mainonly, so a stacked PR gets no automatic run andgh pr checksshows nothing): https://github.com/debugmcp/mcp-debugger/actions/runs/42885469072 (head9132a2e0, 12 of 12 jobs green; run 42879532417 was green on627a3095before the comment-only commit, and run 42865939654 on84d1bad8is the Linux failure described above)🤖 Generated with Claude Code