Skip to content

XERK-278: one WebSocket frame must not be able to kill the hub - #438

Merged
xerhab merged 2 commits into
mainfrom
XERK-278
Aug 12, 2026
Merged

XERK-278: one WebSocket frame must not be able to kill the hub#438
xerhab merged 2 commits into
mainfrom
XERK-278

Conversation

@xerhab

@xerhab xerhab commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Fixes a remote crash of the hub process reachable with the ordinary operator web login.

Found by the adversarial QA pass on #430's follow-up (XERK-262). The code is pre-existing on main (from XERK-245), so it is fixed here rather than in that PR. Ticket: XERK-278.

The defect

String(x) throws TypeError: Cannot convert object to primitive value for a value with no usable primitive conversion — and pure JSON can express one: {"toString":1,"valueOf":1} gives both hooks as own, non-callable properties.

sanitizeLiveAgents called String() directly on a field parsed out of an /agent/control WebSocket frame. That handler runs inside a socket.on("data") listener with no try/catch above it, and turma/ installs no process.on('uncaughtException'), so the throw exits node.

tunnel connected: victim-host
TypeError: Cannot convert object to primitive value
    at String (<anonymous>)
    at sanitizeLiveAgents (turma/server.js:2278:18)
    at turma/server.js:5126:19
    at Socket.<anonymous> (turma/server.js:3093:7)
Node.js v22.23.1

Why it matters:

  • No agent token needed. agentWsAuthorizedagentAuthorizeduserAuthorized, so the single-user Basic login opens the socket. Reproduced with TURMA_USER/TURMA_PASSWORD only. That login is reachable through the public Cloudflare tunnel.
  • DockerOps runs restart: unless-stopped (compose/turma.yaml:35), so repeating the frame is a sustained outage of the entire control plane — dashboard, tunnels, terminal proxy, archive.
  • One packet. No rate limit, no log beyond the crash.

The fix

Two vectors, both closed:

  1. sanitizeLiveAgents coerces through a new safeString that cannot throw. Fixed in the function that handles untrusted input, not by wrapping its three callers — a coercion that throws is the bug, and a caller-side guard is one the next caller gets added without. An unconvertible type drops the row (a blank type already did); an unconvertible label blanks the label and keeps the row, so the "an agent is running" signal hasLiveAgents reads is not lost.
  2. The frame handler requires the session id to be a string, not merely truthy. liveFanout uses it as a property key, and coercing an object to a key runs the same ToPrimitive. This one only bites once a viewer socket is open on that host (the optional chain short-circuits otherwise), which is why safeString alone does not cover it. Every real id is a string, so this only refuses malformed frames.

Deliberately not fixed with a global uncaughtException handler: that converts the crash into a silent half-state rather than removing it. Worth considering separately.

Verification

  • The original one-frame repro now leaves the hub answering (STILL ALIVE, exit 0); before, the listening socket was gone.
  • Each half reverted independently reddens the suite — neither is decorative.
  • Four new tests in turma/tests/server.test.js: the unit coercions, a real /agent/control socket carrying the poisoned row, a real socket carrying non-string ids with a live viewer attached, and the heartbeat ingest (the third caller — already safe behind the request handler's catch, now pinned so it stays reachable-and-refused rather than becoming a 500 or silently accepted).
  • node --test turma/tests/*.test.js agent/tests/*.test.js .github/scripts/tests/*.test.js1090 passed, 0 failed.

The control-socket tests run in the same process as the rest of the suite, so a regression does not fail one assertion — it takes the whole run down. That is the intended signal.

Not verified

Not exercised against the deployed hub — the repro is local rigs only, per rules of engagement. The restart: unless-stopped consequence is read from the DockerOps compose file, not observed in production.

Found by the adversarial QA pass on XERK-262; the code is pre-existing on main
(XERK-245), so it gets its own PR rather than riding that one.

`String(x)` throws `TypeError: Cannot convert object to primitive value` when a
value has no usable primitive conversion, and pure JSON can express one:
`{"toString":1,"valueOf":1}` supplies both hooks as own, non-callable
properties. `sanitizeLiveAgents` called `String()` straight on a field parsed
out of an `/agent/control` WebSocket frame.

That handler runs inside a `socket.on("data")` listener with no try/catch above
it, and this process installs no `uncaughtException` handler, so the throw
exited node. One frame. DockerOps runs `restart: unless-stopped`, so repeating
it is a sustained outage of the fleet's whole control plane — dashboard,
tunnels, terminal proxy, archive.

It does not need an agent token: `agentWsAuthorized` falls back to
`agentAuthorized` falls back to `userAuthorized`, so the ordinary single-user web
login opens the socket, and that login is reachable over the public tunnel.

Two vectors, both fixed:

- `sanitizeLiveAgents` now coerces through a `safeString` that cannot throw.
  Fixed in the function handling untrusted input rather than by wrapping its
  callers — a coercion that throws is the bug, and a caller-side guard is one the
  next caller gets added without. An unconvertible `type` drops the row (a blank
  type already did); an unconvertible `label` blanks the label and keeps the row,
  so the "an agent is running" signal `hasLiveAgents` reads is not lost.
- The frame handler now requires the session id to be a STRING, not merely
  truthy. `liveFanout` uses it as a property key, and coercing an object to a key
  runs the same ToPrimitive — reachable once any viewer socket is open on that
  host, since the optional chain short-circuits otherwise. Every real id is a
  string, so this only refuses malformed frames.

Deliberately NOT fixed with a `process.on('uncaughtException')`: that would
convert the crash into a silent half-state rather than removing it. Worth
considering separately.

Verified: the original one-frame repro now leaves the hub answering. Each half
of the fix was reverted independently and the suite reddens for both. The third
caller (the heartbeat ingest) is covered too — it was already safe behind the
request handler's catch, and now has a test that it stays reachable-and-refused
rather than becoming a 500 or silently accepted. 1090 node tests green.
XERK-262 (#439) landed on main and touches the same two files, so both
conflicts are with that work. Nothing from either side is dropped.

- turma/server.js — main hoisted LIVE_AGENTS_MAX / LIVE_AGENT_FIELD_MAX to the
  top of the file while this branch added `safeString` next to their old
  position. Kept main's placement and only the new function; taking this side
  wholesale would have declared both constants twice.

- turma/tests/server.test.js — a pure append conflict: both branches added tests
  at the end. Kept both, main's first. (The first splice truncated main's last
  test, whose closing brace sat past the conflict marker; restored.)

One semantic interaction, deliberate on both sides. XERK-262's
`heartbeat: a live-agent field with no primitive conversion never reaches the
record` accepts EITHER a 200 or a 400 and asserts only the invariant, precisely
because this fix was known to be coming: with the coercion no longer able to
throw, that beat is now an ordinary 200 with the poisoned row dropped rather
than a refusal. It still passes. This branch's ingest test pins the post-fix
contract exactly, and now says so and cross-references the other.

Verified after the merge: 1112 node tests green; both halves of the fix reverted
independently still redden the suite; the original one-frame repro still leaves
the hub answering.
@xerhab
xerhab merged commit 19e1f5a into main Aug 12, 2026
5 checks passed
@xerhab
xerhab deleted the XERK-278 branch August 12, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant