Skip to content

Retry transient websocket auth failures - #246

Draft
ksbharaj wants to merge 1 commit into
mainfrom
codex/retry-transient-websocket-auth
Draft

Retry transient websocket auth failures#246
ksbharaj wants to merge 1 commit into
mainfrom
codex/retry-transient-websocket-auth

Conversation

@ksbharaj

Copy link
Copy Markdown
Contributor

Summary

  • retry the engine's transient missing-websocket-auth-header response on a fresh zoo-kcl execution
  • keep invalid credentials and unrelated non-retryable KCL errors fail-fast
  • cover the exact response observed in Zookeeper traces

Why

The native zoo-kcl client supplies its bearer token on every WebSocket HTTP
upgrade. In traces 01a045fc-64b2-7a9f-868f-81c9caf0301b and
01a04620-05b2-7f7c-8fc1-7533ecf8697d, the API intermittently returned:

engine: Please send { headers: { Authorization: "Bearer <token>" } } over this websocket

Other engine calls in the same traces succeeded with the same process
credentials. zoo-kcl currently reports this response as non-retryable, so
MCP's existing execution retry loop does not recover. This PR recognizes only
that distinctive server instruction and opens a fresh authenticated socket.

Validation

  • PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 .venv/bin/pytest -p asyncio tests/test_server.py -k 'execute_with_retries' -vv (6 passed)
  • .venv/bin/ruff check --fix .
  • .venv/bin/ruff format .
  • .venv/bin/ruff format --check --diff
  • .venv/bin/ty check .

@ksbharaj

Copy link
Copy Markdown
Contributor Author

CI note: the exact new test
test_execute_with_retries_recovers_from_missing_websocket_auth_header
passes.

The full live-engine job was retried twice after the initial run. Each attempt
failed in a changing set of engine-backed snapshot/session fixtures because
websockets closed with code 1005 or without a close frame:

  • initial: 259 passed, 5 setup errors
  • rerun 1: 262 passed, 2 setup errors
  • rerun 2: 256 passed, 1 failed, 7 setup errors

These failures are outside the changed retry helper and are non-deterministic
across runs. Local targeted tests, Ruff, and ty all pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant