Repository navigation
Fix #1722: fix: memory_search returned error - #2015
Memtensor-AI wants to merge 1 commit into
Conversation
…ensor#1722) Long-running Hermes sessions (1-2 h) were hitting `[timeout] memory.search did not respond within 30.0s` and staying broken for the rest of the session. Three cooperating defects in the Python bridge adapter allowed a dead / hung Node bridge subprocess to poison every subsequent request without ever triggering a reconnect: 1. `MemosBridgeClient._read_loop` returned silently on stdout EOF, so pending JSON-RPC waiters parked for their full 30 s timeout. 2. `MemTensorProvider._start_bridge_keepalive` only reconnected on `transport_closed`; a `BridgeError("timeout", ...)` from the health probe was dropped at DEBUG. 3. `handle_tool_call` for `memos_search` (and the other read-path memory tools) had no reconnect+retry, unlike `sync_turn`. Fix: * Reader thread's `finally` wakes every pending waiter with `transport_closed` (new `_abort_pending` helper). * `request()` fast-fails via `Popen.poll()` when the subprocess is already dead so we never write to a buffered dead pipe. * Keepalive delegates to `_should_reconnect_after_keepalive_failure`, which reconnects on transport_closed / timeout / dead subprocess. * Read-path memory tools now share `_bridge_request_with_retry`, which reconnects and retries once on `transport_closed` (matches sync_turn). Adds 7 unit tests covering reader-exit cleanup, poll()-based fast-fail, read-path recovery, keepalive reconnect triggers, and the "still broken after reconnect" surface-verbatim behaviour. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
🤖 Open Code ReviewTarget: PR #2015 🔍 OpenCodeReview found 7 issue(s) in this PR. 1.
|
|
Description
Fixes GitHub issue #1722 (
memory_searchtiming out after long Hermes sessions) in the Hermes MemOS Python adapter. Three cooperating defects allowed a dead or hung Node bridge subprocess to poison every subsequent request without triggering a reconnect: the reader thread returned silently on subprocess exit leaving JSON-RPC waiters parked for their full 30 s timeout, the keepalive only reconnected ontransport_closed(ignoring thetimeouterror the previous defect produced), and the read-path memory tools had no reconnect+retry unlikesync_turn.Fix layered across
MemosBridgeClientandMemTensorProvider: (1) the reader loop's newfinallyblock calls a shared_abort_pendinghelper that wakes every pending waiter withtransport_closedin under a second; (2)request()fast-fails viaPopen.poll()when the subprocess is already dead so we never write to a buffered dead pipe; (3) a new_should_reconnect_after_keepalive_failurehelper reconnects on transport_closed / timeout / dead subprocess; (4) a new_bridge_request_with_retryhelper runs one reconnect+retry ontransport_closedfor every read-path memory tool. The JSON-RPC wire protocol and the Node bridge are untouched.Test evidence: added 7 unit tests covering reader-exit cleanup, poll()-based fast-fail, read-path recovery, keepalive reconnect triggers, and the "still broken after reconnect" surface-verbatim behaviour. Full bridge-client suite (33 tests) and adjacent Hermes provider pipeline suite (16 tests) pass.
ruff checkandruff format --checkboth clean.Ready to review: 3 files changed, 427 insertions, 88 deletions, commit d63b3f5 on bugfix/autodev-1722. Specs archive pushed to memos-autodev-specs main (commit da2b3b7).
Related Issue (Required): Fixes #1722
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Automated tests are pending.
Checklist
@MatthewZhuang, @CarltonXiang, @syzsunshine219, @World-controller please review this PR.
Reviewer Checklist