Skip to content

Bridge process leak: every turn spawns new bridge.cjs, 4+ processes accumulated #1910

Closed

Description

@haojiajieyangde03-ctrl

Bridge process leak: every turn spawns new bridge.cts, 4+ processes accumulated

Environment

  • @memtensor/memos-local-plugin v2.0 (Hermes adapter)
  • Hermes Agent v0.16.0, macOS 26.5.1
  • Config: skillInjectionMode=full, embedding=local, LLM=deepseek-chat

Problem

Every conversation turn spawns a new bridge.cjs --agent=hermes --no-viewer process. Old processes are not cleaned up. Over a session, 4+ bridge processes accumulate:

$ ps aux | grep bridge.cjs
cctv  509  0.0%  ~70MB  bridge.cjs --agent=hermes --daemon
cctv  2018 0.0%  ~92MB  bridge.cjs --agent=hermes --no-viewer
cctv  2058 0.0%  ~99MB  bridge.cjs --agent=hermes --no-viewer
cctv  6550 0.0%  ~343MB bridge.cjs --agent=hermes --no-viewer  ← 343MB!

The newest process has grown to 343MB RSS. Over longer sessions with many turns, this would lead to significant memory pressure (especially on 16GB macOS devices already constrained by 26.5.1 resource limits).

Expected behavior

Bridge should be initialized once per session, with subsequent turns reusing the existing connection. Or at minimum, old bridge processes should be cleaned up when a new one spawns.

Note

This may be related to NousResearch/hermes-agent#20939 which reports the same symptom from the Hermes side: "MemOS memory provider spawns new bridge.cts process on every single turn."

Impact

  • Memory leak: each process consumes 70-340MB RSS
  • On long-running sessions (50+ turns), this could consume 3-10GB of RAM
  • macOS 26.5.1 already imposes disk write limits; memory pressure from leaked processes compounds this

Activity

  1. added
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    types:bugSomething isn't working | 功能异常
    contrib:help wantedExtra attention is needed | 需要社区帮助
    on Jun 11, 2026
  2. fancyboi999 commented on Jun 12, 2026

    @fancyboi999
    Contributor

    I've reproduced this locally and traced the root cause — claiming this one, fix PR incoming.

    TL;DR: accumulation of live --no-viewer bridges = multiple live MemosBridgeClient instances in the Hermes host process. Two compounding sides:

    1. Host side (hermes-agent): before fix(run_agent): isolate background review fork from external memory plugins NousResearch/hermes-agent#27190, background review / delegate forks rebuilt the memory provider per fork (load_memory_provider() constructs a fresh MemTensorProvider on every call — module is cached, instance is not), and the abandoned fork never calls shutdown(). oneshot.py still lacks skip_memory=True today.
    2. Adapter side (this repo): the bridge subprocess is owned per provider instance with nothing enforcing the one-bridge-per-process invariant, and three reference legs make abandoned instances immortal: (a) bound-method hooks appended to the host's global plugin manager and never removed, (b) a per-instance keepalive thread that never stops — and even resurrects the bridge if you kill -9 it (verified: killed pid, new bridge respawned within one 5s cycle), (c) reader thread ↔ Popen stdin pipe pinning each other so GC can never trigger stdin-EOF exit. Also initialize() doesn't close an existing self._bridge, so re-init leaks too.

    Deterministic repro (real unmodified adapter, stub bridge with identical lifecycle semantics): 3 abandoned provider instances → exactly 4 live --no-viewer processes, matching the ps output in this report.

    Fix direction (defense on the adapter side, independent of host behavior): promote the bridge client to a process-wide shared resource (refcounted module-level singleton — the JSON-RPC protocol is already multi-session), single keepalive bound to the shared client, hook registration deduped at process level and unregistered on shutdown, idempotent initialize().

  3. self-assigned this
    on Jun 15, 2026
  4. Memtensor-AI commented on Jun 15, 2026

    @Memtensor-AI
    Collaborator

    🤖 AutoDev has picked up this issue and started working on it.

    Task ID: dba10126f96caab2
    Working branch: bugfix/autodev-1910
    Target branch: latest dev* branch
    Workflow: opsp (analysis → coding → testing → PR)

    I will post the PR link here once done. If I need more information, I will ask in the comments.

  5. Memtensor-AI commented on Jun 15, 2026

    @Memtensor-AI
    Collaborator

    ✅ AutoDev task dba10126f96caab2 completed.

    Summary: Fixes #1910: bridge.cjs process leak in @memtensor/memos-local-plugin Hermes adapter where every conversation turn spawned a new bridge.cjs --agent=hermes --no-viewer subprocess without reaping the previous one, accumulating 4+ processes per session (RSS up to ~340 MB each). Linked symptom on Hermes side: NousResearch/hermes-agent#20939.

    Applied three layered defenses so at most one live stdio bridge exists per agent:
    (1) bridge_client.py adds a module-level _ACTIVE_CLIENTS singleton tracker keyed by (agent, no_viewer). MemosBridgeClient.__init__ synchronously closes any displaced predecessor; close() only evicts the slot when it is still the current owner, so late closes on stale clients never knock out their replacement.
    (2) MemTensorProvider.initialize() is now idempotent: it closes any pre-existing self._bridge before respawning, eliminating the orphan-leak path when the host re-enters initialize on the same instance.
    (3) bridge.cts extends the existing PID-file singleton (#1765) to cover --no-viewer mode via a dedicated bridge-stdio.pid file, separate from the viewer-port owner's bridge.pid so the two paths never collide. Defense in depth across Python processes; activates after dist/ rebuild.

    Verification: 4 new pytest cases (same-agent reap, distinct-agent isolation, stale-close non-eviction, provider idempotency); full unit suite python3 -m unittest test_bridge_client test_hermes_provider_pipeline runs 46 tests, all pass. ruff check and ruff format are clean across the touched files.

    Base branch: main
    Branch: bugfix/autodev-1910
    Commit: f37d223
    PR: #1922
    Assigned to: @MatthewZhuang, @CarltonXiang, @syzsunshine219
    Reviewers: @MatthewZhuang, @CarltonXiang, @syzsunshine219

  6. added
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审
    and removed
    ai:codingAI agent is coding | AI 正在编码
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    on Jun 15, 2026
  7. added a commit that references this issue on Jun 30, 2026
    8156da7
  8. added and removed
    ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审
    on Jun 30, 2026
  9. added a commit that references this issue on Jul 2, 2026
    dddfbe8
  10. added a commit that references this issue on Sep 18, 2026
    eb456d5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:pluginOpenClaw & Hermescontrib:help wantedExtra attention is needed | 需要社区帮助types:bugSomething isn't working | 功能异常

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions