Skip to content

fix: GraphStructureReorganizer only optimizes the default cube tenant; periodic schedule never runs; cluster summary crashes on non-JSON LLM output #2480

Open

Description

@smoryan

Pre-submission checklist | 提交前检查

  • I have searched existing issues and this hasn't been mentioned before | 我已搜索现有问题,确认此问题尚未提及
  • I have read the project documentation and confirmed this issue doesn't already exist | 我已阅读项目文档并确认此问题尚未存在
  • This issue is specific to MemOS and not a general software issue | 该问题是针对 MemOS 的,而不是一般软件问题

Bug Description | 问题描述

When MOS_ENABLE_REORGANIZE=true, the GraphStructureReorganizer periodic structure optimization silently works on only one graph tenant — the one configured on the shared default cube (memosdefault) — while data written through the REST API is stored under each request's user_name. As a result, no PARENT / INFERS / FOLLOWS edges are ever created for real tenant data, and the feature appears to "write nodes only, never build edges".

There are also two latent bugs in the same code path:

  1. The 100-second periodic schedule never runs. GraphStructureReorganizer._periodic_optimize_structure registers schedule.every(100).seconds.do(self.optimize_structure, ...) but the loop never calls schedule.run_pending(), so these scheduled jobs are dead code. Only the _reorganize_needed new-node trigger actually executes (again, single-tenant).
  2. _summarize_cluster crashes on non-JSON LLM output. If the LLM returns something that is not a JSON object, _parse_json_result yields a non-dict value and the subsequent response_json.get(...) raises, killing the whole optimization pass (including any already-computed sub-clusters).

Environment | 环境信息

  • Version: main @ a7367d0 (2.0.33)
  • Deployment: docker compose (docker/docker-compose.yml), MOS_ENABLE_REORGANIZE=true, NEO4J_BACKEND=neo4j-community

Root Cause | 原因分析

  • src/memos/memories/textual/tree_text_memory/organize/reorganizer.py: _periodic_optimize_structure calls self.optimize_structure(scope=...) without a user_name, so it defaults to self.graph_store.config.user_name — the default cube's tenant (memosdefault), not the tenants that actually received writes via /product/add.
  • The same method registers schedule.every(100).seconds.do(...) jobs but its polling loop lacks schedule.run_pending().
  • _summarize_cluster assumes the LLM always returns a parseable JSON object and passes the result straight into _create_parent_node, so a single malformed response aborts the entire pass.

Proposed Fix | 建议修复

I have a working patch (validated on a live deployment) that:

  1. Adds Neo4jGraphDB.get_all_user_names() (src/memos/graph_dbs/neo4j.py) — MATCH (n:Memory) RETURN DISTINCT n.user_name, with graceful fallback to the configured user_name.
  2. Adds GraphStructureReorganizer._optimize_all_users(scope) — enumerates all tenants in the graph and runs optimize_structure(scope=..., user_name=...) for each, tolerating per-tenant failures.
  3. Routes both the scheduled jobs and the _reorganize_needed trigger through _optimize_all_users, and adds the missing schedule.run_pending() call.
  4. Makes _summarize_cluster return None when the LLM response is not a JSON object, and guards _create_parent_node / add_edge calls against None.

Note: since BaseGraphDB defines the interface, other graph backends (neo4j-community / polardb / postgres) would need an equivalent get_all_user_names() implementation — or the base class could provide a default fallback to config.user_name.

I am not submitting a PR yet per my own workflow, but I can if the approach is endorsed.

Willingness to Implement | 实现意愿

  • I'm willing to implement this myself | 我愿意自己解决

Activity

  1. added
    area:coreMOS 编排层 / 框架底座 / 跨模块问题
    status:needs-product-reviewRequires product owner confirmation before deciding public repo, transfer, or internal backlog.
    types:bugSomething isn't working | 功能异常
    on Oct 10, 2026
  2. akushonkamen commented on Oct 11, 2026

    @akushonkamen

    Confirmed all three root causes on main @ a7367d0:

    1. Single-tenant blind spot — _run_structure_organizer_loop triggers optimize_structure(scope=...) without user_name (reorganizer.py:168-169), and the neo4j.py methods all resolve user_name if user_name else self.config.user_name → only the default cube's tenant (memosdefault) is ever optimized; per-request user_name tenants written via the REST API never get PARENT/INFERS/FOLLOWS edges.
    2. Dead schedule — the schedule.every(100).seconds jobs (reorganizer.py:158-159) never run: the loop never calls schedule.run_pending().
    3. AttributeError on non-JSON LLM output — extract_first_to_last_brace returns None when the text has no braces and _parse_json_result only catches JSONDecodeError, so None/non-dict values reach response_json.get(...) in _summarize_cluster (:572) and _local_subcluster (:442), killing the whole pass including already-summarized sibling clusters. Also, the JSONDecodeError → {} path silently creates empty parent nodes with embed("").

    @smoryan your proposed fix matches exactly what we found — thanks for the thorough analysis. We're preparing a PR along those lines: get_all_user_names() on Neo4jGraphDB plus a fallback default on BaseGraphDB so postgres/polardb keep today's behavior; _optimize_all_users(scope) that enumerates tenants with per-tenant failure isolation, routed through both the _reorganize_needed trigger and the scheduled jobs; schedule.run_pending() in the loop; and _parse_json_result returning None for anything non-dict, with None-short-circuit at both call sites (which also removes the empty-parent-node pollution).

    Since this carries status:needs-product-review: the one product-level question is whether structure optimization should span all tenants. Our patch keeps it conservative — enumerate the tenants actually present in the graph, and fall back to the configured tenant on backends without the new capability (i.e. no behavior change where the interface isn't implemented). Happy to adjust scope, or to review your live-deployment patch instead if you'd rather land it yourself.

    Transparency note: this analysis and the upcoming patch are prepared with AI coding assistance.

  3. akushonkamen commented on Oct 11, 2026

    @akushonkamen

    PR is up: #2481 — get_all_user_names() (Neo4j impl + BaseGraphDB fallback), _optimize_all_users(scope) with per-tenant failure isolation wired into both the trigger and the schedule, schedule.run_pending() in the loop, and _parse_json_result now returning None for non-dict output with short-circuits at both call sites. Tests: 25 focused tests green (17 defect cases red before the fix), graph_dbs 35 passed / 3 skipped, memories regression unchanged vs baseline. Details in the PR description.

  4. github-actions commented on Nov 12, 2026

    @github-actions
    Contributor

    This issue has been automatically marked as stale due to inactivity.

  5. added
    status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.
    on Nov 12, 2026
  6. added a commit that references this issue on Nov 19, 2026
    4e290c4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area:coreMOS 编排层 / 框架底座 / 跨模块问题status:needs-product-reviewRequires product owner confirmation before deciding public repo, transfer, or internal backlog.status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.types:bugSomething isn't working | 功能异常

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions