Skip to content

clean_json_response corrupts JSON values containing code fences #2420

Closed as not planned

Description

@simpleqt

Bug Description

clean_json_response in src/memos/mem_os/utils/format_utils.py strips markdown code fences with a global response.replace("```json", "").replace("```", ""). Any JSON string value that itself contains a fenced code block is silently corrupted before json.loads:

clean_json_response('```json\n{"text": "```py\\ncode```"}\n```')
# actual:   '{"text": "py\\ncode"}'          <- fences stripped from the value
# expected: '{"text": "```py\\ncode```"}'

Both api/handlers/chat_handler.py and api/handlers/suggestion_handler.py pipe LLM responses through this function before json.loads, so suggestions and chat metadata carrying markdown snippets come back with mangled content.

Expected Behavior

Only an opening fence (with its optional json info string) at the very start and a closing fence at the very end are markup; interior occurrences must be preserved.

Environment

MemOS main (a7367d0), python 3.14.

Additional Context

Fix + red/green-verified tests ready; PR to follow referencing this issue.

Activity

  1. added
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    area:coreMOS 编排层 / 框架底座 / 跨模块问题
    status:in-progressSomeone or AI is working on it | 人工或 AI 正在处理
    types:bugSomething isn't working | 功能异常
    on Sep 28, 2026
  2. Memtensor-AI commented on Sep 28, 2026

    @Memtensor-AI
    Collaborator

    🤖 AutoDev has picked up this issue and started working on it.

    Task ID: d7dd51fe8ad9ac03
    Working branch: bugfix/autodev-2420-20260928074010713
    Target branch: latest dev* branch
    Workflow: opsp (analysis → coding → testing → PR)

    I will post the PR link here once done. If I need more information, I will ask in the comments.

  3. self-assigned this
    on Sep 28, 2026
  4. Memtensor-AI commented on Sep 28, 2026

    @Memtensor-AI
    Collaborator

    ✅ AutoDev task d7dd51fe8ad9ac03 completed.

    Summary: Fix issue #2420 — clean_json_response was globally stripping every occurrence via str.replace, corrupting JSON string values that legitimately contained fenced code blocks (e.g. {"text": "py\ncode```"}) before json.loads. Both chat_handler and suggestion_handler feed LLM output through this helper, so any suggestion or chat metadata carrying a markdown snippet was silently mangled.

    The fix rewrites clean_json_response in src/memos/mem_os/utils/format_utils.py to recognise only the outermost fence: an anchored regex ^[^\n`]*\n matches an opening fence line (with optional info string like "json"), paired with a trailing at the very end. When both are present, only those delimiters are sliced off; interior fences are never candidates for removal. The None-input ValueError guard from #1525 is preserved.

    Tests: added 4 red-then-green regression cases in tests/mem_os/test_format_utils.py covering wrapped/unwrapped payloads with interior py blocks and literal json snippets (including the exact reproduction from the issue). Full test run: 12/12 in test_format_utils.py green; 40/40 across tests/mem_os/ green; ruff check + format --check both clean. Commit 95dbd64 pushed to bugfix/autodev-2420-20260928074010713.

    Base branch: main
    Branch: bugfix/autodev-2420-20260928074010713
    Commit: 95dbd64
    PR: #2422
    Assigned to: @syzsunshine219
    Reviewers: @WeiminLee

  5. added
    ai:testingAI agent is running tests | AI 正在运行测试
    and removed
    ai:codingAI agent is coding | AI 正在编码
    on Sep 28, 2026
  6. github-actions commented on Oct 31, 2026

    @github-actions
    Contributor

    This issue has been automatically marked as stale due to inactivity.

  7. added
    status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.
    on Oct 31, 2026
  8. github-actions commented on Nov 8, 2026

    @github-actions
    Contributor

    This issue has been automatically closed due to inactivity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务ai:testingAI agent is running tests | AI 正在运行测试area:coreMOS 编排层 / 框架底座 / 跨模块问题status:in-progressSomeone or AI is working on it | 人工或 AI 正在处理status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.types:bugSomething isn't working | 功能异常

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions