Repository navigation
Fix #2420: clean_json_response corrupts JSON values containing code fences - #2422
Closed
Memtensor-AI wants to merge 2 commits into
Closed
Memtensor-AI wants to merge 2 commits into
Memtensor-AI wants to merge 2 commits into
Conversation
The old implementation stripped every ``` occurrence globally, corrupting
JSON string values that legitimately contained fenced code blocks (e.g.
{"text": "```py\ncode```"}) before json.loads. Both chat_handler and
suggestion_handler pipe LLM output through this helper, so any payload
carrying a markdown snippet came back mangled.
Recognise only the outermost fence: anchor a regex at the start of the
trimmed response for the opening line (``` + optional info string + \n),
pair it with a trailing ``` at the very end, and slice off exactly those
delimiters. Interior fences are never candidates for removal.
Adds 4 red-then-green regression tests covering wrapped/unwrapped
payloads with interior ```py blocks and literal ```json snippets.
Refs: MemTensor#2420
Collaborator
Author
🤖 Open Code ReviewTarget: PR #2422 ✅ OpenCodeReview: Review complete: 0 finding(s) across 2 selected item(s). Generated by cloud-assistant via Open Code Review. |
Collaborator
Author
🔧 Open Code Review requested Agent fixOpen Code Review found 1 issue(s). I have resumed the development Agent to fix them.
The Agent will push a new commit to this PR branch. OCR will recheck after the commit is pushed. |
Tighten clean_json_response so the closing markdown fence must appear on
its own line ('\n```' at end). The prior endswith('```') check was
too permissive: any trailing triple-backtick — including one that
belongs to a JSON string value — could be interpreted as a closing
fence and stripped, corrupting the payload.
Adds a regression test covering '```json\n{"a": "x```"}\n```'
to lock in that interior code fences survive when the wrapper uses a
proper on-its-own-line closing fence.
Addresses Open Code Review finding on PR MemTensor#2422.
Contributor
|
This PR has been automatically marked as stale due to inactivity. |
Contributor
|
This PR has been automatically closed due to inactivity. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix issue #2420 — clean_json_response was globally stripping every
occurrence via str.replace, corrupting JSON string values that legitimately contained fenced code blocks (e.g. {"text": "py\ncode```"}) before json.loads. Both chat_handler and suggestion_handler feed LLM output through this helper, so any suggestion or chat metadata carrying a markdown snippet was silently mangled.The fix rewrites clean_json_response in src/memos/mem_os/utils/format_utils.py to recognise only the outermost fence: an anchored regex ^
[^\n`]*\n matches an opening fence line (with optional info string like "json"), paired with a trailingat the very end. When both are present, only those delimiters are sliced off; interior fences are never candidates for removal. The None-input ValueError guard from #1525 is preserved.Tests: added 4 red-then-green regression cases in tests/mem_os/test_format_utils.py covering wrapped/unwrapped payloads with interior
py blocks and literaljson snippets (including the exact reproduction from the issue). Full test run: 12/12 in test_format_utils.py green; 40/40 across tests/mem_os/ green; ruff check + format --check both clean. Commit 95dbd64 pushed to bugfix/autodev-2420-20260928074010713.Related Issue (Required): Fixes #2420
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Automated tests are pending.
Checklist
@WeiminLee please review this PR.
Reviewer Checklist