Repository navigation
Description
Summary
A malformed LLM extraction response can be converted into an empty mapping by parse_json_result, after which synchronous /product/add reports success with data: [] and no memories are stored.
Related: #1355, fixed by #1977. This appears to be a different failure path: that fix corrected the fallback key in SimpleStructMemReader; this report concerns a parser failure returning {} without raising, so an exception-based fallback is not invoked.
Version and environment
- Upstream
maininspected:a7367d07e55db61099f7b4e2c1108bc5831a24f3(also our checkout base). - Self-hosted Python server, Python 3.11 on macOS.
- LLM:
glm-5.3-flash, through an OpenAI-compatible Z.AI endpoint. - Neo4j Community 5.26.6, Qdrant 1.15.3, local Ollama embeddings.
/product/addwithasync_mode: "sync", explicitly created test cube.
Scope of reproduction: the parser failure below was reproduced from the unmodified upstream function at the pinned commit. The end-to-end API observation came from a local checkout with unrelated keyword-search/feedback and debug-logging changes; we have not reproduced the entire HTTP path on a pristine deployment. At the time of the failure, mem_reader/utils.py was unmodified. The isolated parser reproduction needs no LLM, database or server port.
Minimal deterministic reproduction
Run against the unmodified parser at the commit above, in an environment with MemOS dependencies installed:
from memos.mem_reader.utils import parse_json_result
valid = '{"memory list": [{"value": "synthetic evidence"}]}'
malformed = '{"memory list": [{"value": "synthetic evidence"}],}'
assert parse_json_result(valid) == {
"memory list": [{"value": "synthetic evidence"}]
}
assert parse_json_result(malformed) == {}The trailing comma is invalid JSON. The concern is not that strict parsing rejects it; the concern is that rejection becomes indistinguishable from a successful empty extraction downstream.
Observed HTTP behavior
In a bounded document-ingestion test:
- Created a new cube and submitted the document through synchronous
/product/add. - The extraction provider returned
finish_reason: "stop", with a trailing comma before the final closing brace. This was not a reported output truncation. - The parser logged
Failed to decode JSON: Expecting property name enclosed in double quotesand returned{}. - The downstream log reported
No add/update items prepared. - The API returned:
{"code": 200, "message": "Memory added successfully", "data": []}/product/get_alland subsequent searches in that cube returned zero memories. Later chat requests had no supporting context.
The failed extraction consumed 13,545 provider-reported tokens. We did not retry it inside that frozen evaluation. No source document, private logs, credentials or user identifiers are included here.
Relevant code
- Parser returns an empty mapping after JSON decoding fails.
- Reader calls the parser inside an exception-based fallback.
- Add handler constructs a success response from returned results.
Expected behavior
Please distinguish these outcomes in the extraction/add flow:
- Extraction failed because the model output could not be parsed or validated.
- Extraction succeeded and legitimately found no memories.
- Memories were successfully persisted.
A valid empty extraction should remain supported. An invalid extraction should not be represented solely as “Memory added successfully” with an empty list. A structured diagnostic or propagated extraction failure would let callers handle this without assuming persistence succeeded.
Local mitigation and validation limits
We tested a narrow local mitigation that removes commas immediately before } or ] only outside quoted strings, reparses with json.loads, and emits an explicit repair warning. It does not try to invent missing values or repair arbitrary malformed output.
- Four regression tests cover repair reporting, punctuation/escaped quotes inside strings, unchanged valid JSON, and rejection of a missing value.
- Two of these tests failed before the patch; all four passed afterward.
- The selected complete
tests/mem_readerrun passed: 75 passed, with three warnings; formatting/lint passed in the validation copy. - Replaying the original malformed final response locally recovered five memory objects, with no model calls or database writes.
- A later live run on the patched service stored five memories and supported both searches. However, its fresh model response was already valid JSON, so that live success is not causal evidence that the repair branch handled the original failure.
This mitigation does not solve the general empty-success ambiguity, and syntactic validity does not establish semantic fidelity. We are reporting the failure mode rather than proposing that every empty extraction must be treated as an error.
Activity
Memtensor-AI commentedon Oct 3, 2026
🤖 AutoDev has picked up this issue and started working on it.
Task ID:
f2cc0bcd220b1a33Working branch:
bugfix/autodev-2456-20261003133440133Target branch:
latest dev* branchWorkflow: opsp (analysis → coding → testing → PR)
I will post the PR link here once done. If I need more information, I will ask in the comments.
fix: repair trailing comma in malformed extraction JSON (MemTensor#2456)
Memtensor-AI commentedon Oct 3, 2026
✅ AutoDev task
f2cc0bcd220b1a33completed.Summary: Fix issue #2456: malformed extraction JSON with a trailing comma before
}or](common output from GLM / Z.AI / Kimi style models) was silently converted to{}byparse_json_result, which downstream surfaced asPOST /product/addreturning{"code":200,"message":"Memory added successfully","data":[]}with zero memories persisted. Themulti_modal_struct._process_string_fineexception-based fallback never fired because no exception was raised. Related to #1355 (fixed by #1977) but a distinct failure path — that fix only correctedSimpleStructMemReader's fallback key.The fix adds a narrow last-chance repair in
src/memos/mem_reader/utils.py::parse_json_result: a_strip_trailing_commasscanner walks the payload character-by-character, tracks whether it is inside a quoted string (respecting backslash-escaped quotes), and removes commas that sit immediately before}or]outside strings. On successful repair it emits a[JSONParse] Repaired malformed JSON...warning so the previously-silent failure mode is observable; on genuine failure it still returns{}to preserve the historical contract for the 15 call sites. Zero signature change, zero API contract change.Testing: 11 new regression cases in
tests/mem_reader/test_parse_json_result.pycover the exact repro from the bug body, trailing commas before}/]/newlines/fenced code blocks, safety cases (literal,}inside strings must survive, escaped quotes must not confuse the scanner), and the preserved{}return for genuinely malformed JSON. All 11 pass. Adjacent suites (test_simple_structure,test_simple_struct_fallback) still green: 20/20. Fulltests/mem_reader/run: 80 passed, 2 pre-existing environment failures (test_doc_local_file,test_doc_mixed) requiring the optionalmarkitdownextra — confirmed reproducible on untouched dev-v2.0.36 HEAD. Ruff format + check clean. Archive synced and pushed to memos-autodev-specs main (commit 33ce785).Base branch:
mainBranch:
bugfix/autodev-2456-20261003133440133Commit:
2ea35dfPR: #2457
Assigned to: @syzsunshine219
Reviewers: @WeiminLee
fix(mem_reader): address OCR follow-ups on MemTensor#2456 trailing-co…
fix(mem_reader): raise LLMResponseParseError instead of returning {} …
github-actions commentedon Nov 3, 2026
This issue has been automatically marked as stale due to inactivity.
fix(mem_reader): raise LLMResponseParseError instead of returning {} …
github-actions commentedon Nov 11, 2026
This issue has been automatically closed due to inactivity.
fix: isolate retrieval metadata and propagate extraction failures (#2633
fix(mem_reader): raise LLMResponseParseError instead of returning {} …