Skip to content

fix: isolate retrieval metadata and propagate extraction failures - #2633

Merged
endxxxx merged 5 commits into
dev-v2.0.37from
fix/retrieval-isolation-extraction-errors
Nov 19, 2026
Merged

endxxxx merged 5 commits into
dev-v2.0.37from
fix/retrieval-isolation-extraction-errors

Conversation

@endxxxx

@endxxxx endxxxx commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

Description

Ranked search results could share nested metadata with the original candidates, and malformed LLM extraction responses could be converted into successful empty writes by reader fallback and worker exception handlers.

Selected searched metadata is now copied deeply without Pydantic revalidation. Parsing failures propagate through simple and multimodal extraction, document workers and public reader boundaries. Synchronous /product/add returns HTTP 500 before storage or scheduling on a parse failure, while valid {} and {"memory list": []} remain successful empty results. Async fine-extraction failures retain the existing fast memories and emit a failed task event.

This PR supersedes #2600 and #2607 and targets dev-v2.0.37. Original contributions from @yiwang-one and @RerankerGuo are preserved in separate cherry-picked commits, followed by the metadata-isolation and write-path fixes. Maintainer edits remained disabled on the original PRs.

Related issue #2456.

Type of change

  • Bug fix

How Has This Been Tested?

  • Unit tests and regression tests, using synthetic data and mocked model/storage boundaries
  • Test steps completed with Python 3.12
  • Automated API tests exercise the real AddHandler, SingleCubeView and SimpleStruct reader through POST /product/add

Commands and actual results

make format
All checks passed!
668 files left unchanged

poetry run pytest tests/memories/ tests/mem_reader/ tests/mem_scheduler/ tests/api/ -q -ra --tb=short
475 passed, 1 skipped, 8 warnings in 11.13s

The skipped scheduler configuration test documents that the current implementation accepts all components being None. Existing warnings cover Pydantic serialization, deprecated UTC timestamp calls and the intentional thread-exception fixture.

Regression coverage includes nested metadata independence across repeated searches, unrecoverable JSON versus legitimate empty results, both reader worker boundaries, failed async task reporting and preservation of fast memories.

Both follow-up commits passed the repository pre-commit hooks.

Checklist

  • Self-review completed
  • Comments explain metadata ownership and parse-failure behavior
  • Regression tests cover the changed behavior
  • Documentation scope reviewed for these internal fixes
  • Related issue and superseded PRs linked
  • Original contributors mentioned above for review context

Reviewer Checklist

yiwang-one and others added 5 commits November 19, 2026 17:47
The textual branch of _sort_and_trim rebuilt every selected item
(model_dump + two-level pydantic construction, including re-validating
the full embedding list and sources) only to set metadata.relativity to
the final score. When the metadata type already declares the relativity
field (vector/fulltext recall results), write the score in place with
the same float() coercion pydantic validation would have applied.
Items whose metadata lacks the field (e.g. graph/bm25 results) still
go through the rebuild for the type upgrade, preserving behavior.
…on malformed JSON (#2456)

parse_json_result silently returned {} on every parse failure, so a malformed LLM extraction was reported as an empty (but successful) /product/add response. The HTTP layer returned {"code": 200, "data": []} for an extraction that actually failed, hiding the wasted tokens and never alerting operators that a request had silently dropped.

This makes parse_json_result raise LLMResponseParseError (a new ParserError subclass in memos.exceptions) on any non-recoverable parse failure, so callers can distinguish "the model produced no memories" from "the model produced output we couldn't parse". Every in-tree caller already wraps parse_json_result in a try/except (multi_modal_struct, simple_struct, strategy_struct, file_content_parser, searcher, task_goal_parser), so the exception is caught and surfaces as the synthetic-memory-list fallback instead of the silent {} return path. The literal {} JSON value still returns {} so successful empty extractions are preserved.

The fix keeps test_simple_struct_fallback.py::test_process_chat_data_fine_yields_node_when_llm_unparseable passing: _safe_parse catches the new exception and returns None, which still triggers the chat fallback that emits a salvageable UserMemory item.

- New exception LLMResponseParseError(ParserError) in memos.exceptions
- parse_json_result now raises on empty input, missing JSON object, every unrecoverable JSONDecodeError, while still returning the dict when one is found (including a literal {})
- New tests/mem_reader/test_parse_json_result_strict.py covers the trailing-comma reproducer from the issue, the prose-only 'I cannot output JSON' symptom, empty/None input, no JSON object, and the exception-chain contract (the chained cause preserves the underlying JSONDecodeError for debugging)
- Updated test_simple_structure.py::test_parse_json_result_failure and the docstring on test_simple_struct_fallback.py to match the new contract
The test module injected stub packages for memos/memos.log/memos.exceptions/
memos.mem_reader at import time. Those stubs stayed in sys.modules for the
whole pytest session and broke every later test that imported the real
package: memos.log lost _LOGGING_CONFIGURED_PID/summarize_* helpers, the
stub root package shadowed real submodules (memos.chunkers, memos.mem_user),
and server_router tests saw 'module memos has no attribute log'. The stub
loader also resolved src/... paths against the current working directory,
so collecting the file from any other directory raised FileNotFoundError
(autotest 'collection' failures).

Replace the stub machinery with normal top-level imports — pyproject.toml
already sets pythonpath=src and sibling tests import the same modules the
same way. Also assert the exception chain directly via
exc_info.value.__cause__ being a json.JSONDecodeError, which the current
or-clause ('Failed' in message) could never fail.
@Memtensor-AI Memtensor-AI added area:api 云服务 / FastAPI / OpenAPI / MCP area:memory 记忆存储、检索、更新、召回逻辑 area:scheduler 调度模块 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Nov 19, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2633
Task: a5868555fc237a4e
Base: dev-v2.0.37
Head: fix/retrieval-isolation-extraction-errors
Head SHA: da9fdfb8e2a0e328b6a9eff02b8c4b4985f70f26

🔍 OpenCodeReview found 1 issue(s) in this PR.

⚠️ Review warnings (3)

  • bounded analysis finalized with explicit task_done after reviewing all group files
  • bounded analysis finalized with explicit task_done after reviewing all group files
  • bounded analysis finalized with explicit task_done after reviewing all group files

1. src/memos/mem_reader/utils.py (L92)

The previous warning logged the tail of the raw LLM payload (Tail: Raw {response_text} json: {s}), which is what made these failures diagnosable in production. Now that the raw text is no longer visible anywhere (the raised LLMResponseParseError only carries a generic message), a malformed extraction can't be debugged from logs alone. Consider keeping a truncated snippet of the payload in the warning (and/or in the exception message).

💡 Suggested Change

Before:

    logger.warning("[JSONParse] Failed to decode JSON at position %s", last_err.pos)

After:

    logger.warning(
        "[JSONParse] Failed to decode JSON at position %s; payload head: %r",
        last_err.pos,
        (response_text or "")[:200],
    )

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Nov 19, 2026
@endxxxx
endxxxx merged commit e31e105 into dev-v2.0.37 Nov 19, 2026
18 checks passed
@endxxxx
endxxxx deleted the fix/retrieval-isolation-extraction-errors branch November 19, 2026 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:api 云服务 / FastAPI / OpenAPI / MCP area:memory 记忆存储、检索、更新、召回逻辑 area:scheduler 调度模块 status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants