Skip to content

fix(retrieval): optimize result sorting by removing redundant rebuilds - #2600

Closed
syzsunshine219 wants to merge 1 commit into
MemTensor:mainfrom
yiwang-one:fix/sort-trim-inplace-relativity
Closed

syzsunshine219 wants to merge 1 commit into
MemTensor:mainfrom
yiwang-one:fix/sort-trim-inplace-relativity

Conversation

@syzsunshine219

@syzsunshine219 syzsunshine219 commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Restores the change proposed in #2563 for review against main after its former base branch, dev-v2.0.36, was deleted. The source commits and authors are preserved.

Source head: 1a517115561e91cc7cb214e38afe899b3b7fc1d0

This PR is open for review. Review and CI against the current base are pending; validation claims in the original description below refer to the earlier snapshot. The original discussion and evidence remain in #2563.


Original change description

Description

What & Why

The textual branch of _sort_and_trim rebuilt every selected memory item only to
stamp the final score into metadata.relativity. The rebuild path ran
model_dump() plus two levels of pydantic construction, re-validating the full
metadata of each top-k result — including the embedding list and sources — just to
set one float field. This made the object rebuild the dominant per-request cost of
the sort-and-trim stage.

Approach

  • When the item's metadata is already SearchedTreeNodeTextualMemoryMetadata
    (the case for vector/fulltext recall results, which declare relativity), the
    score is now written in place with an explicit float() cast — the same
    coercion pydantic validation used to apply on reconstruction.
  • Items whose metadata type does not declare the field (e.g. graph/bm25 results)
    still go through the original dump-and-rebuild path for the type upgrade, so
    their behavior is unchanged.
  • The ownership contract is documented in the _sort_and_trim docstring: the
    method takes ownership of results and overwrites relativity in place on
    selected items. No caller in the current search flow holds the candidate list
    after this point (the deep_search flow copies before its second pass).

No new dependencies.

Output equivalence: the serialized output (to_dict()) is compared against a
reference implementation of the previous rebuild path in unit tests, and is
field-by-field identical, so retrieval results and scores are unchanged.

中文说明

_sort_and_trim 的 textual 分支此前为了给入选记忆写入最终相关性分数
metadata.relativity,对每个 top-k 结果做全量 dump + 两层 pydantic 重建(含
embedding 与 sources 的重复校验),是排序截断阶段每请求的主要开销。本 PR 改为:
metadata 类型已声明 relativity 字段时(向量/全文召回结果)就地写入并显式
float() 强转(与原校验行为一致);未声明该字段的类型(如 graph/bm25 来源)
仍走原重建路径完成类型升级,行为不变。输出经单测与旧实现逐字段比对完全一致,
检索结果与分数不受影响,无新增依赖。

Related Issue (Required): Fixes #issue_number

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • Unit Test
  • Test Script Or Steps (please provide)

7 new unit tests in tests/memories/textual/test_tree_searcher.py covering:
hot-path in-place equivalence vs the previous rebuild (full to_dict() deep
comparison), cold-path type upgrade, mixed metadata types, top_k-trimmed items
left untouched, plugin zero-score filtering, idempotent double invocation, and
numpy scalar score coercion.

Reproduce (repo root):

PYTHONPATH=src python -m pytest tests/memories/textual/test_tree_searcher.py -v
# => 12 passed

PYTHONPATH=src python -m pytest tests/memories/textual/ -q
# => 94 passed

PYTHONPATH=src python -m pytest tests/api/ -q
# => 147 passed

PYTHONPATH=src python -m pytest tests/dream/test_context_recall.py -q
# => 4 passed

ruff format --check src/memos/memories/textual/tree_text_memory/retrieve/searcher.py tests/memories/textual/test_tree_searcher.py
# => 2 files already formatted

Checklist

  • I have performed a self-review of my own code | 我已自行检查了自己的代码
  • I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释
  • I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常
  • I have created related documentation issue/PR in MemOS-Docs (if applicable) | 我已在 MemOS-Docs 中创建了相关的文档 issue/PR(如果适用)— N/A: internal performance fix, no user-visible change
  • I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR(如果适用)
  • I have mentioned the person who will review this PR | 我已提及将审查此 PR 的人

Reviewer Checklist

  • closes #issue_number
  • Made sure Checks passed
  • Tests have been provided

@Memtensor-AI Memtensor-AI added area:memory 记忆存储、检索、更新、召回逻辑 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Nov 9, 2026
@syzsunshine219
syzsunshine219 marked this pull request as ready for review November 9, 2026 15:48
@yiwang-one
yiwang-one force-pushed the fix/sort-trim-inplace-relativity branch 2 times, most recently from 93f1d14 to 2d4fda9 Compare November 10, 2026 07:45
@Memtensor-AI

Memtensor-AI commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2600
Task: aed0d6ed43b7ada5
Base: main
Head: fix/sort-trim-inplace-relativity
Head SHA: 306711234de8ae11044d1c9500b6f6482e9346b6

✅ OpenCodeReview: Review complete: 0 finding(s) across 1 selected item(s).

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

Required tests passed (12/12 executed, 0 skipped). Duration: 9s [advisory, non-gating] AI-generated tests on branch test/auto-gen-1d237a30405886d6-20261110155837: 54/58 passed, 4 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: fix/sort-trim-inplace-relativity

Tested source: 2d4fda908fb31beafdc469fea29e083406ced24b
Tested base: 3a16e5fe3048a209f722806f4123fa2e533588fd
Tested commit: 2d4fda908fb31beafdc469fea29e083406ced24b
These results apply only to this immutable source/base snapshot.

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Nov 10, 2026
The textual branch of _sort_and_trim rebuilt every selected item
(model_dump + two-level pydantic construction, including re-validating
the full embedding list and sources) only to set metadata.relativity to
the final score. When the metadata type already declares the relativity
field (vector/fulltext recall results), write the score in place with
the same float() coercion pydantic validation would have applied.
Items whose metadata lacks the field (e.g. graph/bm25 results) still
go through the rebuild for the type upgrade, preserving behavior.
@yiwang-one
yiwang-one force-pushed the fix/sort-trim-inplace-relativity branch from 2d4fda9 to 3067112 Compare November 14, 2026 07:44
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Nov 14, 2026
endxxxx pushed a commit that referenced this pull request Nov 18, 2026
## Description

### What & Why

Vector and fulltext recall constructed a fully validated
`TextualMemoryItem` (pydantic validation of the full metadata, including
the embedding list and sources) for **every recalled candidate** — but
ranking and trimming then keep only the top-k. Validation for discarded
candidates is pure overhead on the retrieval path.

### Approach

- Recall branches now wrap candidate node dicts in a lightweight,
  validation-free view (`DeferredTextualMemoryItem`) exposing the
  passthrough attributes consumed before trimming (`id`, `memory`,
  `metadata` field lookups). Merge / rerank / dedup decisions read the
  same values as before, unchanged.
- Survivors are materialized in `_sort_and_trim` (all memory-type
  branches) via the **same `from_dict` validation as today** — only for
  the top-k instead of for every candidate.
- The `full_recall=True` exit materializes all results, since its
  callers treat them as fully constructed items.
- `retrieve_from_cube` materializes eagerly: it mutates `metadata`
  in place and must keep returning real items.

Note on failure semantics: a malformed candidate payload previously
failed at recall time and took down the whole branch; it now fails at
materialization time (after trimming), and only if it survives ranking.
Valid data is unaffected; returned items pass through identical
validation either way.

Composes with #2600 (in-place relativity write): both touch the textual
loop of `_sort_and_trim`; resolution keeps both lines (materialize
first,
then the in-place write).

No new dependencies.

### 中文说明

向量/全文召回此前对每条候选都做全量 pydantic 校验构造
`TextualMemoryItem`,而排序截断后仅保留 top-k,被淘汰候选的校验
构造属于纯开销。本 PR 改为:召回阶段返回零校验的轻量视图(直通
id/memory/metadata 字段,排序与去重决策读取的值与原先完全一致),
仅对截断后的幸存者通过与原先相同的 `from_dict` 校验构造正式对象;
`full_recall` 出口全量物化以保护既有调用方;`retrieve_from_cube`
因需就地改写 metadata 保持立即构造。输出与原实现逐字段等价,
无新增依赖。与 #2600(就地写 relativity)组合时保留两行即可。

**Equivalence**: unit tests compare view-backed and item-backed inputs
through rerank / dedup / sort-and-trim and deep-compare the resulting
`to_dict()` outputs; `build()` is asserted equivalent to eager
`from_dict()` on a battery of payload shapes.

Related Issue (Required): N/A — performance fix with no functional
change

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

- [x] Unit Test
- [x] Test Script Or Steps (please provide)

11 new tests in `tests/memories/textual/test_deferred_candidate.py`
(view passthrough incl. missing-metadata edge, build equivalence,
rerank/dedup/sort-trim equivalence with mixed views and items,
tool-branch
materialization, plugin zero-score skips construction, deepcopy safety).
One existing test updated to the new `_vector_recall` contract.

Reproduce (repo root):

```bash
PYTHONPATH=src python -m pytest tests/memories/textual/test_deferred_candidate.py -q
# => 11 passed

PYTHONPATH=src python -m pytest tests/memories/textual/ tests/api/ -q
# => 245 passed

ruff format --check src/memos/memories/textual/item.py src/memos/memories/textual/tree_text_memory/retrieve/recall.py src/memos/memories/textual/tree_text_memory/retrieve/searcher.py
# => already formatted
```

## Checklist

- [x] I have performed a self-review of my own code | 我已自行检查了自己的代码
- [x] I have commented my code in hard-to-understand areas |
我已在难以理解的地方对代码进行了注释
- [x] I have added tests that prove my fix is effective or that my
feature works | 我已添加测试以证明我的修复有效或功能正常
- [x] MemOS-Docs not applicable — internal performance fix, no public
API change | 不适用——内部性能修复,无公共 API 变化
- [ ] I have linked the issue to this PR (if applicable) | N/A
- [ ] I have mentioned the person who will review this PR | 我已提及将审查此 PR
的人

## Reviewer Checklist
- [ ] Made sure Checks passed
- [ ] Tests have been provided

---
Targeting `main` following the post-v2.0.36 release flow; happy to
retarget if
a dedicated dev branch is introduced.
@endxxxx

endxxxx commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

Superseded by #2633, targeting dev-v2.0.37. The replacement preserves your original commit and adds deep metadata isolation so nested tags, sources, embeddings and info cannot be shared between returned results and the original candidates.

The replacement's memory, reader, scheduler and API tests completed with 475 passed and 1 existing skip. Closing this PR to consolidate review in #2633. Thank you for the original contribution.

@endxxxx endxxxx closed this Nov 19, 2026
endxxxx added a commit that referenced this pull request Nov 19, 2026
)

## Description

Ranked search results could share nested metadata with the original
candidates, and malformed LLM extraction responses could be converted
into successful empty writes by reader fallback and worker exception
handlers.

Selected searched metadata is now copied deeply without Pydantic
revalidation. Parsing failures propagate through simple and multimodal
extraction, document workers and public reader boundaries. Synchronous
/product/add returns HTTP 500 before storage or scheduling on a parse
failure, while valid {} and {"memory list": []} remain successful empty
results. Async fine-extraction failures retain the existing fast
memories and emit a failed task event.

This PR supersedes #2600 and #2607 and targets dev-v2.0.37. Original
contributions from @yiwang-one and @RerankerGuo are preserved in
separate cherry-picked commits, followed by the metadata-isolation and
write-path fixes. Maintainer edits remained disabled on the original
PRs.

Related issue #2456.

## Type of change

- [x] Bug fix

## How Has This Been Tested?

- [x] Unit tests and regression tests, using synthetic data and mocked
model/storage boundaries
- [x] Test steps completed with Python 3.12
- [x] Automated API tests exercise the real AddHandler, SingleCubeView
and SimpleStruct reader through POST /product/add

Commands and actual results

    make format
    All checks passed!
    668 files left unchanged

poetry run pytest tests/memories/ tests/mem_reader/ tests/mem_scheduler/
tests/api/ -q -ra --tb=short
    475 passed, 1 skipped, 8 warnings in 11.13s

The skipped scheduler configuration test documents that the current
implementation accepts all components being None. Existing warnings
cover Pydantic serialization, deprecated UTC timestamp calls and the
intentional thread-exception fixture.

Regression coverage includes nested metadata independence across
repeated searches, unrecoverable JSON versus legitimate empty results,
both reader worker boundaries, failed async task reporting and
preservation of fast memories.

Both follow-up commits passed the repository pre-commit hooks.

## Checklist

- [x] Self-review completed
- [x] Comments explain metadata ownership and parse-failure behavior
- [x] Regression tests cover the changed behavior
- [x] Documentation scope reviewed for these internal fixes
- [x] Related issue and superseded PRs linked
- [x] Original contributors mentioned above for review context

## Reviewer Checklist

- [x] Issue #2456 and superseded PRs linked
- [x] Local format, pre-commit and relevant pytest checks passed
- [x] Tests provided
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:memory 记忆存储、检索、更新、召回逻辑 status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants