Repository navigation
fix(retrieval): defer candidate construction until after ranking - #2613
Conversation
🤖 Open Code ReviewTarget: PR #2613 🔍 OpenCodeReview found 2 issue(s) in this PR. 1.
|
✅ Automated Test Results: PASSEDRequired tests passed (17/17 executed, 0 skipped). Duration: 17s Branch: Tested source: |
Recall previously constructed a fully validated TextualMemoryItem for every recalled candidate, including the ones later discarded by ranking and trimming. Vector and fulltext recall now wrap candidate node dicts in a validation-free view (DeferredTextualMemoryItem), and real items are built only for survivors, via the same from_dict validation as before. - The view exposes passthrough attributes (id, memory, metadata field lookups), so merge / rerank / dedup decisions are unchanged. - _sort_and_trim materializes survivors in every branch loop; the full_recall exit materializes everything for callers that treat results as fully constructed items. - retrieve_from_cube materializes eagerly: it mutates metadata in place and must keep returning real items.
1dbd799 to
1ee1727
Compare
Description
What & Why
Vector and fulltext recall constructed a fully validated
TextualMemoryItem(pydantic validation of the full metadata, includingthe embedding list and sources) for every recalled candidate — but
ranking and trimming then keep only the top-k. Validation for discarded
candidates is pure overhead on the retrieval path.
Approach
validation-free view (
DeferredTextualMemoryItem) exposing thepassthrough attributes consumed before trimming (
id,memory,metadatafield lookups). Merge / rerank / dedup decisions read thesame values as before, unchanged.
_sort_and_trim(all memory-typebranches) via the same
from_dictvalidation as today — only forthe top-k instead of for every candidate.
full_recall=Trueexit materializes all results, since itscallers treat them as fully constructed items.
retrieve_from_cubematerializes eagerly: it mutatesmetadatain place and must keep returning real items.
Note on failure semantics: a malformed candidate payload previously
failed at recall time and took down the whole branch; it now fails at
materialization time (after trimming), and only if it survives ranking.
Valid data is unaffected; returned items pass through identical
validation either way.
Composes with #2600 (in-place relativity write): both touch the textual
loop of
_sort_and_trim; resolution keeps both lines (materialize first,then the in-place write).
No new dependencies.
中文说明
向量/全文召回此前对每条候选都做全量 pydantic 校验构造
TextualMemoryItem,而排序截断后仅保留 top-k,被淘汰候选的校验构造属于纯开销。本 PR 改为:召回阶段返回零校验的轻量视图(直通
id/memory/metadata 字段,排序与去重决策读取的值与原先完全一致),
仅对截断后的幸存者通过与原先相同的
from_dict校验构造正式对象;full_recall出口全量物化以保护既有调用方;retrieve_from_cube因需就地改写 metadata 保持立即构造。输出与原实现逐字段等价,
无新增依赖。与 #2600(就地写 relativity)组合时保留两行即可。
Equivalence: unit tests compare view-backed and item-backed inputs
through rerank / dedup / sort-and-trim and deep-compare the resulting
to_dict()outputs;build()is asserted equivalent to eagerfrom_dict()on a battery of payload shapes.Related Issue (Required): N/A — performance fix with no functional change
Type of change
How Has This Been Tested?
11 new tests in
tests/memories/textual/test_deferred_candidate.py(view passthrough incl. missing-metadata edge, build equivalence,
rerank/dedup/sort-trim equivalence with mixed views and items, tool-branch
materialization, plugin zero-score skips construction, deepcopy safety).
One existing test updated to the new
_vector_recallcontract.Reproduce (repo root):
Checklist
Reviewer Checklist
Targeting
mainfollowing the post-v2.0.36 release flow; happy to retarget ifa dedicated dev branch is introduced.