Repository navigation
fix(graph_db): optimize get_nodes embedding fetch (per-node to batch) - #2643
Open
syzsunshine219 wants to merge 1 commit into
Open
syzsunshine219 wants to merge 1 commit into
syzsunshine219 wants to merge 1 commit into
Conversation
The inherited Neo4jGraphDB.get_nodes loops self._parse_node, which the community subclass overrides to fetch each embedding with one vec_db.get_by_id round trip — N serial vector-DB round trips per retrieval, one per id. The batched counterpart _parse_nodes already exists and is used by get_all_memory_items, but get_nodes was never switched to it. Override get_nodes in the community subclass: same Cypher, same params, identical results, but all embeddings are fetched in a single get_by_ids call. Unit tests pin result equivalence, the Cypher contract, and the switch from per-node calls to a single batched call.
Collaborator
🤖 Open Code ReviewTarget: PR #2643 ✅ OpenCodeReview: Review complete: 0 finding(s) across 2 selected item(s). Generated by cloud-assistant via Open Code Review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The source branch for the change proposed in #2599 now lives in
MemTensor/MemOS, so repository writers can contribute fixes through the normal upstream branch permissions.The code and commit authors are unchanged. This PR retains target branch
mainand exact source heada4e6b25d53cfeca78adf7d3299bb1483472d3d2ffrom #2599.Previous discussion and review remain in #2599. Review and checks must be re-evaluated for this replacement PR; no previous approval or new test result is asserted by this migration.
Previous PR description and implementation details
Restores the change proposed in #2558 for review against
mainafter its former base branch,dev-v2.0.36, was deleted. The source commits and authors are preserved.Source head:
9b34163e11effc86d9d1fbc616bb5482e8134f6fThis PR is open for review. Review and CI against the current base are pending; validation claims in the original description below refer to the earlier snapshot. The original discussion and evidence remain in #2558.
Original change description
Description
Summary: Override
get_nodesinNeo4jCommunityGraphDBto fetch all nodeembeddings from the vector DB in a single batched call instead of one call per
node.
Problem:
Neo4jCommunityGraphDBinheritsget_nodesfrom the baseNeo4jGraphDB, which parses query results with a per-node loop(
self._parse_node). The community subclass overrides_parse_nodeto fetcheach node's embedding with an individual
vec_db.get_by_idround trip. As aresult, on the community backend a single
get_nodes(ids)call issues Nserial vector-DB requests (one per id) on top of the already-batched Neo4j
query — a classic N+1 pattern. This affects all vector-DB backends, since the
calls go through the
vec_dbabstraction.Implementation: The override keeps the Cypher and params byte-identical to
the base implementation (including
user_namefilter branches) and parsesresults via the existing
_parse_nodes, which fetches all embeddings in oneget_by_idscall._parse_nodesalready exists and is used byget_all_memory_items;get_nodessimply never adopted it. Results andordering are unchanged. For nodes without a stored vector, the batched path
sets
embedding=Noneinstead of omitting the key — consistent with theexisting
_parse_nodesbehavior (consumers read the field via.get()).Dependencies: None.
Related Issue (Required): N/A — self-contained performance fix; the problem,
approach and tests are fully described in this PR (no separate issue filed).
Type of change
How Has This Been Tested?
Test configuration: macOS, Python 3.13, ruff 0.11.8 (version pinned in CI).
The new tests cover:
comparison including ordering and embedding contents)
class (incl.
user_namefilter branches)get_by_idbefore, exactly 1 ×get_by_idsafteridsshort-circuit; missing-vector semanticsis ≥5x faster than the per-node path
Checklist
https://github.com/MemTensor/MemOS-Docs(if applicable) | 我已在https://github.com/MemTensor/MemOS-Docs中创建了相关的文档 issue/PR(如果适用)Reviewer Checklist