You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
PostgresGraphDB._parse_row() and _prepare_node_metadata() assumed metadata["embedding"] is already a list[float]. When include_embedding=True, psycopg2/pgvector often returns the vector column as a JSON string (for example '[-0.047..., ...]').
The reorganizer loads nodes with get_node(..., include_embedding=True) and constructs GraphDBNode(**raw_node). Pydantic then rejects the string embedding, so the reorganizer consumer logs a traceback and skips reorganize work.
This change adds _normalize_embedding_value() to coerce list/tuple/JSON-string embeddings into list[float], and uses it in both metadata preparation and row parsing.
When include_embedding=True but len(row) <= 5 (the guard on line 462 fails), row[5] was never assigned to result["metadata"]["embedding"]. However, the block below still runs and its elif branch will silently delete any "embedding" key that was already present in the props JSON blob — data that wasn't fetched from the embedding column at all.
Consider restructuring so the normalization only applies when the embedding was actually fetched:
This removes the separate assignment on line 463 and the separate block on lines 464–469, avoiding the accidental deletion.
💡 Suggested Change
Before:
if include_embedding and len(row) > 5:
result["metadata"]["embedding"] = row[5]
if include_embedding:
normalized = _normalize_embedding_value(result["metadata"].get("embedding"))
if normalized is not None:
result["metadata"]["embedding"] = normalized
elif "embedding" in result["metadata"]:
del result["metadata"]["embedding"]
After:
if include_embedding and len(row) > 5:
normalized = _normalize_embedding_value(row[5])
if normalized is not None:
result["metadata"]["embedding"] = normalized
else:
result["metadata"].pop("embedding", None)
2. src/memos/graph_dbs/postgres.py (L33-L38)
If embedding is a type not handled by any branch (e.g., a numpy.ndarray returned by some pgvector adapters, a memoryview, or a Decimal), all three isinstance checks fail, the try exits without returning, and None is returned at line 51. In _parse_row, this causes the elif branch to silently delete the embedding from the result with no log or warning, making it very hard to diagnose why embeddings disappear for certain driver versions.
Consider adding a fallback that attempts conversion via list() for iterable types, or at minimum logs a warning before returning None:
The test environment encountered an issue that requires manual attention.
Details: Executor error: Command failed: git clone --depth 1 --branch fix/postgres-embedding-normalize git@github.com:Kwizii/MemOS.git /data/test-workspaces/8d1d826a3e671a05/repo
Cloning into '/data/test-workspaces/8d1d826a3e671a05/repo'...
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists. Branch:fix/postgres-embedding-normalize
The test environment encountered an issue that requires manual attention.
Details: Executor error: Command failed: git clone --depth 1 --branch fix/postgres-embedding-normalize git@github.com:Kwizii/MemOS.git /data/test-workspaces/455b594769bdf0dd/repo
Cloning into '/data/test-workspaces/455b594769bdf0dd/repo'...
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists. Branch:fix/postgres-embedding-normalize
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
PostgresGraphDB._parse_row()and_prepare_node_metadata()assumedmetadata["embedding"]is already alist[float]. Wheninclude_embedding=True, psycopg2/pgvector often returns the vector column as a JSON string (for example'[-0.047..., ...]').The reorganizer loads nodes with
get_node(..., include_embedding=True)and constructsGraphDBNode(**raw_node). Pydantic then rejects the string embedding, so the reorganizer consumer logs a traceback and skips reorganize work.This change adds
_normalize_embedding_value()to coerce list/tuple/JSON-string embeddings intolist[float], and uses it in both metadata preparation and row parsing.Related Issue (Required): Fixes #2270
Type of change
How Has This Been Tested?
Added regression tests in
tests/graph_dbs/test_postgres_embedding_parse.py:_normalize_embedding_value()parses JSON string embeddings_prepare_node_metadata()normalizes string embeddings in metadata_parse_row(..., include_embedding=True)normalizes pgvector string column valuesReproduce with:
Checklist