Skip to content

Fix #2103: fix: Search_memories在默认dedup=mmr下调试用 info 日志打印完整 embedding 向量 - #2104

Closed
Memtensor-AI wants to merge 3 commits into
MemTensor:dev-v2.0.24from
Memtensor-AI:bugfix/autodev-2103-20260714060726960
Closed

Memtensor-AI wants to merge 3 commits into
MemTensor:dev-v2.0.24from
Memtensor-AI:bugfix/autodev-2103-20260714060726960

Conversation

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

Description

Fixed issue #2103: SingleCubeView.search_memories was writing the full assembled memories_result — including high-dimensional per-memory metadata.embedding vectors — to the INFO log on every default /product/search call (because APISearchRequest defaults mode=fast, dedup=mmr, which causes format_memory_item(..., include_embedding=True) in _fast_search). Each request therefore serialized MB of float data to logs, drowned useful signal, and leaked vector content — a privacy / compliance risk.

The fix is logging-only, with no change to search_memories' return payload or any API contract. A new module-level helper _summarize_search_result_for_log in src/memos/multi_mem_cube/single_cube.py returns a compact, embedding-free summary: per-bucket counts across text_mem / pref_mem / tool_mem / skill_mem, a bounded (max 5 per bucket) sample of memory ids / ref_ids / memory_types / relativity scores, and — for each sample — a boolean has_embedding plus an integer embedding_len. The two offending log statements at single_cube.py:132-133 are collapsed into a single sanitized INFO log line; the old len(memories_result) (which returned ~7, the number of top-level dict keys, not the number of matched memories) is replaced by the summary's total_memories field.

Regression coverage is added at tests/multi_mem_cube/test_search_log_sanitization.py (11 tests, all passing): it exercises the helper directly (embedding leak, bucket counts, bounded sampling, malformed input) and drives SingleCubeView.search_memories end-to-end for both dedup="mmr" (embeddings populated) and dedup="no" branches, asserting that distinctive embedding-vector floats never appear in any captured log record while the summary line still emits the useful debug signals. ruff check and ruff format --check on the changed files are clean; pre-existing failures in tests/test_add_stage_logging.py and tests/api/test_mcp_serve.py were confirmed unrelated by re-running against a stash of the branch tip.

Related Issue (Required): Fixes #2103

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactor (does not change functionality, e.g. code style improvements, linting)
  • Documentation update

How Has This Been Tested?

Not run; documentation-only change.

  • Unit Test
  • Test Script Or Test Steps (please provide)
  • Pipeline Automated API Test (please provide)

Checklist

  • I have performed a self-review of my own code
  • I have commented my code in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works
  • I have created related documentation issue/PR in MemOS-Docs (if applicable)
  • I have linked the issue to this PR (if applicable)
  • I have mentioned the person who will review this PR

@MatthewZhuang, @CarltonXiang, @syzsunshine219, @World-controller please review this PR.

Reviewer Checklist

…ing vectors

Under the default `APISearchRequest` (`mode=fast`, `dedup=mmr`),
`_fast_search` calls `format_memory_item(..., include_embedding=True)`, so
every memory keeps its full `metadata.embedding` list. The debug line
`self.logger.info(f"Search memories result: {memories_result}")` in
`SingleCubeView.search_memories` therefore serialized MB of float vectors
into logs on every default /product/search — drowning useful signal and
leaking vector content that can be used to approximate semantic contents.

Fix (logging-only, no API contract change):

* Add `_summarize_search_result_for_log` helper producing a compact,
  embedding-free summary: per-bucket counts across text_mem / pref_mem /
  tool_mem / skill_mem, a bounded (max 5) sample of memory ids /
  ref_ids / types / relativity scores, and a boolean `has_embedding`
  plus `embedding_len` — never the raw vector.
* Replace the two offending log lines with a single INFO log line
  emitting that summary. The old `len(memories_result)` also reported
  the number of top-level dict keys (~7 constants) instead of the
  number of matched memories; the new summary carries an accurate
  `total_memories` field instead.
* Regression suite `tests/multi_mem_cube/test_search_log_sanitization.py`
  covers the helper directly (embedding leak / bucket counts / bounded
  sampling / malformed input) and drives `search_memories` end-to-end
  under both `dedup=mmr` (embeddings populated) and `dedup=no`
  branches, asserting the log line never contains embedding floats.

Fixes MemTensor#2103

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Memtensor-AI Memtensor-AI added ai:generated Generated or modified by AI | 由 AI 生成或修改 area:memcube GeneralMemCube / cube 生命周期 / cube 配置 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Jul 14, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator Author

🤖 Open Code Review

Target: PR #2104
Task: 84d26ab136ba86f0
Base: dev-v2.0.24
Head: bugfix/autodev-2103-20260714060726960
Head SHA: e5fa991c87f004a33c51c1c74be79fa5cacf342a

🔍 OpenCodeReview found 2 issue(s) in this PR.

⚠️ 1 warning(s) occurred during review.


1. tests/multi_mem_cube/test_search_log_sanitization.py (L113)

This assertion is a tautology and tests nothing meaningful. The sample dict built by _summarize_search_result_for_log never contains a key named "embedding" — it only contains "has_embedding" and "embedding_len". As a result, "embedding" not in sample is always True, and the right-hand side sample["embedding_len"] is not None is never evaluated.

If the implementation were ever changed to accidentally include a raw "embedding" key with a non-None value in the sample dict, this assertion would still pass (vacuously), providing no protection.

Consider replacing with a direct structural check that asserts the raw "embedding" key is absent, e.g.:

assert "embedding" not in sample, (
    f"Raw embedding key must never appear in log sample: {list(sample.keys())}"
)

This makes the intent explicit and would catch a regression where the implementation accidentally leaks an "embedding" key.

💡 Suggested Change

Before:

                assert "embedding" not in sample or sample["embedding_len"] is not None

After:

                assert "embedding" not in sample, (
                    f"Raw embedding key must never appear in log sample: {list(sample.keys())}"
                )

2. src/memos/multi_mem_cube/single_cube.py (L94-L97)

Bug: bucket_count and total_memories undercount when the sample limit is hit mid-way through groups.

The outer break at line 94–95 exits the for group in groups loop as soon as _LOG_SAMPLE_PER_BUCKET samples are collected. Because bucket_count += len(mems) (line 74) only runs for groups that have been visited before the break, any remaining groups are silently skipped. In a multi-cube deployment where a single bucket (e.g. text_mem) can have one group per cube, total_memories in the log would undercount the real number of memories, potentially misleading operators who monitor memory sizes.

Fix: accumulate counts over all groups unconditionally, and only guard the sampling loop:

for group in groups:
    if not isinstance(group, dict):
        continue
    mems = group.get("memories") or []
    bucket_count += len(mems)  # always count

    if len(samples) < _LOG_SAMPLE_PER_BUCKET:
        for mem in mems:
            if len(samples) >= _LOG_SAMPLE_PER_BUCKET:
                break
            if not isinstance(mem, dict):
                continue
            # … build and append sample …

This separates counting (full coverage) from sampling (bounded), so count in the summary is always accurate.

💡 Suggested Change

Before:

            if len(samples) >= _LOG_SAMPLE_PER_BUCKET:
                break

        summary[bucket] = {"count": bucket_count, "samples": samples}

After:

        for group in groups:
            if not isinstance(group, dict):
                continue
            mems = group.get("memories") or []
            bucket_count += len(mems)  # always count, regardless of sample limit

            if len(samples) < _LOG_SAMPLE_PER_BUCKET:
                for mem in mems:
                    if len(samples) >= _LOG_SAMPLE_PER_BUCKET:
                        break
                    if not isinstance(mem, dict):
                        continue
                    metadata = mem.get("metadata") or {}
                    embedding = metadata.get("embedding")
                    embedding_len = len(embedding) if isinstance(embedding, (list, tuple)) else 0
                    samples.append(
                        {
                            "id": mem.get("id"),
                            "ref_id": mem.get("ref_id"),
                            "memory_type": metadata.get("memory_type"),
                            "relativity": metadata.get("relativity"),
                            "has_embedding": embedding_len > 0,
                            "embedding_len": embedding_len,
                        }
                    )

        summary[bucket] = {"count": bucket_count, "samples": samples}

🧹 Filtered 2 low-confidence OCR finding(s) before posting/fix-loop (duplicate: 2).

Generated by cloud-assistant via Open Code Review.

…ict test assertion

Round 1/2 of automated review fixes for PR MemTensor#2104 (issue MemTensor#2103).

Two findings from the review:

1. src/memos/multi_mem_cube/single_cube.py L94-97 — bucket_count undercount.
   The outer `break` on the `for group in groups` loop exited the moment
   `_LOG_SAMPLE_PER_BUCKET` samples were collected, so any remaining groups
   were silently skipped by `bucket_count += len(mems)`. In a multi-cube
   deployment where a single bucket (e.g. text_mem) can have one group per
   cube, `total_memories` in the log undercounted the real number of
   memories. Fix: always accumulate `bucket_count += len(mems)` for every
   group, guard only the sampling loop with `len(samples) <
   _LOG_SAMPLE_PER_BUCKET`. Sampling stays bounded, counting is now
   complete.

2. tests/multi_mem_cube/test_search_log_sanitization.py L113 —
   tautological assertion. `"embedding" not in sample or
   sample["embedding_len"] is not None` was vacuously true because the
   sample dict never has an "embedding" key (only "has_embedding" and
   "embedding_len"). If a future change accidentally added a raw
   "embedding" key with a real value, the check would still pass. Replaced
   with a direct structural check that asserts the raw "embedding" key
   is absent, with a descriptive message listing sample keys.

Also added regression test
`test_count_is_accurate_across_groups_after_sample_cap` that pins the
multi-group counting behaviour (two cubes x 10 memories per group;
sampling caps at 5 in the first group; count must still be 20).

Verification (all 12 tests in tests/multi_mem_cube/test_search_log_sanitization.py pass):
  $ python3 -m pytest tests/multi_mem_cube/test_search_log_sanitization.py -v
  ...
  ======================== 12 passed, 2 warnings in 4.17s ========================

Ruff: All checks passed!

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Memtensor-AI

Copy link
Copy Markdown
Collaborator Author

🤖 Open Code Review

Target: PR #2104
Task: 84d26ab136ba86f0
Base: dev-v2.0.24
Head: bugfix/autodev-2103-20260714060726960
Head SHA: 8a318170e09e1fd6e3a274379b9a01c1b0ee524e

🔍 OpenCodeReview found 2 issue(s) in this PR.

⚠️ 1 warning(s) occurred during review.


1. src/memos/multi_mem_cube/single_cube.py (L101-L103)

When pref_note is a non-string truthy value (e.g., a list or dict), isinstance(pref_note, str) is False, so pref_note_len is silently logged as 0. This hides an unexpected type in production without any warning — the operator sees pref_note_len: 0 rather than realising the value was of the wrong type.

Consider using len(pref_note) unconditionally (since all standard container types support it), or log the actual type when the isinstance check fails so the anomaly is visible:

if pref_note:
    summary["pref_note_len"] = len(pref_note) if isinstance(pref_note, str) else f"non-str({type(pref_note).__name__})"

Alternatively, just len(pref_note) works for both strings and sequences.

💡 Suggested Change

Before:

    pref_note = memories_result.get("pref_note")
    if pref_note:
        summary["pref_note_len"] = len(pref_note) if isinstance(pref_note, str) else 0

After:

    pref_note = memories_result.get("pref_note")
    if pref_note:
        summary["pref_note_len"] = (
            len(pref_note) if isinstance(pref_note, str) else f"non-str({type(pref_note).__name__})"
        )

2. tests/multi_mem_cube/test_search_log_sanitization.py (L338)

The third condition "'count': 2" in joined is an exact duplicate of the first condition — it is dead code and will never be evaluated independently. This appears to be a copy-paste mistake. Consider replacing it with a different format variant you actually intend to cover (e.g. '"count": 2' in joined for JSON-style keys), or simply remove the redundant third branch.

💡 Suggested Change

Before:

        assert "'count': 2" in joined or "count=2" in joined or "'count': 2" in joined

After:

        assert "'count': 2" in joined or "count=2" in joined or '"count": 2' in joined

🧹 Filtered 8 low-confidence OCR finding(s) before posting/fix-loop (duplicate: 8).

Generated by cloud-assistant via Open Code Review.

…f_note + drop duplicate test condition

Round 2/2 of automated review fixes for PR MemTensor#2104 (issue MemTensor#2103).

Two findings from the review:

1. src/memos/multi_mem_cube/single_cube.py L101-L103 — pref_note silent
   type coercion.
   When `pref_note` was a truthy non-string value (list, dict, etc.),
   `isinstance(pref_note, str)` returned False and `pref_note_len` was
   silently logged as 0. Operators saw `pref_note_len: 0` and could not
   distinguish "empty pref note" from "wrong type from upstream". Replaced
   the `else 0` fallback with a type-marker string
   `f"non-str({type(pref_note).__name__})"` so the anomaly is visible in
   logs. Behaviour when `pref_note` is a str (length) or falsy (key
   omitted) is unchanged.

2. tests/multi_mem_cube/test_search_log_sanitization.py L338 — duplicate
   condition in `or` chain.
   The third condition `"'count': 2" in joined` was an exact copy-paste of
   the first, dead code. Replaced with the intended JSON-key variant
   `'"count": 2' in joined` so all three branches now cover distinct
   rendering formats (Python repr / kwargs / JSON).

Also added regression test `test_pref_note_non_string_surfaces_type`
pinning the four cases: str (numeric length), list / dict (type marker),
falsy (key omitted).

Verification (13 tests, all pass):
  $ python3 -m pytest tests/multi_mem_cube/test_search_log_sanitization.py -v
  ...
  ======================== 13 passed, 2 warnings in 4.24s ========================

Ruff: All checks passed!

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Memtensor-AI

Copy link
Copy Markdown
Collaborator Author

🤖 Open Code Review

Target: PR #2104
Task: 84d26ab136ba86f0
Base: dev-v2.0.24
Head: bugfix/autodev-2103-20260714060726960
Head SHA: ac3eebc96ea51b247d55840280df88010505e3e1

🔍 OpenCodeReview found 4 issue(s) in this PR.


1. tests/multi_mem_cube/test_search_log_sanitization.py (L328-L331)

If search_memories raises an exception before reaching the log statement, caplog.records will be empty, and every subsequent assertion of the form assert "0.12345" not in joined passes vacuously — giving a false-green result. The test should assert that the call succeeded (no exception) AND that at least one log record was emitted before checking message content.

Suggested fix: capture the return value and add a guard assertion that caplog captured the expected summary line.

💡 Suggested Change

Before:

        with caplog.at_level(logging.INFO, logger="test.single_cube.search"):
            view.search_memories(req)

        return [rec.getMessage() for rec in caplog.records]

After:

        with caplog.at_level(logging.INFO, logger="test.single_cube.search"):
            result = view.search_memories(req)

        # Guard: ensure the method ran to completion and the summary was logged.
        assert result is not None, "search_memories raised before returning"
        messages = [rec.getMessage() for rec in caplog.records]
        assert any(
            "Search memories result summary" in m for m in messages
        ), f"No summary log emitted at all; records: {messages}"
        return messages

2. tests/multi_mem_cube/test_search_log_sanitization.py (L283-L308)

formatters_handler is imported and used only in the final assert sanity check, but the variable is otherwise dead — the test never monkeypatches or calls formatters_handler.post_process_textual_mem. The import and assertion do not affect test behavior and mislead readers into thinking formatters_handler integration is being exercised. Remove the import and the assertion, or replace them with a meaningful stub if integration coverage is desired.

💡 Suggested Change

Before:

    from memos.api.handlers import formatters_handler
    from memos.multi_mem_cube import single_cube as sc_module

    logger = logging.getLogger("test.single_cube.search")

    view = sc_module.SingleCubeView(
        cube_id="cube_test",
        naive_mem_cube=MagicMock(),
        mem_reader=MagicMock(),
        mem_scheduler=MagicMock(),
        logger=logger,
        searcher=MagicMock(),
        feedback_server=None,
    )

    # Stub _search_text to return already-formatted memory dicts drawn from
    # memories_result so post_process_textual_mem re-appends them.
    formatted = []
    for bucket in ("text_mem",):
        for group in memories_result.get(bucket) or []:
            formatted.extend(group.get("memories") or [])

    monkeypatch.setattr(view, "_search_text", lambda *a, **kw: formatted)

    # Guarantee post_process_textual_mem is the real one (it is, but be explicit).
    assert formatters_handler.post_process_textual_mem is not None

After:

    from memos.multi_mem_cube import single_cube as sc_module

    logger = logging.getLogger("test.single_cube.search")

    view = sc_module.SingleCubeView(
        cube_id="cube_test",
        naive_mem_cube=MagicMock(),
        mem_reader=MagicMock(),
        mem_scheduler=MagicMock(),
        logger=logger,
        searcher=MagicMock(),
        feedback_server=None,
    )

    # Stub _search_text to return already-formatted memory dicts drawn from
    # memories_result so post_process_textual_mem re-appends them.
    formatted = []
    for bucket in ("text_mem",):
        for group in memories_result.get(bucket) or []:
            formatted.extend(group.get("memories") or [])

    monkeypatch.setattr(view, "_search_text", lambda *a, **kw: formatted)

3. tests/multi_mem_cube/test_search_log_sanitization.py (L202-L203)

The test's regression scenario (sample cap causes early exit from the outer loop, undercounting bucket_count) is only exercised if _LOG_SAMPLE_PER_BUCKET < 10. If someone raises the constant to ≥ 10 in the future, both groups fit inside the cap, the sampling loop never terminates early, and the test still passes — but the undercounting regression would go undetected. Add an explicit guard assertion at the start of the test.

💡 Suggested Change

Before:

        group_a = [_make_memory_dict(f"a-{i}", embedding) for i in range(10)]
        group_b = [_make_memory_dict(f"b-{i}", embedding) for i in range(10)]

After:

        # The regression is only exercisable when the sample cap is less than a single group's size.
        assert _LOG_SAMPLE_PER_BUCKET < 10, (
            f"_LOG_SAMPLE_PER_BUCKET ({_LOG_SAMPLE_PER_BUCKET}) must be < 10 for this "
            "regression test to exercise the early-exit-from-sampling scenario."
        )
        group_a = [_make_memory_dict(f"a-{i}", embedding) for i in range(10)]
        group_b = [_make_memory_dict(f"b-{i}", embedding) for i in range(10)]

4. tests/multi_mem_cube/test_search_log_sanitization.py (L240-L247)

assert summary['total_memories'] >= 0 is vacuously true for any integer and does not verify correct behavior. With tool_mem containing [{"memories": [None, "junk"]}], the helper should count 2 items (both non-dict items are skipped during sampling but len(mems) still counts them). The assertion should pin the expected count to catch silent regressions.

💡 Suggested Change

Before:

        summary = _summarize_search_result_for_log(
            {
                "text_mem": None,
                "pref_mem": ["not-a-dict"],
                "tool_mem": [{"memories": [None, "junk"]}],
            }
        )
        assert summary["total_memories"] >= 0

After:

        summary = _summarize_search_result_for_log(
            {
                "text_mem": None,
                "pref_mem": ["not-a-dict"],
                "tool_mem": [{"memories": [None, "junk"]}],
            }
        )
        # tool_mem has 2 items (non-dict, skipped during sampling but counted);
        # text_mem is None → 0, pref_mem group is non-dict → skipped → 0.
        assert summary["total_memories"] == 2
        assert summary["tool_mem"]["count"] == 2
        assert summary["tool_mem"]["samples"] == []  # both items are non-dict, skipped

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator Author

⚠️ Open Code Review automatic OCR fix limit reached

Open Code Review still found 4 issue(s), but the automatic OCR fix loop has reached 2/2 attempts.

I stopped auto-fixing this PR to avoid an infinite loop. Please review the latest OCR comment and fix or dismiss the remaining findings manually.

PR: #2104

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:generated Generated or modified by AI | 由 AI 生成或修改 area:memcube GeneralMemCube / cube 生命周期 / cube 配置 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants