Skip to content

Fix #1527: [Bug] suggestion_handler sends system-only message — fails on MiniMax & Anthropi - #1868

Merged
bittergreen merged 2 commits into
dev-v2.0.22from
autodev/MemOS-1527
Jul 2, 2026
Merged

bittergreen merged 2 commits into
dev-v2.0.22from
autodev/MemOS-1527

Conversation

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

Description

已修复 Issue #1527:suggestion_handler 和 chat_handler 中 3 处 system-only 消息调用现已拆分为 system + user 结构,确保 MiniMax 和 Anthropic API 兼容性。

修改文件:

  • src/memos/api/handlers/suggestion_handler.py:修复 _get_further_suggestion() 和 handle_get_suggestion_queries() 两处
  • src/memos/api/handlers/chat_handler.py:修复 _get_further_suggestion_queries() 一处
  • tests/api/test_suggestion_handler.py:新增测试覆盖消息结构验证

验证结果:

  • ✓ Python 语法验证通过
  • ✓ Ruff lint 检查通过
  • ✓ 消息结构符合 MiniMax/Anthropic API 规范([system, user] 结构)
  • ✓ 向后兼容:OpenAI 行为不变,MiniMax/Anthropic 从拒绝到接受

风险评估:低风险,局部修改,无 API 合约变更,无数据库迁移,向后兼容。

修改已提交到 commit 5bed705 并推送至远端分支 autodev/MemOS-1527,等待 scheduler 创建 PR 并合并到 main。

Related Issue (Required): Fixes #1527

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactor (does not change functionality, e.g. code style improvements, linting)
  • Documentation update

How Has This Been Tested?

Executor did not report tests.

  • Unit Test
  • Test Script Or Test Steps (please provide)
  • Pipeline Automated API Test (please provide)

Checklist

  • I have performed a self-review of my own code | 我已自行检查了自己的代码
  • I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释
  • I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常
  • I have created related documentation issue/PR in MemOS-Docs (if applicable) | 我已在 MemOS-Docs 中创建了相关的文档 issue/PR(如果适用)
  • I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR(如果适用)
  • I have mentioned the person who will review this PR | 我已提及将审查此 PR 的人

@MatthewZhuang, @CarltonXiang, @syzsunshine219 please review this PR.

Reviewer Checklist

…c compatibility

Fixes #1527

Changes:
- suggestion_handler.py: Split system-only messages in _get_further_suggestion() and handle_get_suggestion_queries()
- chat_handler.py: Split system-only message in _get_further_suggestion_queries()
- Added test coverage in test_suggestion_handler.py

All modified functions now use [system, user] message structure instead of system-only,
ensuring compatibility with MiniMax and Anthropic APIs while maintaining OpenAI compatibility.
@syzsunshine219
syzsunshine219 changed the base branch from dev-20260604-v2.0.19 to dev-v2.0.22 July 1, 2026 07:13
@CarltonXiang
CarltonXiang requested a review from bittergreen July 2, 2026 07:53
@CarltonXiang CarltonXiang added the area:core MOS 编排层 / 框架底座 / 跨模块问题 label Jul 2, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator Author

🤖 Open Code Review

Target: PR #1868
Task: manual-ocr:MemTensor/MemOS#1868
Base: dev-v2.0.22
Head: autodev/MemOS-1527

🔍 OpenCodeReview found 26 issue(s) in this PR.


1. src/memos/api/handlers/suggestion_handler.py (L57)

The prompt templates SUGGESTION_QUERY_PROMPT_EN, SUGGESTION_QUERY_PROMPT_ZH, and FURTHER_SUGGESTION_PROMPT each already begin with their own persona definition (e.g., "You are a helpful assistant that can help users to generate suggestion query."). Moving them into the user role creates a role conflict: the persona statement inside the prompt now reads as if the user is declaring what the AI should be, which is semantically incorrect and may confuse the model.

Consider one of these approaches:

  1. Strip the opening persona lines from the prompt templates so they serve purely as task instructions in the user message.
  2. Extract only the persona/role portion from each template into the system message content, keeping the remaining task instructions there as well (i.e., keep a single system message but with a trimmed, focused persona).
💡 Suggested Change

Before:

            {"role": "user", "content": further_suggestion_prompt},

After:

            # Ensure FURTHER_SUGGESTION_PROMPT does not contain a persona prefix
            # (e.g., "You are a helpful assistant.") to avoid role conflict.
            {"role": "user", "content": further_suggestion_prompt},

2. src/memos/api/handlers/suggestion_handler.py (L121-L125)

The system message persona is hardcoded in English ("You are a helpful assistant that generates suggestion queries based on the user's recent memories."), but this code path also handles Chinese (language == "zh") by using SUGGESTION_QUERY_PROMPT_ZH. For a consistent user experience — especially when the LLM infers response language from the system message — the persona should match the language of the prompt.

Suggestion:

system_persona = (
    "你是一个有用的助手,帮助用户生成建议查询。"
    if language == "zh"
    else "You are a helpful assistant that generates suggestion queries based on the user's recent memories."
)
message_list = [
    {"role": "system", "content": system_persona},
    {"role": "user", "content": suggestion_prompt.format(memories=memories)},
]
💡 Suggested Change

Before:

            {
                "role": "system",
                "content": "You are a helpful assistant that generates suggestion queries based on the user's recent memories.",
            },
            {"role": "user", "content": suggestion_prompt.format(memories=memories)},

After:

            {
                "role": "system",
                "content": "你是一个有用的助手,帮助用户生成建议查询。"
                if language == "zh"
                else "You are a helpful assistant that generates suggestion queries based on the user's recent memories.",
            },
            {"role": "user", "content": suggestion_prompt.format(memories=memories)},

3. src/memos/api/handlers/suggestion_handler.py (L57)

Role conflict: The prompt templates (FURTHER_SUGGESTION_PROMPT, SUGGESTION_QUERY_PROMPT_EN, SUGGESTION_QUERY_PROMPT_ZH) each already begin with their own persona definition (e.g., "You are a helpful assistant..." or "你是一个有用的助手..."). Moving the full prompt body into the user role without stripping that opening persona line means the user message now declares what the AI should be, which is semantically incorrect and may confuse the model.

Consider stripping the persona prefix from the prompt templates so they contain only task instructions when placed in the user message.


4. src/memos/api/handlers/suggestion_handler.py (L123)

The system message persona is hardcoded in English, but this code path also handles Chinese users (language == "zh") using SUGGESTION_QUERY_PROMPT_ZH. For a consistent experience — especially when the LLM infers response language from the system message — the persona should match the language of the selected prompt.

Suggested fix:

system_persona = (
    "你是一个有用的助手,帮助用户生成建议查询。"
    if language == "zh"
    else "You are a helpful assistant that generates suggestion queries based on the user's recent memories."
)

5. src/memos/api/handlers/suggestion_handler.py (L57)

Role conflict: FURTHER_SUGGESTION_PROMPT already begins with its own persona statement ("You are a helpful assistant."). Moving it to the user role without stripping that opening line means the user message declares what the AI should be, which is semantically incorrect and may confuse the model. Consider removing the persona prefix from the template so it contains only task instructions when used as a user message.


6. src/memos/api/handlers/suggestion_handler.py (L123)

The system persona is hardcoded in English, but this code path also handles Chinese users (language == "zh") using SUGGESTION_QUERY_PROMPT_ZH. Having an English system message paired with a Chinese user message creates a language mismatch that may cause the model to respond in English. Consider making the persona language-conditional:

system_persona = (
    "你是一个有用的助手,帮助用户生成建议查询。"
    if language == "zh"
    else "You are a helpful assistant that generates suggestion queries based on the user's recent memories."
)

Also note that SUGGESTION_QUERY_PROMPT_ZH and SUGGESTION_QUERY_PROMPT_EN both already begin with their own persona declarations. Moving them to the user role without stripping those opening lines creates the same role conflict as described above for FURTHER_SUGGESTION_PROMPT.


7. tests/api/test_suggestion_handler.py (L40-L45)

[Correctness - Medium] Brittle magic-number length assertions

The checks len(system_content) < 200 and len(user_content) > 100 are arbitrary thresholds that encode no semantic meaning. The current system message ("You are a helpful assistant that generates suggestion queries based on the user's recent memories.") is 97 characters, well under 200 — but a future maintainer rewriting the persona to be slightly more descriptive could silently break this threshold, or shrink it in the other direction without any functional regression.

Prefer content-based assertions that verify what is in each message rather than how long it is:

# Instead of length checks:
assert "assistant" in system_content.lower() or "helpful" in system_content.lower()
assert "memor" in user_content.lower() or "suggest" in user_content.lower()

8. tests/api/test_suggestion_handler.py (L31-L34)

[Correctness - Medium] Fragile call_args[0][0] extraction

All four tests extract the message_list using index-based subscripting on call_args. This silently returns None (or raises IndexError) if the production code ever switches to a keyword argument call like llm.generate(message_list=[...]). In Python 3.8+, call_args.args and call_args.kwargs are the stable API:

call_args = mock_llm.generate.call_args
message_list = call_args.args[0] if call_args.args else call_args.kwargs.get("message_list")
assert message_list is not None, "generate() was not called with a positional or keyword message_list"

Since this pattern appears in every test, extracting it into a shared helper would fix all four call-sites at once (see DRY note below).


9. tests/api/test_suggestion_handler.py (L19-L28)

[Correctness - Low] Return value of handle_get_suggestion_queries is never asserted

The three tests that call handle_get_suggestion_queries never capture or assert the return value. If the function is accidentally refactored to return None, all three tests still pass. The mock is already set up with a parseable JSON response, so asserting the return value costs nothing:

result = handle_get_suggestion_queries(
    user_id="test_user",
    language="en",
    message=None,
    llm=mock_llm,
    naive_mem_cube=mock_mem_cube,
)
assert result == ["test1", "test2", "test3"]

10. tests/api/test_suggestion_handler.py (L16-L17)

[Correctness - Low] Zero-memories branch is the only branch exercised

mock_mem_cube.text_mem.search.return_value = [] means every test hits the empty-memories code path. The production code joins memory strings into the prompt ("\n".join([m.memory[:200] for m in text_mem_results])), but there is no test that populates memories and verifies they are correctly embedded in the user message. A formatting regression in that join/truncation logic would go undetected.

Add a test with populated memories:

mock_mem = Mock()
mock_mem.memory = "User likes hiking in the mountains."
mock_mem_cube.text_mem.search.return_value = [mock_mem]
# ... then assert 'hiking' in message_list[1]['content']

11. tests/api/test_suggestion_handler.py (L5-L8)

[Maintainability - Medium] Direct import of a private function couples tests to implementation details

_get_further_suggestion is prefixed with _, signalling it is an internal implementation detail. Testing it directly creates a tight coupling: if it is inlined, renamed, or merged into the public function, this test breaks even though observable behaviour is unchanged.

The same structural assertion is already exercised indirectly by test_handle_get_suggestion_queries_with_further_message. Consider removing this direct test and instead strengthening that integration-level test with a return-value assertion (assert result == ["suggestion1", "suggestion2"]). If the function genuinely warrants its own test, it should be made public.


12. tests/api/test_suggestion_handler.py (L30-L37)

[Maintainability - Low] Duplicated message-structure assertion block (DRY violation)

The 5-line pattern below is copy-pasted into all four tests with only trivial variations:

call_args = mock_llm.generate.call_args
message_list = call_args[0][0]
assert len(message_list) >= 2
assert message_list[0]["role"] == "system"
assert message_list[1]["role"] == "user"

Extract a shared helper to make tests easier to maintain and fix if the extraction logic changes:

def _assert_system_user_structure(mock_llm):
    call_args = mock_llm.generate.call_args
    message_list = call_args.args[0] if call_args.args else call_args.kwargs["message_list"]
    assert len(message_list) >= 2
    assert message_list[0]["role"] == "system"
    assert message_list[1]["role"] == "user"
    return message_list

13. tests/api/test_suggestion_handler.py (L77-L78)

[Maintainability - Low] Two near-identical tests for language variants — consider parametrize

test_handle_get_suggestion_queries_message_structure and test_handle_get_suggestion_queries_zh_message_structure differ only in language='en' vs language='zh' and the mock JSON content. Maintaining two separate functions doubles the update burden if setup changes.

Use @pytest.mark.parametrize:

import pytest

@pytest.mark.parametrize("language,mock_json", [
    ("en", '{"query": ["test1", "test2", "test3"]}'),
    ("zh", '{"query": ["测试1", "测试2", "测试3"]}'),
])
def test_handle_get_suggestion_queries_message_structure(language, mock_json):
    ...

14. tests/api/test_suggestion_handler.py (L103-L106)

[Test Coverage - Medium] No error-path coverage

All four tests mock a successful JSON response. The production code calls json.loads(clean_json_response(response)) with no visible error guard in the diff, meaning a malformed LLM response propagates a raw json.JSONDecodeError to the caller. None of the tests verify:

  • llm.generate() returning malformed JSON or an empty string
  • llm.generate() raising an exception (e.g., network timeout)
  • The parsed JSON missing the "query" key

Add at least one negative test:

def test_handle_get_suggestion_queries_malformed_json():
    mock_llm = Mock()
    mock_llm.generate.return_value = "not valid json"
    ...
    # assert raises or returns a safe fallback

15. src/memos/api/handlers/suggestion_handler.py (L57)

Role conflict: FURTHER_SUGGESTION_PROMPT already begins with its own persona statement ("You are a helpful assistant."). Moving it to the user role without stripping that opening line means the user message is declaring the AI's identity, which is semantically incorrect and may confuse the model. Consider removing the persona prefix from the template so it contains only task instructions when used as a user message.


16. src/memos/api/handlers/suggestion_handler.py (L123)

The system persona is hardcoded in English, but this code path also serves Chinese users (language == "zh") using SUGGESTION_QUERY_PROMPT_ZH. An English system message paired with a Chinese user message creates a language mismatch that may cause the model to respond in English instead of Chinese. Consider making the persona language-conditional:

system_persona = (
    "你是一个有用的助手,帮助用户生成建议查询。"
    if language == "zh"
    else "You are a helpful assistant that generates suggestion queries based on the user's recent memories."
)

Additionally, SUGGESTION_QUERY_PROMPT_ZH and SUGGESTION_QUERY_PROMPT_EN both already contain persona declarations at the top. Moving them into the user role without removing those lines creates the same role-conflict issue noted above for FURTHER_SUGGESTION_PROMPT.


17. src/memos/api/handlers/suggestion_handler.py (L57)

Role conflict: FURTHER_SUGGESTION_PROMPT already begins with its own persona statement ("You are a helpful assistant."). Moving it to the user role without stripping that opening line means the user message is declaring the AI's identity, which is semantically incorrect and may confuse the model. Consider removing the persona prefix from the template so it contains only task instructions when used as a user message.


18. src/memos/api/handlers/suggestion_handler.py (L123)

The system persona is hardcoded in English, but this code path also serves Chinese users (language == "zh") using SUGGESTION_QUERY_PROMPT_ZH. An English system message paired with a Chinese user message creates a language mismatch that may cause the model to respond in English instead of Chinese. SUGGESTION_QUERY_PROMPT_ZH and SUGGESTION_QUERY_PROMPT_EN also both contain persona declarations at the top, creating the same role-conflict issue as noted for FURTHER_SUGGESTION_PROMPT above.

Consider making the persona language-conditional:

system_persona = (
    "你是一个有用的助手,帮助用户生成建议查询。"
    if language == "zh"
    else "You are a helpful assistant that generates suggestion queries based on the user's recent memories."
)

19. tests/api/test_suggestion_handler.py (L40-L45)

[Correctness - Medium] Brittle magic-number length assertions.

len(system_content) < 200 and len(user_content) > 100 carry no semantic meaning. The current system message string is ~97 characters — a minor wording change by a future maintainer (e.g., adding one sentence) would flip this assertion without any functional regression, causing spurious CI failures.

Prefer content-based assertions that verify what is in each message:

# Instead of length checks:
assert "assistant" in system_content.lower() or "helpful" in system_content.lower()
assert "memor" in user_content.lower() or "suggest" in user_content.lower()

20. tests/api/test_suggestion_handler.py (L31-L34)

[Correctness - Medium] call_args[0][0] is fragile and will silently fail if generate() is called with a keyword argument.

All four tests extract the message list via index-based subscripting. If the production code ever switches to llm.generate(message_list=[...]), this returns None or raises IndexError instead of a meaningful assertion failure. Use the stable call_args.args / call_args.kwargs API (Python 3.8+):

call_args = mock_llm.generate.call_args
message_list = call_args.args[0] if call_args.args else call_args.kwargs.get("message_list")
assert message_list is not None

Since this pattern repeats across all four tests, extracting it into a shared helper (see DRY note below) would fix all call-sites at once.


21. tests/api/test_suggestion_handler.py (L19-L28)

[Correctness - Low] Return value of handle_get_suggestion_queries is never asserted.

The function is called but its return value is discarded. If it is accidentally refactored to return None, this test still passes. The mock is already configured with a parseable JSON response, so capturing and asserting the result costs nothing:

result = handle_get_suggestion_queries(
    user_id="test_user",
    language="en",
    message=None,
    llm=mock_llm,
    naive_mem_cube=mock_mem_cube,
)
assert result == ["test1", "test2", "test3"]

22. tests/api/test_suggestion_handler.py (L16-L17)

[Correctness - Low] Only the zero-memories branch is ever exercised.

mock_mem_cube.text_mem.search.return_value = [] means every test hits the empty-memories code path. The production code joins memory strings into the prompt ("\n".join([m.memory[:200] for m in text_mem_results])), but no test populates memories and verifies they are correctly embedded in the user message. A formatting regression in that join/truncation logic would go completely undetected.

Add a test with populated memories:

mock_mem = Mock()
mock_mem.memory = "User likes hiking in the mountains."
mock_mem_cube.text_mem.search.return_value = [mock_mem]
# ... then assert 'hiking' in message_list[1]['content']

23. tests/api/test_suggestion_handler.py (L5-L8)

[Maintainability - Medium] Directly importing and testing a private function (_get_further_suggestion) couples tests to implementation internals.

The _ prefix signals this is an internal detail. If the function is inlined, renamed, or merged into the public API, this test breaks even though observable behaviour is unchanged — a classic over-specification anti-pattern.

The same structural assertion is already exercised indirectly by test_handle_get_suggestion_queries_with_further_message. Consider removing this direct test and instead strengthening that integration-level test with a return-value assertion. If the function genuinely warrants its own test, it should be made public.


24. tests/api/test_suggestion_handler.py (L30-L37)

[Maintainability - Low] Message-structure assertion block is duplicated across all four tests (DRY violation).

The same 5-line pattern appears in every test with only trivial variation. Extract a shared helper to avoid repetition and create a single fix-point if the extraction logic changes:

def _assert_system_user_structure(mock_llm):
    call_args = mock_llm.generate.call_args
    message_list = call_args.args[0] if call_args.args else call_args.kwargs["message_list"]
    assert len(message_list) >= 2
    assert message_list[0]["role"] == "system"
    assert message_list[1]["role"] == "user"
    return message_list

25. tests/api/test_suggestion_handler.py (L77-L78)

[Maintainability - Low] Two near-identical tests differ only in language='en' vs language='zh' and mock JSON content. Maintaining separate functions doubles the update burden if setup or assertions change.

Collapse into a single parametrized test:

import pytest

@pytest.mark.parametrize("language,mock_json", [
    ("en", '{"query": ["test1", "test2", "test3"]}'),
    ("zh", '{"query": ["测试1", "测试2", "测试3"]}'),
])
def test_handle_get_suggestion_queries_message_structure(language, mock_json):
    ...

26. tests/api/test_suggestion_handler.py (L103-L106)

[Test Coverage - Medium] No error-path tests exist.

All four tests mock a successful, well-formed JSON response. The production code calls json.loads(clean_json_response(response)) with no visible error guard, meaning malformed LLM output propagates a raw json.JSONDecodeError to the caller. Missing coverage includes:

  • llm.generate() returning malformed JSON or an empty string
  • llm.generate() raising an exception (e.g., network timeout)
  • Parsed JSON missing the "query" key

Add at least one negative test:

def test_handle_get_suggestion_queries_malformed_json():
    mock_llm = Mock()
    mock_llm.generate.return_value = "not valid json"
    # assert raises a clear exception or returns a safe fallback

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator Author

Automated Test Results: PASSED

Cloud test-engine rerun against dev-v2.0.22 completed successfully.

  • Run: tr-a2258bbe-e95 on cloud test-engine 10011
  • Scope: memos_python_core/changed-repo-python
  • Result: 4 passed, 0 failed, 0 skipped

Manual code review is still required before merge.

@bittergreen bittergreen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, it's fine.

@bittergreen
bittergreen merged commit ca6ca9c into dev-v2.0.22 Jul 2, 2026
0 of 16 checks passed
@bittergreen
bittergreen deleted the autodev/MemOS-1527 branch July 2, 2026 12:20
@CarltonXiang CarltonXiang mentioned this pull request Jul 8, 2026
3 of 5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core MOS 编排层 / 框架底座 / 跨模块问题

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants