Skip to content

[Bug] captureRunner uses reflectLlm (skill-evolver) for batch reflection — should use main llm #2148

Closed

Description

@7889545

Pre-submission checklist

  • I have searched existing issues and this hasn't been mentioned before
  • I have read the project documentation and confirmed this issue doesn't already exist
  • This issue is specific to MemOS and not a general software issue

Bug Description

Summary

In core/pipeline/deps.ts, the captureRunner is wired with reflectLlm: deps.reflectLlm (the skill-evolver model). This means the capture pipeline's batch reflection — which requires JSON output — runs on the skill-evolver model instead of the main llm. When skillEvolver.enableThinking=true and llm.enableThinking=false (as configured for Qwen3), the capture reflection gets enable_thinking=true, producing malformed JSON with <think> tags that fail to parse.

Root Cause

// deps.ts line ~216
const captureRunner = createCaptureRunner({
    ...
    llm: deps.llm,
    reflectLlm: deps.reflectLlm,  // ← BUG: should be deps.llm
    ...
});

Inside capture.ts (line ~286), the batch reflection prefers reflectLlm:

const rLlm = deps.reflectLlm ?? deps.llm;

So when reflectLlm is set (skill-evolver with enableThinking=true), the capture reflection runs on the wrong model.

Impact

  • Capture batch reflection produces malformed JSON (thinking tags mixed into JSON output)
  • malformed JSON errors appear in logs during capture summarization
  • The reflection quality degrades because the JSON-output task runs on a thinking-enabled model

Environment

  • MemOS: v2.0.20 (memos-local-plugin)
  • LLM: Qwen/Qwen3-8B via SiliconFlow
  • Config: llm.enableThinking=false, skillEvolver.enableThinking=true

Suggested Fix

Change the wiring so capture reflection uses the main llm (no thinking for JSON tasks):

const captureRunner = createCaptureRunner({
    ...
    llm: deps.llm,
    reflectLlm: deps.llm,  // capture reflection needs JSON, not thinking
    ...
});

Note: The dedicated l3Llm config slot (PR #1959) already addresses L3 abstraction routing. A similar approach could be considered for capture reflection if independent model control is desired.

Activity

  1. added
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    status:in-progressSomeone or AI is working on it | 人工或 AI 正在处理
    types:bugSomething isn't working | 功能异常
    on Jul 23, 2026
  2. self-assigned this
    on Jul 23, 2026
  3. Memtensor-AI commented on Jul 23, 2026

    @Memtensor-AI
    Collaborator

    🤖 AutoDev has picked up this issue and started working on it.

    Task ID: 7fd6a9dcc8cfb82e
    Working branch: bugfix/autodev-2148-20260723095748278
    Target branch: latest dev* branch
    Workflow: opsp (analysis → coding → testing → PR)

    I will post the PR link here once done. If I need more information, I will ask in the comments.

  4. Memtensor-AI commented on Jul 23, 2026

    @Memtensor-AI
    Collaborator

    ✅ AutoDev task 7fd6a9dcc8cfb82e completed.

    Summary: Fixes issue #2148 by wiring the capture pipeline's batch reflection to the main llm instead of the skill-evolver reflectLlm client. In apps/memos-local-plugin/core/pipeline/deps.ts, createCaptureRunner({ ..., reflectLlm: deps.reflectLlm, ... }) was routing the JSON-output batch reflection pass to whichever model was configured under skillEvolver.*. When operators set skillEvolver.enableThinking=true alongside llm.enableThinking=false (the typical Qwen3 setup on SiliconFlow), the reflection call returned prose containing <think>...</think> blocks that broke JSON parsing and produced malformed JSON errors during capture summarisation.

    The one-line wiring change now passes deps.llm into the capture runner's reflectLlm slot, matching the l3Llm-style routing rationale from PR #1959: JSON tasks stay on the non-thinking main model, and the dedicated reflectLlm client remains exposed on PipelineDeps for reward-evaluator metadata and the overview health endpoint (both unchanged). Inline comments in deps.ts explain the rationale so future edits can't reintroduce the regression.

    A dedicated regression test at apps/memos-local-plugin/tests/unit/pipeline/capture-reflect-llm-wiring.test.ts intercepts createCaptureRunner via vi.mock, feeds distinguishable fake llm / reflectLlm clients into buildPipelineSubscribers, and asserts the capture runner receives the main llm in the reflectLlm slot in both normal and lightweight modes. The test was verified RED on the unfixed code and GREEN after the fix.

    Verification: pnpm exec vitest run tests/unit — 142 files / 1192 passed / 1 skipped. pnpm exec tsc -p tsconfig.json --noEmit — no errors. Branch pushed to origin. Reviewers: @whipser030, @hijzy.

    Base branch: main
    Branch: bugfix/autodev-2148-20260723095748278
    Commit: 6cb5af1
    PR: #2151
    Assigned to: @bittergreen
    Reviewers: @whipser030, @hijzy

  5. added
    ai:testingAI agent is running tests | AI 正在运行测试
    and removed
    ai:codingAI agent is coding | AI 正在编码
    on Jul 23, 2026
  6. added
    ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审
    and removed
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    on Jul 23, 2026
  7. added a commit that references this issue on Jul 23, 2026
    44fc2b5
  8. removed
    ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审
    on Jul 23, 2026
  9. added a commit that references this issue on Jul 24, 2026
    e820406
  10. added a commit that references this issue on Sep 14, 2026
    d5b3bea
  11. added a commit that references this issue on Sep 18, 2026
    a447579
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area:pluginOpenClaw & Hermesstatus:in-progressSomeone or AI is working on it | 人工或 AI 正在处理types:bugSomething isn't working | 功能异常

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions