Skip to content

[Bug] attachSkillSubscriber wired with bgLlm, not bgReflectLlm — skillEvolver dedicated client never invoked #2362

Closed as not planned

Description

@chiefmojo

Pre-submission checklist

  • I have searched existing issues and this hasn't been mentioned before
  • I have read the project documentation and confirmed this issue doesn't already exist
  • This issue is specific to MemOS and not a general software issue

Bug Description

Summary

In core/pipeline/deps.ts, attachSkillSubscriber is wired with llm: bgLlm (the main model), never bgReflectLlm (the dedicated client built from the skillEvolver config block). As a result the skill subscriber's LLM-backed work (crystallization evaluation, evolution) always runs on the main model regardless of skillEvolver.* config, and skillEvolver.lastOkAt never updates because the dedicated client is never invoked.

Root cause

// deps.ts, attachSkillSubscriber call
const skillHandle = attachSkillSubscriber({
    repos: deps.repos,
    embedder: bgEmbedder,
    llm: bgLlm,          // ← BUG: should be bgReflectLlm
    bus: buses.skill,
    l2Bus: buses.l2,
    rewardBus: buses.reward,
    log: log.child({ channel: "core.skill" }),
    config: algorithm.skill,
    ...
});

bgReflectLlm is constructed earlier in the same file specifically from the skillEvolver config block, but is only read for metadata (health/overview endpoint), never passed into the skill subscriber itself.

Verified via git log -L on this call site: it has never been wired to bgReflectLlm since the v2.0 Reflect2Evolve rewrite (60b97444) — this is not a regression, it's never worked.

Relation to #2148

This is a distinct instance of the same wiring-bug class fixed in #2148 (PR #2151) — that issue was about captureRunner's reflectLlm slot getting the wrong client (skill-evolver client where main llm was needed). This issue is the opposite direction: the skill subscriber never gets the dedicated skillEvolver client it's supposed to have. Different call site, not fixed by #2151.

Impact

  • Operators configuring a distinct skillEvolver.* model (e.g. to isolate skill-evolution cost/quality from the main model) get silently ignored — the main llm handles it instead.
  • skillEvolver.lastOkAt in the overview/health endpoint reads null indefinitely on any install that relies on it as a liveness signal for skill evolution, even when skill evolution is running fine (just on the wrong model).

Suggested Fix

const skillHandle = attachSkillSubscriber({
    ...
    llm: bgReflectLlm ?? bgLlm,
    ...
});

Activity

  1. added
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    area:coreMOS 编排层 / 框架底座 / 跨模块问题
    status:in-progressSomeone or AI is working on it | 人工或 AI 正在处理
    types:bugSomething isn't working | 功能异常
    on Sep 14, 2026
  2. self-assigned this
    on Sep 14, 2026
  3. Memtensor-AI commented on Sep 14, 2026

    @Memtensor-AI
    Collaborator

    🤖 AutoDev has picked up this issue and started working on it.

    Task ID: 864b1ca0f6c45de2
    Working branch: bugfix/autodev-2362-20260914203903343
    Target branch: latest dev* branch
    Workflow: opsp (analysis → coding → testing → PR)

    I will post the PR link here once done. If I need more information, I will ask in the comments.

  4. added a commit that references this issue on Sep 14, 2026
    d5b3bea
  5. Memtensor-AI commented on Sep 14, 2026

    @Memtensor-AI
    Collaborator

    ✅ AutoDev task 864b1ca0f6c45de2 completed.

    Summary: Fixes #2362 — attachSkillSubscriber in apps/memos-local-plugin/core/pipeline/deps.ts was wired with llm: bgLlm (the main model) instead of bgReflectLlm (the dedicated client built from the skillEvolver.* config block). As a result, operators configuring a distinct skillEvolver.provider / skillEvolver.model were silently ignored — skill crystallization and evolution ran on the main model regardless — and overview.skillEvolver.lastOkAt was pinned to null because the dedicated client was never invoked. This is a distinct instance of the wiring-bug class fixed in #2148 (opposite direction: that fix corrected captureRunner's slot; this one corrects the skill subscriber's slot). Verified via git log -L in the issue: this call site has never been wired to bgReflectLlm since the v2.0 Reflect2Evolve rewrite.

    Change is a single line at deps.ts:339: llm: bgReflectLlm ?? bgLlm. The ?? fallback preserves existing behaviour for installs without a distinct skillEvolver config block.

    Guarded by a new dedicated regression test apps/memos-local-plugin/tests/unit/pipeline/skill-reflect-llm-wiring.test.ts (2 cases) mirroring the pattern of capture-reflect-llm-wiring.test.ts (the sibling #2148 guard). RED was reproduced before the fix (expected 'main-llm' to be 'skill-evolver-llm'); GREEN confirmed after. Full pipeline + skill-subscriber suite: 14 files / 92 tests pass, including the sibling capture-reflect-llm-wiring and health-model-display neighbours. npm run lint (tsc --noEmit) clean.

    Path classification: Bug quick-fix — one-line DI correction, no openspec artifacts required. Task file archived to sibling specs repo (memos-autodev-specs / main, commit ff25a7f). Reviewer: @WeiminLee.

    Base branch: main
    Branch: bugfix/autodev-2362-20260914203903343
    Commit: d5b3bea
    PR: #2365
    Assigned to: @wustzdy
    Reviewers: @WeiminLee

  6. added
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审
    and removed
    ai:codingAI agent is coding | AI 正在编码
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    on Sep 14, 2026
  7. chiefmojo commented on Sep 16, 2026

    @chiefmojo
    ContributorAuthor

    Production confirmation — three-install fleet, all affected

    Environment: memos-local-plugin v2.0.16, Hermes host agent, skillEvolver configured with an explicit (non-inherited) provider/model. Three independent production installs.

    Call site still present. Verified against main (176d4f67) and the newest dev-v2.0.30: deps.ts:339 is still llm: bgLlm, and bgReflectLlm (constructed at :220) is read only for metadata at :279-280. The AutoDev PR #2365 targeting dev-v2.0.34 was closed unmerged, and no equivalent change has landed on either branch.

    Observed state — GET /api/v1/health, identical shape on all three:

    "llm":          { "available": true, "model": "<main model>",     "lastOkAt": 1789591389007 },
    "embedder":     { "available": true, "model": "<local embedder>", "lastOkAt": 1789591387015 },
    "skillEvolver": { "available": true, "inherited": false, "lastOkAt": null, "lastError": null }

    Row evidence — api_logs where tool_name='system_model_status':

    install llm embedding skillEvolver newest skillEvolver row
    A 5,130 1,044 0 —
    B 4,964 986 144 2026-08-29
    C 5,695 1,022 195 2026-08-23

    The dedicated client is demonstrably not being used, and the skill work is running on the main model. Nothing looks wrong in the logs, which is the point: install A's reflection pass completes normally all day — capture.reflect.done … llmCalls={reflectionSynth,alphaScoring,batchedReflection,…} warnings=0 — against a dedicated model that is configured, built (reflectLlm.ready at every boot) and never called.

    Worth deciding alongside #2148 / PR #2151: that change deliberately moved the capture batch-reflection path off the dedicated client onto the main llm. Taken together with this issue, both the capture and the skill-subscriber reflection paths now land on the main model, so a distinct skillEvolver.* config is effectively inert on a current install. Confirming the intended routing for both call sites at once seems worth doing while this is open.

    A second defect in the same card, independent of the wiring

    Even once the wiring above is fixed, this card can still misreport, because the lookup is a fixed top-N scan that filters after the limit: findLatestPersistedModelStatus() (memory-core.ts:5900) calls apiLogs.list({ toolName: "system_model_status", limit: 500 }) (:5913) and only then compares role / provider / model.

    On installs B and C the newest skillEvolver row sits 2,193 and 4,896 status rows below the head of that table — outside the 500-row window — so the row exists but is unreachable. The endpoint then falls back to a per-process in-memory counter that resets to null on every bridge restart, which is what presents as "Not called yet" indefinitely.

    The tell is that B and C do have history: their most recent skillEvolver rows are dated 2026-08-29 and 2026-08-23. On those installs the card is not reporting a stale slot — it is reporting nothing, because it cannot see the rows.

    Since that is a distinct root cause from the wiring bug, I've opened it separately as #2380 so it isn't lost in this thread.

    Environment and fuller logs available on request.

  8. chiefmojo commented on Sep 17, 2026

    @chiefmojo
    ContributorAuthor

    Addendum — the rolling cap, and a date for the silence

    Re-running the same query three hours later produced two things.

    1. api_logs is a rolling 10,000-row window, so the counts above decay.

    install skillEvolver rows newest row rows above it
    A 0 — —
    B 144 2026-08-29 2,193 (unchanged)
    C 99 (was 195) 2026-08-23 5,276 (was 4,896)

    Install C lost 96 skillEvolver rows in three hours — its oldest rows were the evicted ones — while gaining 380 rows above the target. That's a corollary for this class rather than a problem with the fix: the evidence that a low-frequency slot used to fire is itself on a clock, and on a busy install it ages out within weeks of the slot going quiet.

    2. The silence has a date, and it matches a change in this same wiring family.

    The two installs that still hold history show:

    • one stopped at 2026-08-23 — the day after it took a release-candidate build for a week-long soak
    • the other stopped at 2026-08-29 — the day its production deployment landed

    git log -S"reflectLlm: deps.reflectLlm" on core/pipeline/deps.ts returns exactly two commits: the v2.0 Reflect2Evolve rewrite that introduced it, and the merge that replaced it with reflectLlm: deps.llm — carrying the fix for #2148 / PR #2151, with the explanatory comment citing that issue.

    That reconciles this report with #2148 rather than contradicting it. Before that change landed here, captureRunner was the only caller of the dedicated skillEvolver client, which is why it was the only source of system_model_status rows for that role. Once it moved to the main llm, the slot went silent — while bgReflectLlm is still constructed and still advertised to the Overview card, so the card shows a configured model that nothing invokes.

    Which is why the two issues are worth landing as a pair:

    Neither alone gets the card telling the truth. Raw row dumps, the per-install dates, or the git log -S output available on request.

  9. github-actions commented on Oct 18, 2026

    @github-actions
    Contributor

    This issue has been automatically marked as stale due to inactivity.

  10. added
    status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.
    on Oct 18, 2026
  11. github-actions commented on Oct 27, 2026

    @github-actions
    Contributor

    This issue has been automatically closed due to inactivity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审area:coreMOS 编排层 / 框架底座 / 跨模块问题status:in-progressSomeone or AI is working on it | 人工或 AI 正在处理status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.types:bugSomething isn't working | 功能异常

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions