Skip to content

l3.abstraction hard-fails entire world-model generation on empty LLM title (no retry/fallback) #2335

Closed as not planned

Description

@hxyinan

Summary

In plugin v2.0.17, l3.abstraction throws LLM_OUTPUT_MALFORMED: 'title' must be a non-empty string whenever the abstraction LLM call returns an empty/whitespace title. This aborts the entire world-model generation for that cluster — no retry, no fallback title, no skip-and-continue — so a single malformed LLM response kills the whole run.

Location

dist/core/memory/l3/abstract.js (v2.0.17), line ~56:

throw new MemosError(ERROR_CODES.LLM_OUTPUT_MALFORMED, "l3.abstraction: 'title' must be a non-empty string", { got: o.title });

Observed behavior (production evidence)

From api_logs in our deployment (Hermes adapter, Windows, openai_compatible LLM):

Period Attempts Failures Dominant failure
2.0.14 era (Aug 10–30) ~2000 ~1990 mostly host handler queue is full (separate issue)
2.0.17 (Aug 30 → Sep 3) 12 1 'title' must be a non-empty string

Sample failure event:

{"kind":"l3.failed","stage":"abstract","error":{"code":"llm_failed","message":"l3.abstraction: 'title' must be a non-empty string"},"clusterKey":"python|_"}

The feature demonstrably works when the LLM behaves (6 world models successfully created since Aug 11), but the empty-title case turns one bad sample into a full pipeline failure.

Suggested fix

Instead of throwing on empty title:

  1. Retry once with a repair prompt ("return JSON with a non-empty title"), or
  2. Derive a fallback title (e.g. join domainTags, or first line of the body), and only skip storing that one model — never fail the whole abstraction stage.

Note: dist/core/memory/l2/induce.js:54 has the same throw-on-malformed pattern for multiple fields; the same hardening may be worth applying there.

Environment

  • memos-local plugin 2.0.17 (npm memtensor-memos-local-plugin-2.0.17)
  • Adapter: Hermes (adapters/hermes/memos_provider)
  • Windows 11, Node v24, LLM provider openai_compatible (deepseek-family via local gateway)

Activity

  1. added
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    status:in-progressSomeone or AI is working on it | 人工或 AI 正在处理
    types:bugSomething isn't working | 功能异常
    on Sep 3, 2026
  2. self-assigned this
    on Sep 3, 2026
  3. Memtensor-AI commented on Sep 3, 2026

    @Memtensor-AI
    Collaborator

    🤖 AutoDev has picked up this issue and started working on it.

    Task ID: 60cd97aef73c495f
    Working branch: bugfix/autodev-2335-20260903031432962
    Target branch: latest dev* branch
    Workflow: opsp (analysis → coding → testing → PR)

    I will post the PR link here once done. If I need more information, I will ask in the comments.

  4. Memtensor-AI commented on Sep 3, 2026

    @Memtensor-AI
    Collaborator

    ✅ AutoDev task 60cd97aef73c495f completed.

    Summary: Fixes issue #2335: l3.abstraction no longer aborts a whole cluster's world-model generation when the LLM returns an empty title. The validate callback still requires the triple (environment / inference / constraints) but no longer throws on empty title; normaliseDraft synthesises a fallback title from domain_tags (title-cased, joined with " · "), the first non-empty environment[].label, or a static "Untitled world model". abstractDraft emits an abstract.title_fallback warn breadcrumb so ops can still spot LLM regressions.

    Applied the same softening to l2.induction: title becomes a display attribute (falls back to signatureLabel, else "Untitled policy") while trigger and procedure remain load-bearing and still fail the draft when missing.

    Verified with vitest: the two focused test files run 16 tests (all passing) and the full tests/unit/memory/l2 + l3 suite runs 89 tests (all passing). tsc -p tsconfig.json --noEmit is clean. Changes touch four files under apps/memos-local-plugin/{core,tests}/memory/{l2,l3}; opsp task file and openspec change artefacts are staged in .ai-tasks/ and openspec/changes/ for scheduler archival.

    Base branch: main
    Branch: bugfix/autodev-2335-20260903031432962
    Commit: c2cf01b
    PR: #2338
    Assigned to: @whipser030
    Reviewers: @whipser030, @hijzy

  5. added
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审
    and removed
    ai:codingAI agent is coding | AI 正在编码
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    on Sep 3, 2026
  6. github-actions commented on Oct 4, 2026

    @github-actions
    Contributor

    This issue has been automatically marked as stale due to inactivity.

  7. added
    status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.
    on Oct 4, 2026
  8. github-actions commented on Oct 14, 2026

    @github-actions
    Contributor

    This issue has been automatically closed due to inactivity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审area:pluginOpenClaw & Hermesstatus:in-progressSomeone or AI is working on it | 人工或 AI 正在处理status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.types:bugSomething isn't working | 功能异常

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions