Repository navigation
Description
Summary
With @memtensor/memos-local-plugin v2.0.18 (Hermes local adapter, full self-evolution mode), L3 world-model generation fails almost always with a misleading validation error:
[core.memory.l3.abstract] abstract.llm_failed err="l3.abstraction: 'title' must be a non-empty string"
[llm.json] malformed op="l3.abstraction.v2"
The model's answer is actually valid JSON with a non-empty title at the top level. The real failure chain is:
- The L3 answer is truncated at
maxTokens(long answer + a reasoning model burning 5–12k tokens on thoughts). JSON.parseon the full text fails (Unterminated string).parseLlmJsonfalls back toextractFirstJsonBlock, which returns the first balanced{...}/[...]— a nested block (in our case thedomain_tagsarray).validatethen runs against that nested value and reports'title' must be a non-empty string— pointing the debugging effort at the model, not at the truncation.
Related: #1755 lists these schema failures as a secondary symptom; the root cause was not analyzed there.
Environment
- Plugin:
@memtensor/memos-local-pluginv2.0.18 (Hermes local adapter, Windows) - Memory mode: full self-evolution (
algorithm.lightweightMemory.enabled: false) - LLM: OpenAI-compatible endpoint,
deepseek-v4-flash(reasoning model),temperature: 0,response_format: {"type":"json_object"} - Config before the fix:
llm.maxTokens: 8192,llm.timeoutMs: 120000 - Volume:
world_model_generate— 6 successes out of 456 calls (1/22 today).
Evidence (raw answers captured from completeJson)
attempt 1: len 13082, JSON.parse error: Unterminated string starting at: line 1 column 11856 (char 11855)
raw starts with: {"title":"...","domain_tags":["network","http","local-services","cron"],...
attempt 2: len 0 -> "LLM output not valid JSON: empty response"
parsedShape (what parseLlmJson actually returned): array[4] <- the `domain_tags` array
Model-side limit probe against the same endpoint:
| max_tokens | finish_reason | completion_tokens | reasoning_tokens |
|---|---|---|---|
| 8192 | length | 8192 | 5004 |
| 16384 | length | 16384 | 5081 |
| 32768 | stop | 17374 | 11827 |
The prompt asks for a body field (full markdown rendering that duplicates environment/inference/constraints), so an L3 answer is ~13k chars — reasoning + answer do not fit in the old 8192 budget.
Fix that works locally
llm:
maxTokens: 32768
timeoutMs: 300000After restarting the bridge/daemon, a manual L3 run over the live DB:
run.done: clusters=6 created=0 merged=6 skipped=0
5 world models updated, confidence ~1.0, no further malformed warnings.
Suggested improvements
- Don't let the parse fallback masquerade as a schema error. When the top-level text fails to parse, either restrict the extracted-block fallback to a value that satisfies the caller's shape, or surface the original parse error (with
rawPreview) instead of validating a nested value. - Log the raw answer on malformed output.
logs/llm.jsonlstays empty andmemos.logonly carries"msg":"malformed"without the payload — which makes this whole bug class invisible. - Handle
finishReason === "length"explicitly: retry with a "shorter answer" instruction (or raise the budget) instead of a blind retry; the current retry sometimes returns an empty response. - Trim the L3 schema:
bodyduplicates the three structured sections; a short body (or a derived one) would roughly halve the answer size.
Happy to attach the small local diagnostic patch (adds rawPreview / rawLen / parsedShape to the malformed warning) if it helps.
Activity
Memtensor-AI commentedon Sep 10, 2026
🤖 AutoDev has picked up this issue and started working on it.
Task ID:
035ad74cb9dcd82aWorking branch:
bugfix/autodev-2356-20260910132328679Target branch:
latest dev* branchWorkflow: opsp (analysis → coding → testing → PR)
I will post the PR link here once done. If I need more information, I will ask in the comments.
fix(llm): surface truncation at maxTokens instead of misleading schem…
Memtensor-AI commentedon Sep 10, 2026
✅ AutoDev task
035ad74cb9dcd82acompleted.Summary: Fixed the misleading "title must be a non-empty string" error in L3 world-model generation caused by two cooperating bugs.
Bug 1 —
extractFirstJsonBlock(json-mode.ts): The old loop continued scanning for a nested balanced sub-block when the root opener returnednullfromwalkToClose. A truncated root object like{"title":"...","domain_tags":["network","http"],...left the root{unbalanced, causing the loop to continue and find the innerdomain_tagsarray, which was fully balanced.JSON.parsethen succeeded on that array, and the caller'svalidate()received["network","http","local-services","cron"]instead of the full object — correctly reporting'title' must be a non-empty stringbut pointing blame at the model rather than at token truncation. Fix: stop at the first opener; ifwalkToClosereturnsnull, returnnullimmediately.Bug 2 —
completeJson(client.ts):finishReason === "length"was already present onLlmCompletionbutcompleteJsonpassed the truncated text straight toparseLlmJsonwithout inspecting it. Fix: checkfinishReason === "length"before parsing; throwLLM_OUTPUT_MALFORMEDimmediately withfinishReason: "length"andrawLenin the error details. Also addedrawLento all existing malformed warn log rows for easier token-budget diagnosis.Tests: 5 new test cases added (2 in
json-mode.test.ts, 3 inclient.test.ts). All 43 LLM unit tests pass. Pre-existing failures insubscriber.test.tsare unrelated (SQLite native binding absent in this environment).Base branch:
mainBranch:
bugfix/autodev-2356-20260910132328679Commit:
4ce26ccPR: #2357
Assigned to: @wustzdy
Reviewers: @whipser030, @hijzy
github-actions commentedon Oct 11, 2026
This issue has been automatically marked as stale due to inactivity.
github-actions commentedon Oct 19, 2026
This issue has been automatically closed due to inactivity.