Repository navigation
Fix #2356: l3.abstraction: misleading "title must be a non-empty string" error — output tru - #2576
Open
syzsunshine219 wants to merge 1 commit into
Open
syzsunshine219 wants to merge 1 commit into
syzsunshine219 wants to merge 1 commit into
Conversation
…a error When an LLM output is truncated at maxTokens, two cooperating bugs caused a misleading 'title must be a non-empty string' validation error that pointed blame at the model output rather than the real cause. Bug 1 — extractFirstJsonBlock (json-mode.ts): the old loop continued scanning for a nested balanced sub-block when the root opener returned null from walkToClose. This allowed a fully-balanced inner value (e.g. a domain_tags array) to pass JSON.parse successfully, after which the caller's validate() received that nested value and threw a field-level schema error. Fix: stop at the first opener. If walkToClose returns null (root block is truncated), return null immediately rather than hunting for a balanced child. Bug 2 — completeJson (client.ts): finishReason=length was set on LlmCompletion by providers when the token budget was exhausted, but completeJson passed the truncated text straight to parseLlmJson without inspecting it. Fix: check finishReason === 'length' before attempting to parse. Throw LLM_OUTPUT_MALFORMED immediately with finishReason and rawLen in details so the log clearly names the cause. Also adds rawLen to all existing malformed warn rows so token-budget diagnosis is easier without a separate log query. Closes MemTensor#2356
syzsunshine219
marked this pull request as ready for review
November 9, 2026 15:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Restores the change proposed in #2357 for review against
mainafter its former base branch,dev-v2.0.34, was deleted. The source commits and authors are preserved.Source head:
4ce26cc592ef6c63c9417e6e7a55fb0d38c54baaThis PR is open for review. Review and CI against the current base are pending; validation claims in the original description below refer to the earlier snapshot. The original discussion and evidence remain in #2357.
Original change description
Description
Fixed the misleading "title must be a non-empty string" error in L3 world-model generation caused by two cooperating bugs.
Bug 1 —
extractFirstJsonBlock(json-mode.ts): The old loop continued scanning for a nested balanced sub-block when the root opener returnednullfromwalkToClose. A truncated root object like{"title":"...","domain_tags":["network","http"],...left the root{unbalanced, causing the loop to continue and find the innerdomain_tagsarray, which was fully balanced.JSON.parsethen succeeded on that array, and the caller'svalidate()received["network","http","local-services","cron"]instead of the full object — correctly reporting'title' must be a non-empty stringbut pointing blame at the model rather than at token truncation. Fix: stop at the first opener; ifwalkToClosereturnsnull, returnnullimmediately.Bug 2 —
completeJson(client.ts):finishReason === "length"was already present onLlmCompletionbutcompleteJsonpassed the truncated text straight toparseLlmJsonwithout inspecting it. Fix: checkfinishReason === "length"before parsing; throwLLM_OUTPUT_MALFORMEDimmediately withfinishReason: "length"andrawLenin the error details. Also addedrawLento all existing malformed warn log rows for easier token-budget diagnosis.Tests: 5 new test cases added (2 in
json-mode.test.ts, 3 inclient.test.ts). All 43 LLM unit tests pass. Pre-existing failures insubscriber.test.tsare unrelated (SQLite native binding absent in this environment).Related Issue (Required): Fixes #2356
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Automated tests are pending.
Checklist
@whipser030,@hijzyplease review this PR.Reviewer Checklist