Skip to content

fix(cli): render thinking blocks above the answer when the model streams them last - #287

Open
li-xiu-qi wants to merge 1 commit into
stepfun-ai:mainfrom
li-xiu-qi:fix/thinking-before-answer
Open

li-xiu-qi wants to merge 1 commit into
stepfun-ai:mainfrom
li-xiu-qi:fix/thinking-before-answer

Conversation

@li-xiu-qi

@li-xiu-qi li-xiu-qi commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Resolves #286

Problem

Under upstream model streaming protocols (such as the OpenAI Responses API streaming format and compatible gateways), reasoning events are occasionally flushed after the text generation events. Specifically, the SSE stream can emit response.output_item.added (type: "message") followed by text deltas before emitting response.output_item.added (type: "reasoning") for the reasoning summary or signature.

The client's provider streaming layer (packages/providers/src/api/openai-responses-shared.ts) appends incoming blocks to output.content in packet arrival order. Consequently, in those turns, message.content is assembled with the answer block preceding the thinking block ([text, thinking]). Because the TUI transcript walks message.content sequentially during rendering, the collapsed thinking block was painted below the answer in those turns, but jumped above the answer in turns where reasoning arrived first.

Mutating message.content in the provider or session data layer would be architectural malpractice:

  1. Wire fidelity and replay safety: message.content is persisted and re-sent to upstream APIs as conversation context in subsequent turns. Arbitrarily reordering raw blocks risks schema rejection on strict APIs or corrupting prompt context.
  2. Preserving interleaved reasoning: Advanced models and agents interleave thoughts with tool executions (thinking -> toolCall -> thinking -> answer). Flattening or reordering blocks at the data layer would destroy causal execution sequence.

Changes

  • Introduce withThinkingBeforeAnswer() in apps/cli/src/ui/view/transcript/assistant-message.ts as a pure, display-layer normalization function.
  • It detects the degenerate pattern where visible answer blocks strictly precede thinking blocks with zero tool calls. In that specific case, it hoists the trailing thinking suffix ahead of the answer for UI presentation.
  • Interleaved runs (thinking leading, alternating thinking/text) and any message containing tool calls are returned completely untouched, preserving multi-step agent workflows.
  • Apply the normalization in AssistantMessageComponent.updateContent() (pi skin) and partitionAssistantContent() (step skin).
  • Add regression coverage in both assistant-message.test.ts and step-message.test.ts verifying that trailing thinking is hoisted, while leading and interleaved thoughts remain in place.

Behavior and compatibility

  • Only assistant messages where thinking blocks all arrive after the first answer block change visual presentation; the standard shape (thinking leading) and alternating/tool-calling shapes are byte-for-byte unchanged.
  • message.content itself is never mutated in place. No persisted session data, API wire format, provider request, or CLI configuration changes.
  • Tool-call layout is completely unaffected: tool calls are rendered by their own components and continue to act as run boundaries.

Validation

  • Reproduced the inconsistent placement end-to-end against live model streams (screenshot attached to issue [Bug] Thinking block position is inconsistent across turns: sometimes above the answer, sometimes below it #286).
  • Verified via an end-to-end HTTP/SSE simulation that upstream out-of-order item arrival produces [text, thinking] and that withThinkingBeforeAnswer cleanly normalizes presentation without altering stored data.
  • New unit tests:
    • "hoists thinking ahead of the answer when the model streams it last" in apps/cli/test/assistant-message.test.ts and apps/cli/test/step-message.test.ts.
    • "keeps segmented reasoning in place when thinking already leads" and "keeps later reasoning behind the first answer run when thinking already leads".
  • npx vitest run apps/cli/test/assistant-message.test.ts apps/cli/test/step-message.test.ts: all 29/29 tests pass (25 existing + 4 new).
  • npx tsgo -p tsconfig.build.json --noEmit in apps/cli: clean.
  • npx biome check on changed files: clean.
  • Guard checks passed: check:ui-layer, check:layer-direction, check:ts-relative-imports.
  • Full apps/cli and packages/* suite run on Windows: compared against an unmodified stash baseline, confirming zero candidate-introduced regressions. Fork/upstream Linux CI is clean.

Limitations

Not applicable: this is a display-only rendering normalization covered by unit tests across both skins; no unresolved boundary or behavioral regression is known.

@li-xiu-qi
li-xiu-qi marked this pull request as draft October 28, 2026 17:54
@li-xiu-qi
li-xiu-qi force-pushed the fix/thinking-before-answer branch from a1a645b to 58381d6 Compare October 28, 2026 18:39
@li-xiu-qi
li-xiu-qi marked this pull request as ready for review October 28, 2026 18:52
@li-xiu-qi
li-xiu-qi marked this pull request as draft October 29, 2026 09:10
…ams them last

Provider streaming assembles message.content in arrival order, so turns
that flush the reasoning stream after the answer text rendered the
thinking block below the answer while other turns rendered it above.
Normalize the render path: when no thinking block precedes the first
answer block, hoist thinking ahead of the answer in both the pi and step
skins. Alternating runs are left untouched.
@li-xiu-qi
li-xiu-qi force-pushed the fix/thinking-before-answer branch from 58381d6 to 2dbe319 Compare October 29, 2026 09:13
@li-xiu-qi
li-xiu-qi marked this pull request as ready for review October 29, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Thinking block position is inconsistent across turns: sometimes above the answer, sometimes below it

1 participant