Skip to content

Keep what arrived when a stream dies - #5072

Open
elias-ba wants to merge 1 commit into
timeout-stuck-message-reaperfrom
timeout-keep-partial-response
Open

Keep what arrived when a stream dies#5072
elias-ba wants to merge 1 commit into
timeout-stuck-message-reaperfrom
timeout-keep-partial-response

Conversation

@elias-ba

@elias-ba elias-ba commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Description

The saved message was built entirely from Apollo's complete event. If the stream died before that arrived, everything the user had just watched appear on screen was thrown away, including workflow YAML that had already been sent.

The stream fold now keeps the text and the YAML as they arrive, and saves them if the complete event never comes. The message is marked as failed, so it reads as an interrupted answer rather than a finished one.

Job chat's suggested code can't be rescued this way. It's assembled from a tool call that only completes when the model finishes, so mid-stream it doesn't exist on either side.

Closes #__

Validation steps

  1. mix test test/lightning/ai_assistant/ai_assistant_test.exs
  2. Start a workflow chat against an Apollo you can kill, kill it once text has appeared, and check the message that saves has the text in it.

AI Usage

  • I have used Claude Code

Pre-submission checklist

  • I have performed an AI review of my code
  • I have updated the changelog. (for the whole stack, in #5074)
  • I have ticked a box in "AI usage" in this PR

@github-project-automation github-project-automation Bot moved this to New Issues in Core Aug 15, 2026
@github-actions

Copy link
Copy Markdown

I have enough context to render the review. The PR is entirely AI-assistant reliability plumbing: partial-response preservation on stream death, a stuck-message reaper, failure-category storage/broadcast, shutdown_grace_period sizing, and detached telemetry handler IDs.

Security Review ✅

  • S0 (project scoping): No new user-facing entrypoints; the reaper is a background Oban worker reading ai_chat_messages and oban_jobs internally, and the new failure_category/failure_message fields flow through the existing ai_assistant_channel.ex:39 authorize_session_access gate before broadcast (ai_assistant_channel.ex:296, message_processor.ex:316).
  • S1 (authorization): N/A — no new handle_in/controller actions or policy surface; the reaper and telemetry :stop handler are non-user-invoked, and channel changes only enrich an existing authorized broadcast payload.
  • S2 (audit trail): N/A — chat messages are conversational state, not project/instance configuration, so failure_category/failure_message writes (ai_assistant.ex:1113, message_processor.ex:194, stuck_message_reaper.ex:75) fall outside the S2 audit-resource list.

@elias-ba
elias-ba force-pushed the timeout-stuck-message-reaper branch from 01753c3 to 55042f3 Compare August 15, 2026 22:50
@elias-ba
elias-ba force-pushed the timeout-keep-partial-response branch from 043e22e to 1bb4437 Compare August 15, 2026 22:50
A reply the user watched appear was thrown away the moment the stream
failed. Every event except the final one returned the accumulator
unchanged, so the saved message was built entirely from a complete event
that never came - the panel typed out an answer and then replaced it with
a red box.

The accumulator now carries the text as it arrives, and the yaml that
workflow and global chat send ahead of the text. If the stream ends
without completing, whatever did arrive is saved as an assistant message
marked incomplete rather than discarded.

It is deliberately not presented as a finished answer: the message
carries the incomplete_response category, so the panel can show it as
cut off and offer a retry.

Job chat's suggested code is not recoverable this way and is not
attempted. It is assembled from the tool call after the model finishes,
so when a stream dies mid-answer the code does not exist yet on either
side - there is nothing to save, rather than something being dropped.

Chunks are prepended and reversed once at the end rather than appended to
a growing binary, since a long reply arrives as thousands of them.
@elias-ba
elias-ba force-pushed the timeout-stuck-message-reaper branch from 55042f3 to 3abf115 Compare August 15, 2026 23:40
@elias-ba
elias-ba force-pushed the timeout-keep-partial-response branch from 1bb4437 to 46eb44f Compare August 15, 2026 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New Issues

Development

Successfully merging this pull request may close these issues.

1 participant