Skip to content

Save the conversation at midpoints, and when it fails - #58

Open
gvdongen wants to merge 2 commits into
mainfrom
save-conversation-frequently
Open

gvdongen wants to merge 2 commits into
mainfrom
save-conversation-frequently

Conversation

@gvdongen

Copy link
Copy Markdown
Contributor

Save the conversation while a run is active, and when it fails

run(input, {conversation}) saved once, in finish(), and a failed run saved nothing. An application with a live UI, such as the Restate reference agent (restatedev/agent), needs two things this didn't provide:

  • A shared handler that reads a running turn with LazyStateConversation.page().
  • A failed turn that stays in the conversation, so the next run knows what happened.

With this change, a run saves as it goes and saves on failure.

When a run saves

  • After each model call. A tool step's assistant message and its calls are saved before the tools run. A final answer is saved once afterRun accepts it.
  • When a tool batch ends, with all its results. A background call is saved with its pending acknowledgment, and its real result is saved when the run collects it.
  • When steering is accepted. Steering that arrives mid-batch is saved after the batch's results, because results must follow their calls.
  • When the run ends, as before.

Each save appends the new part of the committed transcript, so the stored conversation stays the full, uncompacted record. Error notes, rejected answers and in-run compaction views are never saved. A run with one tool step makes three appends. A run that answers in one step makes two.

When a run fails

When a terminal error fails the run, such as an AgentTerminalError, a terminal model error or a cancelled invocation, the run:

  1. Stops the work that is still running.
  2. Gives each unfinished call a cancelled result with the message Run failed: <reason>.
  3. Adds a note that the next run's model reads. The type is run-cancelled for a cancelled invocation:
    {"source": "agent-runtime", "type": "run-failed", "message": "model unavailable"}
  4. Saves, then rethrows the original error unchanged.

Some failures save nothing:

  • A retryable error adds no note. Restate retries the invocation, and a journaled save would not match a retry that takes another path.
  • A run that fails before beforeRun accepts its input records nothing. Throwing AgentTerminalError from beforeRun still keeps the input out of the history, as documented.
  • A ConversationConflictError skips the failure save, because saving would conflict again.

The gen SDK allows journal calls after a caught CancelledError, so a cancelled run saves too. An integration test cancels a real invocation mid-tool.

New export: messagesOf(error) returns the failed run's history, with the closed calls and the note, for every run. It works like usageOf(error). A run that uses messages can pass this history to the next run.

Tool calls without results in storage

A batch's calls are saved before their results, so while a batch runs, the stored history ends with calls that have no results:

  • LazyStateConversation.load(), prepareCompaction() and applyCompaction() accept that history. The compaction boundary already stays before calls without results.
  • append() accepts messages that start with results for the calls the previous append left open. The metadata records those calls in a new optional open field. Conversations saved before this change don't have the field, which is correct because they always end complete.
  • checkProtocol returns the unfinished calls instead of throwing. history() throws a new UnfinishedToolCallsError, a RunInputError, unless {unfinished: true} allows them.

Only a killed invocation leaves such a history behind: a retry replays and finishes the batch, and a failed run closes its calls. The next run closes them with a cancelled result saying that the call may have taken effect, and its first save stores those results. The repair runs only when checkProtocol reports unfinished calls. A comment on continuedHistory explains how to replace the full-history check with a check of the last assistant message.

Behavior changes for callers

  • Retries: the input is saved with the first model call. A caller that retries a failed handler with the same input now starts a new turn after the failed one. Use an idempotency key to retry the same request.
  • Failed runs: a failed run with a conversation now leaves its turn in the conversation.
  • Progress events: the steering progress event is emitted after its note is saved.

Tests

  • conversation-progress.test.ts:
    • A shared reader mid-run, and background results.
    • A failure mid-batch, a model error read by the next run, a retryable error, and cancellation.
    • Steering during a model call and mid-batch, and interruption.
    • The repair after a kill, and compaction prepared mid-run and applied after it.
    • A conflicting writer, messagesOf for a messages run, and the number of appends.
  • conversation-replay.test.ts: a completed run and a failed run, resumed from every journal entry with a store that journals its reads and writes. The output and the stored conversation are identical every time.
  • runtime.integration.ts, with always-replay:
    • Turn persistence after completed, stopped, interrupted, failed and failed mid-batch.
    • A shared handler reading a running turn.
    • A cancelled turn.
  • errors.test.ts: an AgentTerminalError from any site now leaves a loadable conversation that ends with the failure note. The exception is beforeRun, which still records nothing.

pnpm build, the 353 unit tests, and the runtime, AG-UI, examples and coding-agent integration suites pass.

Docs

  • README capability table.
  • CONTEXT.md ("Conversation").
  • restate-agents skill references: conversations-and-controls.md, error-handling.md, approvals-and-guardrails.md and agent-basics.md.
  • Comments in lessons 03 and 09.

🤖 Generated with Claude Code

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Approach: sound — persist committed transcript checkpoints and terminal-failure closure while avoiding failure writes on retryable errors.

Reviewed changes The initial PR diff was reviewed across all 18 changed files:

  • Progress and failure persistence: AgentRun saves model/tool progress, accepted steering, and terminal failure histories; messagesOf(error) exposes history for callers using messages.
  • Conversation protocol: LazyStateConversation and validation support appends that leave calls open, live reads and compaction over those histories, and recovery of calls left open by a killed invocation.
  • Coverage and docs: tests exercise progress visibility, cancellation and failure, recovery, compaction, and replay; examples and skill references document the behavior and retry implications.

Important

This run used your free trial credits ($9.08 of $10 left) on GPT Luna, a fast, lightweight model. Connect a provider → to run the model you choose.

Pullfrog  | View workflow run | Using gpt-6-luna (free trial credits) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes This run reviewed the invocation-scoped state-read cache added since the prior Pullfrog review.

  • Journal-read caching: Added a contextLocal cache shared across LazyStateConversation instances using the same store, and updated it after writes to avoid re-reading journaled state keys during loads, compaction, and saves.
  • Snapshot behavior: Added coverage for cache reuse within an invocation and refresh in a later invocation, and documented that repeated reads within one invocation use the first-read snapshot.

Important

This run used your free trial credits ($8.93 of $10 left) on GPT Luna, a fast, lightweight model. Connect a provider → to run the model you choose.

Pullfrog  | View workflow run | Using gpt-6-luna (free trial credits) | 𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant