Follow-up from #1166 (the compaction wipes). While auditing that report I traced the persistence side and found a second, independent path to the same symptom — "the model lost all chat history" — with a different cause. Verified against the current code with executable repros.
The failure chain
-
The atomic write is not durable. cli/src/utils/write-file-atomic.ts writes a temp file and renames it over run-state.json — but never fsyncs. The rename is atomic against concurrent processes, yet after a power loss / hard hang / battery pull, the filesystem can land the rename while the file's data blocks were never written. Result: a truncated or empty run-state.json exactly where the atomic rename was supposed to guarantee a complete file. (The in-code comment acknowledges torn non-atomic writes; the same tear reaches through this pattern without the fsync.)
-
The load path has no recovery. loadMostRecentChatState (cli/src/utils/run-state-storage.ts) parses run-state.json and, on failure, falls back to a placeholder:
runState ??= { output: { type: 'error', message: 'Previous run state could not be restored.' } } as RunState
The placeholder has no sessionState. Meanwhile chat-messages.json is parsed independently and usually survives — so the UI transcript is fully intact.
-
A missing sessionState means a fresh session. sdk/src/run.ts:436-467: when previousRun.sessionState is undefined, runOnce builds a brand-new initialSessionState. The next turn's model context = empty history.
Net effect: the user resumes a chat, sees their whole transcript, sends a message — and the assistant has no idea who they are or what it did. That is precisely the "lost all chat history instead of compacting" report shape, on machines that crashed or lost power mid-session. Reproduced with a temp-dir test: a truncated primary + intact transcript yields a placeholder RunState without sessionState.
Two aggravating details found on the way:
- Recoverable state is thrown away.
writeFileAtomic leaves run-state.json.<pid>.<uuid>.tmp behind when the process dies between write and rename — that temp is usually a complete generation of the state. The loader ignores it.
- The loss is silent in the UI. Nothing tells the user the assistant's memory is gone; they just experience the model being broken.
Suggested fix (all verified in my PR)
- fsync the temp file before the rename in both
writeFileAtomic and writeFileAtomicAsync (rename becomes durable and its data with it).
- Recovery order on unreadable primary: rotated
run-state.json.bak (previous complete generation, rotated aside by each synchronous save) → newest complete checkpoint .tmp → give up. Self-heal the primary from whichever recovered.
- Surface the loss:
loadMostRecentChatState reports whether agent context survived, and the resume flow prepends an error-variant notice ("agent context could not be restored; the assistant starts without memory of earlier turns; transcript is intact") instead of failing silently.
PR to follow.
Follow-up from #1166 (the compaction wipes). While auditing that report I traced the persistence side and found a second, independent path to the same symptom — "the model lost all chat history" — with a different cause. Verified against the current code with executable repros.
The failure chain
The atomic write is not durable.
cli/src/utils/write-file-atomic.tswrites a temp file and renames it overrun-state.json— but never fsyncs. The rename is atomic against concurrent processes, yet after a power loss / hard hang / battery pull, the filesystem can land the rename while the file's data blocks were never written. Result: a truncated or emptyrun-state.jsonexactly where the atomic rename was supposed to guarantee a complete file. (The in-code comment acknowledges torn non-atomic writes; the same tear reaches through this pattern without the fsync.)The load path has no recovery.
loadMostRecentChatState(cli/src/utils/run-state-storage.ts) parsesrun-state.jsonand, on failure, falls back to a placeholder:The placeholder has no
sessionState. Meanwhilechat-messages.jsonis parsed independently and usually survives — so the UI transcript is fully intact.A missing
sessionStatemeans a fresh session.sdk/src/run.ts:436-467: whenpreviousRun.sessionStateis undefined,runOncebuilds a brand-newinitialSessionState. The next turn's model context = empty history.Net effect: the user resumes a chat, sees their whole transcript, sends a message — and the assistant has no idea who they are or what it did. That is precisely the "lost all chat history instead of compacting" report shape, on machines that crashed or lost power mid-session. Reproduced with a temp-dir test: a truncated primary + intact transcript yields a placeholder RunState without
sessionState.Two aggravating details found on the way:
writeFileAtomicleavesrun-state.json.<pid>.<uuid>.tmpbehind when the process dies between write and rename — that temp is usually a complete generation of the state. The loader ignores it.Suggested fix (all verified in my PR)
writeFileAtomicandwriteFileAtomicAsync(rename becomes durable and its data with it).run-state.json.bak(previous complete generation, rotated aside by each synchronous save) → newest complete checkpoint.tmp→ give up. Self-heal the primary from whichever recovered.loadMostRecentChatStatereports whether agent context survived, and the resume flow prepends an error-variant notice ("agent context could not be restored; the assistant starts without memory of earlier turns; transcript is intact") instead of failing silently.PR to follow.