fix(voice): commit interrupted speech before snapshotting the next turn - #2438
fix(voice): commit interrupted speech before snapshotting the next turn#2438chasef07 wants to merge 1 commit into
Conversation
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if ( | ||
| this.schedulingPaused || | ||
| this.newTurnsBlocked || | ||
| (!(this.llm instanceof RealtimeModel) && this.agentSession._closing) | ||
| ) { |
There was a problem hiding this comment.
🟡 Shutdown can schedule a late reply
If shutdown starts during onUserTurnCompleted, the earlier _closing guard has already passed. Scheduling stays active until drain finishes onExit, so the callback can create another reply. It can play during shutdown and delay closure.
Prompt for agents
The new closing guard only checks AgentSession._closing before Agent.onUserTurnCompleted runs. Shutdown can begin while that callback is pending. AgentActivity.drain does not set schedulingPaused until onExit finishes, so the existing post-callback schedulingPaused/newTurnsBlocked guard can pass and userTurnCompleted can call generateReply after shutdown began. Recheck AgentSession._closing after onUserTurnCompleted, alongside the second scheduling guard, and preserve the intended pipeline-only behavior for RealtimeModel sessions. Add a regression test that blocks onUserTurnCompleted, starts session.close(), then releases the callback and verifies no second LLM request is created.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Description
Interrupted pipeline speech can appear in final conversation history while being absent from the very next LLM request.
userTurnCompleted()interrupts the current speech and copiesagent.chatCtxbefore the interrupted pipeline has committed the spoken prefix. The next reply therefore uses stale context, even though inspecting history after the call looks correct.This waits for the interrupted, authorized pipeline generation to complete before taking the next context snapshot. That boundary follows speech commitment and precedes settlement of interrupted tool executions. A shutdown check prevents the resumed turn from scheduling a new reply during session close while retaining the caller's message.
Changes Made
AgentSession,FakeLLM, synthetic audio frames, and an output sink that controls playback completion. Capture each actual LLM input independently of final history.Pre-Review Checklist
Testing
restaurant_agent.tsandrealtime_agent.tswork properly (for major changes): interactive examples were not run; no public API or realtime behavior changes.Reproducible before/after evidence
Base: upstream
mainat64c0bf8f5f43edfcb287e21901046f0a568342f5(@livekit/agents1.8.0). Node 24.19.0, pnpm 11.13.1.The synthetic assistant generates
The appointment is available on Thursday at ten.Playback reports onlyThe appointment is availableas spoken. The caller then saysActually, a different day.Run the regression file against either runtime version:
LIVEKIT_INFERENCE_TESTS=0 pnpm test agents/src/voice/agent_activity_interrupted_context.test.tsCoverage includes interruption at end of turn and before end of turn, with preemptive generation enabled/disabled; completed-speech controls; unauthorized and never-played speech; pending tools; consecutive caller corrections; session close while cleanup is pending; and progress through the existing interruption watchdog when the output never acknowledges completion. The pending-tool test checks that the next LLM input already contains the spoken prefix while the tool result is still unresolved.
Local validation on the final diff:
LIVEKIT_INFERENCE_TESTS=0 pnpm test agents --silent: 152 files passed; 2,502 tests passed, 5 skipped. Skips are three credential-gated inference integration suites and two existing Zod cases.pnpm build: 39 tasks passed.pnpm lint: 39 tasks passed.pnpm typecheck: 33 tasks passed.pnpm format:check,pnpm throws:check, andgit diff --check: passed.pnpm -w format:writeandpnpm -w lint:fixwere run; unrelated formatter output was excluded from this PR.Live provider calls and interactive examples were not run.
Additional Notes
Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.