Add voice agent WebSocket clients, tests, and samples - #50519
Merged
Merged
Conversation
Mike Guo (guotuofeng)
requested review from
a team and
Ryan Hurey (rhurey)
as code owners
September 18, 2026 11:37
3 tasks
|
Azure Pipelines: Successfully started running 1 pipeline(s). 35 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Mike Guo (guotuofeng)
added this pull request to stack #50521
September 18, 2026 11:40
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The async transport can ignore WebSocket ping frames, while README examples do not compile and a live test assumes asynchronous call dispatch is immediate.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds preview synchronous/asynchronous voice-agent WebSocket clients, realtime event handling, telephony/conversation coverage, and voice-agent samples.
Changes:
- Adds authenticated WebSocket sessions with audio/text input, bounded queues, raw events, cancellation, timeouts, and secure URL validation.
- Adds lifecycle, realtime, conversation, telephony, serialization, and live tests.
- Adds voice-agent samples, module dependencies, README guidance, and changelog updates.
File summaries
| File | Description |
|---|---|
VoiceAgentTelephonyLiveTests.java |
Live Twilio lifecycle and call validation. |
VoiceAgentRealtimeLiveTests.java |
Live and synthetic realtime scenarios. |
VoiceAgentLiveAudioConversationAsyncSampleTests.java |
Audio sample lifecycle tests. |
VoiceAgentCrudTests.java |
Synchronous CRUD coverage. |
VoiceAgentCrudAsyncTests.java |
Asynchronous CRUD coverage. |
VoiceAgentConversationsTests.java |
Persisted conversation tests. |
VoiceAgentConversationsAsyncTests.java |
Async persisted conversation tests. |
VoiceAgentDefinitionSerializationTests.java |
Voice definition round-trip tests. |
VoiceAgentWithToolsSample.java |
Voice tools configuration sample. |
VoiceAgentVersionsSample.java |
Version and draft sample. |
VoiceAgentSampleUtils.java |
Shared sample definition helper. |
VoiceAgentRealtimeSampleUtils.java |
Realtime playback and conversation helpers. |
VoiceAgentReadConversationSample.java |
Conversation reading sample. |
VoiceAgentReadConversationAudioSample.java |
Conversation audio download sample. |
VoiceAgentLiveTextConversationSample.java |
Synchronous realtime text sample. |
VoiceAgentLiveTextConversationAsyncSample.java |
Async realtime text sample. |
VoiceAgentLiveFunctionToolSample.java |
Function-tool execution sample. |
VoiceAgentLiveAudioConversationAsyncSample.java |
Async microphone/audio sample. |
VoiceAgentGenerateSample.java |
Guided generation sample. |
VoiceAgentBasicSample.java |
Basic synchronous lifecycle sample. |
VoiceAgentBasicAsyncSample.java |
Basic asynchronous lifecycle sample. |
ReadmeSamples.java |
README snippet synchronization. |
module-info.java |
WebSocket transport module requirements. |
VoiceAgentWebSocketOverflowStrategy.java |
Queue overflow policies. |
VoiceAgentWebSocketConnectionOptions.java |
WebSocket connection options. |
RawRealtimeServerEvent.java |
Unknown event preservation. |
VoiceAgentWebSocketUtils.java |
URI, header, event, and close-frame utilities. |
VoiceAgentWebSocketHttpResponse.java |
Handshake response adaptation. |
VoiceAgentWebSocketHandshakeHandler.java |
Rejected handshake handling. |
VoiceAgentWebSocketClientConfiguration.java |
Internal WebSocket configuration. |
BetaVoiceAgentWebSocketSessionClient.java |
Synchronous WebSocket session. |
BetaVoiceAgentWebSocketSessionAsyncClient.java |
Asynchronous WebSocket session. |
BetaVoiceAgentWebSocketClient.java |
Synchronous WebSocket client. |
BetaVoiceAgentWebSocketAsyncClient.java |
Asynchronous WebSocket client. |
AgentsClientBuilder.java |
Beta WebSocket client builders. |
README.md |
WebSocket API and sample documentation. |
pom.xml |
Transport dependencies and module arguments. |
CHANGELOG.md |
Preview API and behavior updates. |
Review details
Suppressed comments (2)
sdk/ai/azure-ai-agents/README.md:1086
- These are also APIView display names, not Java classes. The documented audio and function-tool event types should be
RealtimeResponseAudioDeltaEventandRealtimeResponseFunctionCallArgumentsDoneEvent, otherwise readers cannot implement the described handlers.
Use `appendInputAudio` to send PCM16 chunks, `commitInputAudio` to commit buffered audio when server-side voice activity detection is not configured, and `clearInputAudio` to discard pending input. Audio output arrives through `RealtimeServerEventResponseAudioDelta` events. When a `RealtimeServerEventResponseFunctionCallArgumentsDone` event requests a client-side tool, execute the function and call `sendFunctionCallOutput` with its call ID and serialized result.
sdk/ai/azure-ai-agents/README.md:1076
- The asynchronous example repeats the same non-existent APIView display names, so this code also cannot compile for Java consumers. Replace them with
RealtimeResponseTextDeltaEventandRealtimeResponseDoneEvent.
if (event instanceof RealtimeServerEventResponseTextDelta) {
System.out.print(((RealtimeServerEventResponseTextDelta) event).getDelta());
}
})
.takeUntil(event -> event instanceof RealtimeServerEventResponseDone)
- Files reviewed: 40/40 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
Alan Zimmer (alzimmermsft)
approved these changes
Sep 18, 2026
Mike Guo (guotuofeng)
deleted the
myguo/voice-agents-websocket-split
branch
September 18, 2026 13:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Splits the voice-agent work from #50408 into a focused bottom PR.
wss://connections before token acquisition.This is the bottom PR for the Python-parity follow-up stack.
Validation