Skip to content

Add voice agent WebSocket clients, tests, and samples - #50519

Merged
Mike Guo (guotuofeng) merged 25 commits into
mainfrom
myguo/voice-agents-websocket-split
Sep 18, 2026
Merged

Mike Guo (guotuofeng) merged 25 commits into
mainfrom
myguo/voice-agents-websocket-split

Conversation

@guotuofeng

Copy link
Copy Markdown
Member

Description

Splits the voice-agent work from #50408 into a focused bottom PR.

  • Adds synchronous and asynchronous authenticated voice-agent WebSocket clients and sessions.
  • Supports typed and raw realtime events, text and PCM16 audio input, response cancellation, function-call output, bounded receive queues, malformed-event handling, custom close frames, and per-event receive timeouts.
  • Enforces secure same-origin wss:// connections before token acquisition.
  • Adds voice-agent lifecycle, conversations, telephony, realtime samples, and focused test coverage.
  • Keeps OkHttp and Reactor Netty as internal implementation details rather than public option types.

This is the bottom PR for the Python-parity follow-up stack.

Validation

  • Azure AI Agents package passed: 473 tests, 445 passed, 28 expected skips, 0 failures/errors.
  • Checkstyle passed with 0 violations.
  • Revapi and Javadoc passed.
  • Java 8/25 main, test, and sample compilation passed.
  • Git whitespace checks passed.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
35 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@guotuofeng
Mike Guo (guotuofeng) added this pull request to stack #50521 September 18, 2026 11:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The async transport can ignore WebSocket ping frames, while README examples do not compile and a live test assumes asynchronous call dispatch is immediate.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds preview synchronous/asynchronous voice-agent WebSocket clients, realtime event handling, telephony/conversation coverage, and voice-agent samples.

Changes:

  • Adds authenticated WebSocket sessions with audio/text input, bounded queues, raw events, cancellation, timeouts, and secure URL validation.
  • Adds lifecycle, realtime, conversation, telephony, serialization, and live tests.
  • Adds voice-agent samples, module dependencies, README guidance, and changelog updates.
File summaries
File Description
VoiceAgentTelephonyLiveTests.java Live Twilio lifecycle and call validation.
VoiceAgentRealtimeLiveTests.java Live and synthetic realtime scenarios.
VoiceAgentLiveAudioConversationAsyncSampleTests.java Audio sample lifecycle tests.
VoiceAgentCrudTests.java Synchronous CRUD coverage.
VoiceAgentCrudAsyncTests.java Asynchronous CRUD coverage.
VoiceAgentConversationsTests.java Persisted conversation tests.
VoiceAgentConversationsAsyncTests.java Async persisted conversation tests.
VoiceAgentDefinitionSerializationTests.java Voice definition round-trip tests.
VoiceAgentWithToolsSample.java Voice tools configuration sample.
VoiceAgentVersionsSample.java Version and draft sample.
VoiceAgentSampleUtils.java Shared sample definition helper.
VoiceAgentRealtimeSampleUtils.java Realtime playback and conversation helpers.
VoiceAgentReadConversationSample.java Conversation reading sample.
VoiceAgentReadConversationAudioSample.java Conversation audio download sample.
VoiceAgentLiveTextConversationSample.java Synchronous realtime text sample.
VoiceAgentLiveTextConversationAsyncSample.java Async realtime text sample.
VoiceAgentLiveFunctionToolSample.java Function-tool execution sample.
VoiceAgentLiveAudioConversationAsyncSample.java Async microphone/audio sample.
VoiceAgentGenerateSample.java Guided generation sample.
VoiceAgentBasicSample.java Basic synchronous lifecycle sample.
VoiceAgentBasicAsyncSample.java Basic asynchronous lifecycle sample.
ReadmeSamples.java README snippet synchronization.
module-info.java WebSocket transport module requirements.
VoiceAgentWebSocketOverflowStrategy.java Queue overflow policies.
VoiceAgentWebSocketConnectionOptions.java WebSocket connection options.
RawRealtimeServerEvent.java Unknown event preservation.
VoiceAgentWebSocketUtils.java URI, header, event, and close-frame utilities.
VoiceAgentWebSocketHttpResponse.java Handshake response adaptation.
VoiceAgentWebSocketHandshakeHandler.java Rejected handshake handling.
VoiceAgentWebSocketClientConfiguration.java Internal WebSocket configuration.
BetaVoiceAgentWebSocketSessionClient.java Synchronous WebSocket session.
BetaVoiceAgentWebSocketSessionAsyncClient.java Asynchronous WebSocket session.
BetaVoiceAgentWebSocketClient.java Synchronous WebSocket client.
BetaVoiceAgentWebSocketAsyncClient.java Asynchronous WebSocket client.
AgentsClientBuilder.java Beta WebSocket client builders.
README.md WebSocket API and sample documentation.
pom.xml Transport dependencies and module arguments.
CHANGELOG.md Preview API and behavior updates.
Review details

Suppressed comments (2)

sdk/ai/azure-ai-agents/README.md:1086

  • These are also APIView display names, not Java classes. The documented audio and function-tool event types should be RealtimeResponseAudioDeltaEvent and RealtimeResponseFunctionCallArgumentsDoneEvent, otherwise readers cannot implement the described handlers.
Use `appendInputAudio` to send PCM16 chunks, `commitInputAudio` to commit buffered audio when server-side voice activity detection is not configured, and `clearInputAudio` to discard pending input. Audio output arrives through `RealtimeServerEventResponseAudioDelta` events. When a `RealtimeServerEventResponseFunctionCallArgumentsDone` event requests a client-side tool, execute the function and call `sendFunctionCallOutput` with its call ID and serialized result.

sdk/ai/azure-ai-agents/README.md:1076

  • The asynchronous example repeats the same non-existent APIView display names, so this code also cannot compile for Java consumers. Replace them with RealtimeResponseTextDeltaEvent and RealtimeResponseDoneEvent.
            if (event instanceof RealtimeServerEventResponseTextDelta) {
                System.out.print(((RealtimeServerEventResponseTextDelta) event).getDelta());
            }
        })
        .takeUntil(event -> event instanceof RealtimeServerEventResponseDone)
  • Files reviewed: 40/40 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread sdk/ai/azure-ai-agents/README.md Outdated
@guotuofeng
Mike Guo (guotuofeng) merged commit f3b703c into main Sep 18, 2026
26 checks passed
@guotuofeng
Mike Guo (guotuofeng) deleted the myguo/voice-agents-websocket-split branch September 18, 2026 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants