Add voice agent WebSocket transport, tests, samples, and AI Projects parity

[Mike Guo (guotuofeng)]

Description

Follow-up to the generated Projects + Agents foundation merged through #50407. This PR now targets main directly and adds handwritten voice-agent WebSocket transport, AI Projects parity improvements, tests, samples, and documentation.

Azure AI Agents

Voice agents and WebSocket transport

• Adds synchronous and asynchronous authenticated WebSocket clients and sessions through AgentsClientBuilder.beta().
• Supports typed and raw realtime events, text and PCM16 audio input, response creation/cancellation, function-call output, persisted conversations, and custom close codes/reasons.
• Adds connection options for session IDs, structured inputs, API versions, credential scopes, preview features, headers/query parameters, receive capacity, message size, overflow behavior, malformed-event recovery, timeouts, transport customization, and same-host connection URL overrides.
• Enforces secure wss:// endpoints and validates URL overrides before token acquisition.
• Adds preview persisted-conversation and telephony clients for conversation responses/items/audio, bindings, outbound calls, campaigns, recipient import, validation, publication, pause/resume, and cancellation.

Builders, preview features, and OpenAI integration

• Beta clients automatically add their area-specific Foundry-Features values; non-beta preview APIs remain controlled by allowPreview(true).
• Agent-scoped OpenAI clients automatically include agent preview features and use an overridable API version.
• Adds native OpenAI factory callbacks for URL, credentials, headers, query parameters, and transport overrides while preserving custom Azure pipelines and authentication policies.
• Retrieves Azure tokens asynchronously for native async OpenAI clients and ResponsesAsyncClient.
• Adds Java opt-in guidance to 403 preview_feature_required errors while preserving service response details.
• Preserves explicit empty preview headers, custom credentials, and user-agent overrides.
• Replaces preview generateAgent methods with createAgentFromPrompt methods on beta Agents clients.

Polling, uploads, and logging

• Adds GET-only resumption for memory updates and optimization jobs without creating replacement jobs.
• Maps service statuses including completed, succeeded, superseded, and cancelled to Azure Core polling states.
• Supplies empty operations and zero usage when a completed memory update omits or nulls its final result.
• Streams replayable code-upload content while hashing and rejects paths without a filename explicitly.
• Logs SSE chunks as consumed with incremental UTF-8 decoding, while omitting multipart request and response bodies.
• Adds public StreamingResponseUtils adapters for OpenAI streaming responses.

Samples and tests

• Adds synchronous/asynchronous voice lifecycle, persisted-conversation, live text, live audio, barge-in, and client function-tool samples.
• Adds WebSocket protocol, buffering, malformed input, message limit, close, timeout, and sync/async parity tests.
• Adds voice CRUD/conversation, telephony, polling, serialization, HTTP bridge, preview-header, and sample-flow coverage.

Azure AI Projects

File and model uploads

• Adds configurable sync/async dataset file and folder upload helpers, including response variants.
• Adds local model file/folder upload, Blob Storage transfer, model registration, and optional availability polling.
• Adds filename filtering, metadata, request conditions, headers, transfer settings, connection selection, polling intervals, and timeout options.
• Validates files and selections before requesting storage, prevents registration after upload failures, and accepts both pending-upload response shapes.
• Uses Reactor-managed file streams to release handles deterministically, including on Windows and cancellation paths.

Projects APIs and parity

• Adds synchronous and asynchronous telemetry clients backed by project Application Insights connections.
• Adds evaluation helpers that convert Azure evaluation models to native OpenAI evaluation types.
• Adds GET-only resumption for data-generation jobs, evaluator-generation jobs, and agent-insight runs.
• Applies the same preview-header, OpenAI options, async authentication, custom pipeline, logging, and error-guidance behavior as Agents.
• Adds upload, polling, telemetry, evaluation, preview-header, multipart logging, and async authentication tests and samples.

Compatibility and Security

• Prevents credential transmission over insecure WebSocket connections.
• Preserves caller-provided credentials, headers, user agents, transports, and Azure pipelines.
• Avoids emitting multipart upload bodies through normal HTTP body logging.
• Preserves UTF-8 characters split across streamed response chunks.
• Avoids blocking credential suppliers in asynchronous OpenAI paths.
• Stabilizes Windows file cleanup, macOS handshake timing, and Reactor cancellation tests.

PR History

• Regenerate Azure AI Projects and Agents clients #50407, containing the combined Projects + generated Agents foundation, has merged into main.
• Add voice agent WebSocket transport, tests, samples, and AI Projects parity #50408 now targets main directly and has no active lower-PR dependency.
• Former middle PR Regenerate Azure AI Agents client #50412 was absorbed into Regenerate Azure AI Projects and Agents clients #50407 and is recorded as merged.

Current TypeSpec pins: Projects c12a439255c265d647357b4762e095656dda91c0; Agents 2ba065c423a4c08ddb4e517a9f16deb17cb378c2.

Validation

Combined foundation validation:

• Projects: 103 tests total, 79 passed, 24 skipped, zero failures/errors.
• Agents: 320 tests total, 300 passed, 20 skipped, zero failures/errors.
• Both modules passed Checkstyle with zero violations and strict RevApi against 2.5.0.
• Main, test, sample, Javadoc, and Java 8 compilation completed as part of both clean package builds.
• The PowerShell 7 pipeline-equivalent Verify TypeSpec Code Generation task passed for both modules with exit code 0.
• README code-snippet synchronization passed for both modules.

Core validation before the latest foundation update:

• Fresh detached-worktree Agents package: 476 tests total, 448 passed, 28 skipped, zero failures/errors.
• Checkstyle passed with zero violations and strict RevApi against 2.5.0 completed without failures.

Telephony validation:

• Offline VoiceAgentTelephonyTests: 14 passed, covering sync/async request paths, status codes, typed responses/errors, ETags, idempotency keys, and call-job revisions.
• The main Twilio live flow passed, placing a real call and validating binding creation, listing, retrieval, update, and deletion; transfer targets and live call transfer; inbound call list/get/end; immediate and scheduled outbound call jobs; and cancellation.
• Campaign APIs remain excluded because they are currently unsupported.

Known Limits

• Campaign APIs are currently unsupported and remain excluded from live validation.

These are dated local validation results, not a claim that current CI or merge requirements are complete. See the PR checks for current status.

Contribution Checklist

• Changelog, samples, and relevant documentation updated.
• Behavioral tests included; validation dates and limitations are documented.
• Contribution guidelines reviewed.

[Copilot]

🟡 Changes recommended

Unresolved critical and moderate findings remain in WebSocket behavior and model serialization.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds generated voice/realtime agent models and authenticated synchronous/asynchronous WS/WSS sessions to azure-ai-agents.

Changes:

• Regenerates voice, realtime, telephony, and agent model surfaces.
• Adds WebSocket authentication, proxying, typed events, audio, tools, cancellation, and close handling.
• Updates TypeSpec metadata, module dependencies, vocabulary, and release metadata.

Unresolved findings remain in connection validation and cancellation, bounded close buffering, handshake error preservation, and PromptAgentDefinition serialization.

File summaries

File	Summary
sdk/ai/cspell.yml	Adds domain vocabulary.
sdk/ai/azure-ai-agents/tsp-location.yaml	Updates TypeSpec source metadata.
sdk/ai/azure-ai-agents/src/main/java/module-info.java	Updates module dependencies.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceType.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceResponseBaseStatus.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceResponseBaseOutputModality.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceResponseBaseObject1.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceResponseBaseObject.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceResponseAudio.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceOutputModality.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceModelType.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceConversationStatus.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAudioRole.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAudioContainerFormat.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAudioCodec.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentTurnDetectionType.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentTransport.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentToolResponseScheduling.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentSystemToolName.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentSubagentAbortReason.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentSessionIncludeOption.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentSemanticVadTurnDetectionEagerness.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentResponseCreateParamsConversation.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentNoiseReductionType.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentInterimResponseTrigger.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentFunctionToolType.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentEndOfUtteranceThresholdLevel.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentEchoCancellationReferenceSource.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentAvatarType.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentAvatarOutputProtocol.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentAudioTimestampType.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentAudioInputConfigTranscriptionDelay.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/VoiceAgentAnimationOutputType.java	Generated voice model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TranscriptionLanguage.java	Generated transcription model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/ToolType.java	Generated tool model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/ToolConfig.java	Generated tool model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/ToolChoiceOptions.java	Generated tool model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/ToolboxVersions.java	Generated toolbox model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/ToolboxToolType.java	Generated toolbox model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/ToolboxTool.java	Generated toolbox model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/Tool.java	Generated tool model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyTransferDestinationKind.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyProvider.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyOutboundRetryPolicyType.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyOutboundDestinationType.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyOperationStatus.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCampaignScheduleType.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCampaignRecipientImportStatus.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCampaignRecipientImportFormat.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCampaignDuplicateHandling.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCampaignConfigurationStatus.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCallTraceStatus.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCallTraceMode.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCallTimestampSource.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCallStatus.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCallLifecycleEventSource.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCallLifecycleEventOutcome.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyCallDurationBasis.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/TelephonyBindingStatus.java	Generated telephony model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/SessionAffinitySource.java	Generated session-affinity model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/SessionAffinityRequestMode.java	Generated session-affinity model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/SessionAffinityMode.java	Generated session-affinity model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/SessionAffinityDecision.java	Generated session-affinity model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeTurnDetectionType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeSessionCreateRequestUnionType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeServerEventResponseContentPartDonePartType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeServerEventResponseContentPartAddedPartType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeServerEventRateLimitsUpdatedRateLimitsName.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeServerEventConversationCreatedConversationObject.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeResponseStatusDetailsType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeResponseStatusDetailsReason.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeReasoningEffort.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeMcpErrorType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemObject.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemMessageUserStatus.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemMessageUserContentType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemMessageUserContentDetail.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemMessageType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemMessageSystemStatus.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemMessageSystemContentType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemMessageAssistantStatus.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemMessageAssistantContentType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemFunctionCallStatus.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeConversationItemFunctionCallOutputStatus.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeClientEventSessionUpdateSessionTruncation.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeClientEventSessionUpdateSessionOutputModality.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeAudioFormatsType.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RealtimeAudioFormatsAudioPcmRate.java	Generated realtime model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RaiInvocationMode.java	Generated responsible-AI model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/RaiInvocationContentType.java	Generated responsible-AI model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/PublishTelephonyCampaignRequest.java	Generated telephony request model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/PromptCacheBreakpointConfig.java	Generated prompt model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/PromptAgentDefinition.java	Prompt agent model; serialization finding remains.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/NoiseReductionType.java	Generated audio model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/ModelRouterDetails.java	Generated agent model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/McpListToolsToolInputSchema.java	Generated MCP model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/McpListToolsToolAnnotations.java	Generated MCP model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/ImageDetail.java	Generated content model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/GitHubCopilotHarness.java	Generated GitHub Copilot model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/GitHubCopilotBuiltInTool.java	Generated GitHub Copilot model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/FileInputDetail.java	Generated content model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/CreateTranscriptionResponseJsonUsageType.java	Generated transcription model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/AzureCreateResponseOptions.java	Generated agent model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/AgentOptimizationJobProgress.java	Generated agent model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/AgentKind.java	Generated agent model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/AgentEndpointProtocol.java	Generated agent model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/AgentDetails.java	Generated agent model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/models/AgentDefinition.java	Generated agent model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/SyncOperationLocationPollingStrategy.java	Polling customization.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/realtime/VoiceAgentWebSocketHandshakeHandler.java	WebSocket handshake handling.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/OperationLocationPollingStrategy.java	Polling customization.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/models/TransferTelephonyCallRequest.java	Generated implementation model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/models/AgentDefinitionOptInKeys.java	Generated implementation model.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/JsonMergePatchHelper.java	JSON merge-patch helper.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/AgentsClientImpl.java	Generated client implementation.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/BetaVoiceAgentWebSocketClient.java	Synchronous WebSocket entry point; validation finding remains.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/BetaVoiceAgentWebSocketAsyncClient.java	Asynchronous WebSocket entry point.
sdk/ai/azure-ai-agents/CHANGELOG.md	Documents the feature.
sdk/ai/azure-ai-agents/assets.json	Updates package metadata.

Review details

Suppressed comments (4)

sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/VoiceAgentWebSocketSessionAsyncClient.java:404

• Disabling automatic ping handling leaves ping frames to the receive path, but handleFrame only processes text and binary frames and never sends a Pong. A server keepalive ping can therefore go unanswered and cause the session to be closed. Keep automatic ping handling enabled or explicitly process PingWebSocketFrame messages.
  sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/VoiceAgentWebSocketSessionClient.java:412
• The OkHttp transport configures a fixed proxy but ignores ProxyOptions.getNonProxyHosts(), whereas the Reactor Netty path honors that setting. Synchronous sessions therefore route hosts configured to bypass the proxy through it, which can break internal endpoints and makes the two clients behave differently. Configure the OkHttp proxy selector to honor the bypass pattern.
  sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/VoiceAgentWebSocketSessionClient.java:92
• The synchronous constructor accepts an empty agent name and builds /agents//endpoint/protocols/voice, whereas the async constructor rejects empty names. connect("") should fail before issuing this malformed service request so both public clients have the same validation.
  sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/realtime/VoiceAgentWebSocketHttpResponse.java:37
• Netty can expose multiple header entries with the same name, but HttpHeaders.set overwrites the previous value on each iteration. A rejected handshake with repeated headers (for example, authentication challenges) is therefore reported with only its last value, unlike the OkHttp adapter. Append each entry instead of replacing the header.

• Files reviewed: 74/418 changed files
• Comments generated: 5
• Review effort level: Lite

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[azure-pipelines]

Azure Pipelines:
Successfully started running 278 pipeline(s).
36 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[azure-pipelines]

Azure Pipelines:
Successfully started running 278 pipeline(s).
36 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Kaylie (kaylieee)]

Scope: this PR contains a substantial set of changes that are unrelated to the websocket/voice agent functionality. Examples include:

• Additional models eg. AzureAIEvaluationDataSource, FileUploadOptions, ModelUploadOptions
• Addition of handwritten methods across multiple clients
• Addition of telemetryclient
• Modifications to polling strategy
• azure-ai-projects builder configuration updates

I have not listed every affected file. Can you please clarify why these changes are required for the websocket implementation? Unless they are direct dependencies, please revert them here so this PR remains focused.

[Mike Guo (guotuofeng)]

Thanks for calling this out. The listed changes are not direct dependencies of the voice-agent WebSocket implementation. They came from a broader Java/Python SDK parity audit that was consolidated into this PR:

• AzureAIEvaluationDataSource and the evaluation factories cover Python evaluation data-source parity.
• FileUploadOptions, ModelUploadOptions, and the handwritten upload helpers/overloads cover Python dataset and model upload workflows.
• BetaTelemetryClient / BetaTelemetryAsyncClient cover the corresponding project telemetry operations.
• The polling changes cover Python parity for resuming memory, optimization, evaluation, and related long-running jobs, including service-specific terminal states and omitted results.
• The Agents/Projects builder and HTTP changes cover preview opt-in guidance, header/pipeline preservation, multipart logging behavior, and asynchronous Azure token acquisition found during the same parity audit.

The PR title and description were broadened when that parity work was consolidated, but I agree these changes are separable from WebSocket/voice functionality. Would you prefer that I split the non-voice parity work into a separate stacked PR and keep #50408 focused on voice agents and their direct dependencies? I can do that before further review.

[Kaylie (kaylieee)]

Would you prefer that I split the non-voice parity work into a separate stacked PR and keep #50408 focused on voice agents and their direct dependencies? I can do that before further review.

Thanks for clarifying. I don't think we should carry the unrelated parity changes forward as a stacked PR from this work. Since they aren't direct dependencies of the websocket implementation, could we revert them here and revisit them independently if needed?

[Jose Alvarez (jpalvarezl)]

Thanks for removing the public OkHttp/Reactor Netty callbacks in eec49fe. That addresses the transport-type leak and also removes the path by which a caller could supply shared OkHttp resources that the session would subsequently shut down.

I still have a few concerns about the WebSocket API/configuration boundary:

1. Reuse the existing client configuration machinery rather than creating a parallel builder path. In createVoiceAgentWebSocketConfiguration, we reconstruct configuration/User-Agent/header handling, but the Azure HTTP client, pipeline, and custom policies configured on that same builder are not carried over. For example, a policy-added handshake header or TLS configuration on the supplied HTTP client will not apply. Please avoid a separate/custom builder unless there is a demonstrated requirement that the existing machinery cannot support. Endpoint configuration and User-Agent/header construction should primarily be a plumbing/reuse job, with only the necessary WebSocket-specific adaptation. I am not suggesting that an arbitrary Azure HTTP transport automatically supports WebSocket upgrades; where settings genuinely cannot apply, make that limitation explicit rather than silently accepting them.

   sdk/openai/azure-ai-openai-realtime is a useful reference for the encapsulation boundary: the public client privately owns its WebSocket client/session, and endpoint/transport implementation types stay internal. Please use that separation as guidance, rather than copying its lifecycle implementation wholesale.

2. Can we justify and narrow the remaining public connection overrides? VoiceAgentWebSocketConnectionOptions exposes a full connection URL, an arbitrary API-version string, the preview-feature header, credential scopes, overriding query parameters, and a transport selector on a WebSocket-specific API. Several duplicate builder configuration or let callers replace SDK-managed protocol details. Please identify the customer scenario that requires each of these and remove the unnecessary ones before they become a compatibility commitment. Legitimate endpoint/audience configuration is different from allowing arbitrary per-session overrides.

   Typed send/receive operations, explicit close, agent/session options, and raw BinaryData events for forward compatibility are reasonable public APIs. Buffering/timeouts can also be justified. The distinction is customer-facing behavior versus exposing transport/configuration internals, not simply making every helper non-public.

3. The synchronous receive API does not enforce its single-iterator contract. receiveEvents(Duration) claims the receiver when returning the IterableStream, but its factory creates a new EventIterator every time .iterator() is called. Two iterators from the same returned stream therefore consume the same queue, split events, and can leave one waiting indefinitely after the other consumes the terminal signal. Please enforce the single-consumer claim at iterator creation as well, and cover calling .iterator() twice on the same stream.

[Jose Alvarez (jpalvarezl)]

Thanks for clarifying the parity work's origin. I agree with Kaylie's request to remove the unrelated changes here and revisit them independently if needed, rather than carrying them forward as a stacked dependency of this work.

At eec49fe, these separable groups are still in the diff:

• Projects evaluation data-source models and conversion factories.
• Dataset/model upload options, helpers, filtering, and registration workflows.
• Projects telemetry clients.
• Memory, optimization, and Projects job polling/resumption parity.
• General OpenAI factory customization, asynchronous authentication, preview-error guidance, header handling, and multipart/SSE logging changes.
• Code-upload hashing and filename-validation changes.

Let's keep #50408 focused on VoiceAgents functionality, the WebSocket transport and wiring it actually needs, and the corresponding documentation, samples, and tests. The listed parity entry points are not called by the voice samples/tests, and the WebSocket configuration path does not depend on the new OpenAI factory path. If a particular shared-infrastructure change really is a prerequisite, please identify the concrete dependency and retain only that necessary hunk.

Please move/remove the associated non-voice tests, documentation, changelog entries, and generation customizations together with the implementation changes, and narrow the PR title/description accordingly. This is about separating reviewable changes, not deleting existing shared infrastructure or dismissing potentially useful fixes.

[Jose Alvarez (jpalvarezl)]

One polling follow-up for when this work is revisited independently: GET-only resumption from a saved job ID is a reasonable requirement, and these helpers do use Azure Core's PollerFlux/SyncPoller lifecycle. Service-status aliases and nested result extraction can be handled by narrow adapters; I am not treating this as a wholesale replacement of core polling.

There is, however, a concrete issue in async activation in both ProjectsServicePollUtils and AgentsServicePollUtils.

The GET callback already constructs a complete PollResponse, including the status and any parsed Retry-After. Passing activation through map(PollResponse::getValue) discards both: the selected PollerFlux constructor replaces them with NOT_STARTED and no retry delay. An already-completed job therefore waits and issues another GET unnecessarily, and an initial service Retry-After is ignored.

Please use the existing PollerFlux.create factory with the full-response GET callback as activation as well as polling. That preserves the initial response; the final-result callbacks here already deserialize the cached body and do not need another request. Please add coverage for a successful completed first GET requiring no second GET, and for honoring Retry-After on an in-progress first GET.

[Jose Alvarez (jpalvarezl)]

Following up specifically on the customizations changes: the current diff goes substantially beyond the small generated-annotation adjustments discussed in Alan's earlier review.

Both AgentsCustomizations.customizeBuilder and ProjectsCustomizations.customizeBuilder now:

• Clone buildInnerClient into a preview-aware construction path.
• Replace the generated logging-options initialization and HttpLoggingPolicy construction.
• Clone the generated HTTP pipeline into an OpenAI-specific pipeline and remove its bearer-token policy, as part of the separate OpenAI authentication path.

Agents additionally rewrites both polling strategies' getResult methods to use the memory-update missing-result fallback. The existing status-remapping customization predates this PR; it is not the new change I am questioning.

I understand the engineering purpose: these transformations make the corresponding behavior survive regeneration, and cloning generated methods avoids manually freezing their constructor/pipeline details. But that explains why the customizations accompany those changes, not why the changes belong in a VoiceAgents PR. The WebSocket construction path does not call these cloned inner-client/OpenAI pipeline methods, and preview-aware REST client construction already existed at the base of this PR.

Please remove these new AST transformations together with the unrelated logging/OpenAI/preview-error and memory-polling changes, unless there is a concrete VoiceAgents requirement for a particular transformation. Preserve existing customizations and any minimal generated metadata adjustment genuinely needed for voice clients. For ordinary handwritten builder additions, use the existing partial-update mechanism rather than adding AST injection; generated-member changes may still legitimately need customization.

This should not be fixed by deleting the customizations while leaving their generated effects behind, or by copying the entire generated pipeline into handwritten code. Please keep the retained source changes and their regeneration inputs consistent. The narrowly scoped voice feature should not require maintaining two new copies of template-sensitive pipeline-rewriting logic.

[Mike Guo (guotuofeng)]

Split the work into a new two-PR stack as requested:

• Add voice agent WebSocket clients, tests, and samples #50519: voice-agent WebSocket clients, voice tests, and samples only (base: main).
• Add AI Agents and Projects Python SDK parity #50520: AI Agents and Projects Python SDK parity only (base: Add voice agent WebSocket clients, tests, and samples #50519).

The original myguo/voice-agents-core branch and #50408 remain unchanged. Both new layers were independently scope-audited and validated before publication.