[MikeAlhayek]

Fixes #273

A cascaded realtime deployment spoke at its speaking leg's default pace and nothing could change it. With gpt-4o-mini-tts, callers found that pace slow. This follows the issue's proposal; the default pace lives on the cascade's metadata.

Changes

• CascadedRealtimeMetadata gets two optional settings:
  • SpeechSpeed: the cascade's pace, as a multiple of the speaking leg's normal pace (1.2 is 20% faster).
  • SpeakingStyle: how to speak, in words ("Speak at a brisk, natural phone pace…").
• CascadedRealtimeClient passes both with every fragment: the speed as TextToSpeechOptions.Speed, and the style as the new TextToSpeechOptionsProperties.Instructions additional property, since TextToSpeechOptions has no field for it. DefaultAIClientFactory sets them from the metadata.
• The listener's speed control works on a cascade.
  • DefaultRealtimeOrchestrator no longer turns SupportsSpeechSpeed off for a cascade.
  • DefaultRealtimeConversation.UpdateSpeechSpeedAsync hands the speed to the cascaded session instead of sending a session.update to its transcription leg.
  • The session applies it on top of the configured pace from the next fragment: at 1.2, a listener's 1.25× speaks at 1.5.
• OpenAI (new OpenAITextToSpeechStyleClient, wrapped inside the existing format client):
  • The style goes to the model as instructions, through the SDK's SpeechGenerationOptions (RawRepresentationFactory). It is left out for tts-1 and tts-1-hd, which take no instructions.
  • The speed is kept within the 0.25 to 4 OpenAI accepts.
• Azure Speech honors TextToSpeechOptions.Speed, which it ignored before. A speed other than 1 is spoken as SSML with prosody rate, clamped to the 0.5 to 2 Azure accepts. The voice is named in its own locale, and the text is XML-escaped. Without a voice to name, the plain-text path is kept. Azure has no free-text style, so SpeakingStyle does not apply to it.
• Docs:
  • core/realtime-voice.md: a "Pace and speaking style" section, and the speaking-speed control on a cascade.
  • providers/openai.md: speed and instructions.
  • A changelog entry.

Tests

• CascadedRealtimeSessionTests:
  • every fragment is spoken at the configured pace and style;
  • without them, the speaking leg keeps its own;
  • a listener's speed set through DefaultRealtimeConversation multiplies the configured pace (4 cases) and sends nothing to the transcription leg.
• OpenAITextToSpeechTests: through a loopback stand-in for OpenAI, the request carries speed (clamped to 0.25 and 4) and instructions for gpt-4o-mini-tts, but no instructions for tts-1 and tts-1-hd.
• AzureSpeechServiceSpeedTests: the SSML's prosody rate (including clamping), the voice and locale, escaping, and when the plain text is kept.
• The full test project passes locally (4777).

Verified live

• Azure Speech: the Azure Speech deployment in the sample host's config, voice en-US-JennyNeural, raw PCM at 24 kHz. One sentence took 7.15 s at the normal pace, 5.50 s at 1.3× and 8.94 s at 0.8×, which match 7.15/1.3 and 7.15/0.8 exactly. Streamed and single-shot synthesis gave the same result.
• OpenAI: not run live, since there is no OpenAI key locally. The loopback test checks the request body the SDK sends.

Note

providers/openai.md and OpenAIClientProvider are next to lines that #281 (OpenAI PCM resampling for #279) also changes. Whichever merges second may need a small rebase.

🤖 Generated with Claude Code