feat(agents): add sampleRate option to OpenAI.TTS - #2448
Conversation
🦋 Changeset detectedLatest commit: d737a2d The changes in this PR will be included in the next version bump. This PR includes changesets to release 38 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Devin Review found 3 potential issues.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| const buffer = await this.stream.then((r) => r.arrayBuffer()); | ||
| const requestId = shortuuid(); | ||
| const audioByteStream = new AudioByteStream(OPENAI_TTS_SAMPLE_RATE, OPENAI_TTS_CHANNELS); | ||
| const audioByteStream = new AudioByteStream(this.sampleRate, OPENAI_TTS_CHANNELS); |
There was a problem hiding this comment.
🔴 Invalid sample rates freeze synthesis
With sampleRate zero or negative, synthesis enters an endless loop after receiving audio. AudioByteStream.write cannot consume non-positive frame sizes, blocking the Node.js event loop.
Prompt for agents
Validate the new sampleRate constructor option before it reaches the base TTS class or AudioByteStream. Accept only finite positive sample rates, preferably positive integers, and throw a clear configuration error otherwise. Ensure the validated or defaulted value is used consistently for both TTS metadata and PCM framing in plugins/openai/src/tts.ts.
Was this helpful? React with 👍 or 👎 to provide feedback.
| }, | ||
| { signal }, | ||
| ), | ||
| this.#opts.sampleRate!, |
There was a problem hiding this comment.
🟡 Undefined sample rate silences output
With explicit sampleRate: undefined, construction advertises 24 kHz while #opts retains undefined. Each synthesis then emits no audio frames.
| this.#opts.sampleRate!, | |
| this.sampleRate, |
Was this helpful? React with 👍 or 👎 to provide feedback.
| const buffer = await this.stream.then((r) => r.arrayBuffer()); | ||
| const requestId = shortuuid(); | ||
| const audioByteStream = new AudioByteStream(OPENAI_TTS_SAMPLE_RATE, OPENAI_TTS_CHANNELS); | ||
| const audioByteStream = new AudioByteStream(this.sampleRate, OPENAI_TTS_CHANNELS); |
There was a problem hiding this comment.
🟡 Final speech audio is discarded
Responses outside exact 100 ms multiples lose their final partial frame. AudioByteStream.write retains that audio, but this stream closes without flushing it.
Prompt for agents
In ChunkedStream.run in plugins/openai/src/tts.ts, include AudioByteStream.flush() output after write(buffer), then feed both complete and final partial frames through the existing last-frame logic. Preserve exactly one final=true event for the final emitted frame.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
|
Add sampleRate option for OpenAI.TTS in agents plugin.
3 participants
Description
Currently, the
OpenAI.TTSclass hardcodes a sample rate of 24000 hz.Some models, such as FishAudio S2 Pro, are using a different one, such as 44100 hz.
This PR adds the ability to override the sample rate as part of the constructor, which falls back to 24000 hz as before if not supplied.
Pre-Review Checklist
Testing
restaurant_agent.tsandrealtime_agent.tswork properly (for major changes)Additional Notes
Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.
Please tell me what I can do to get this merged asap!