Skip to content

feat(speechify): add Speechify TTS plugin - #2434

Open
luke-speechify wants to merge 5 commits into
livekit:mainfrom
luke-speechify:lo/speechify-sse
Open

feat(speechify): add Speechify TTS plugin#2434
luke-speechify wants to merge 5 commits into
livekit:mainfrom
luke-speechify:lo/speechify-sse

Conversation

@luke-speechify

Copy link
Copy Markdown

Summary

Adds @livekit/agents-plugin-speechify — a Speechify text-to-speech plugin with streaming synthesis and word-level timestamps, built on Speechify's official @speechify/api SDK. It follows the existing plugin conventions and mirrors the cartesia package layout.

Features

  • Streaming TTS with word-level timestamps. Uses Speechify's /v1/audio/stream/with-timestamps endpoint (Server-Sent Events) via client.audio.streamWithTimestamps(), so audio and aligned speech marks arrive incrementally while the audio is still being generated. Declares capabilities: { streaming: true, alignedTranscript: true }.
  • SynthesizeStream chunks streamed input into sentences and streams one request per sentence; ChunkedStream handles one-shot synthesize(). Both defer final: true to the last frame of the stream and attach each request's word timestamps to a frame (speech-mark times are absolute milliseconds, converted to seconds).
  • 24 kHz mono PCM output (pcm_24000).
  • Configurable voice, model, language, and loudness/text normalization via TTSOptions. Defaults to the dominic_32 voice and the simba-3.2 model.
  • Reads the API key from the apiKey option or the SPEECHIFY_API_KEY environment variable.

Custom headers

Every request carries two custom headers so Speechify can attribute API usage to this integration per release:

  • Speechify-Caller: livekit-typescript
  • Speechify-Caller-Version: <plugin version>

They are set on the SDK client and merged into every request; the SDK sets no caller of its own, and per-request options do not clobber them.

Tests

  • Deterministic unit tests mock the SSE endpoint and cover one-shot + streaming synthesis, audio framing, word-timestamp conversion, and the attribution headers.
  • A gated end-to-end test (via @livekit/agents-plugins-test) runs when SPEECHIFY_API_KEY and OPENAI_API_KEY are set.
  • pnpm build (ESM + CJS + type declarations) and api-extractor pass.

Notes

  • New files carry SPDX/REUSE headers; public options and methods are documented for TypeDoc.
  • Added to the root README plugin table and the turbo.json env allowlist (SPEECHIFY_API_KEY).

Usage

import * as speechify from '@livekit/agents-plugin-speechify';

const tts = new speechify.TTS({ voiceId: 'dominic_32', model: 'simba-3.2' });

@luke-speechify
luke-speechify requested a review from a team as a code owner September 7, 2026 18:51
@changeset-bot

changeset-bot Bot commented Sep 7, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: faef238

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages
Name Type
@livekit/agents-plugin-speechify Minor
@livekit/agents Minor
@livekit/agents-plugin-anam Minor
@livekit/agents-plugin-anthropic Minor
@livekit/agents-plugin-assemblyai Minor
@livekit/agents-plugin-azure Minor
@livekit/agents-plugin-baseten Minor
@livekit/agents-plugin-bey Minor
@livekit/agents-plugin-cartesia Minor
@livekit/agents-plugin-cerebras Minor
@livekit/agents-plugin-deepgram Minor
@livekit/agents-plugin-did Minor
@livekit/agents-plugin-elevenlabs Minor
@livekit/agents-plugin-fishaudio Minor
@livekit/agents-plugin-google Minor
@livekit/agents-plugin-hume Minor
@livekit/agents-plugin-inworld Minor
@livekit/agents-plugin-krisp Minor
@livekit/agents-plugin-lemonslice Minor
@livekit/agents-plugin-liveavatar Minor
@livekit/agents-plugin-livekit Minor
@livekit/agents-plugin-minimax Minor
@livekit/agents-plugin-mistral Minor
@livekit/agents-plugin-mistralai Minor
@livekit/agents-plugin-neuphonic Minor
@livekit/agents-plugin-openai Minor
@livekit/agents-plugin-perplexity Minor
@livekit/agents-plugin-phonic Minor
@livekit/agents-plugin-protoface Minor
@livekit/agents-plugin-resemble Minor
@livekit/agents-plugin-rime Minor
@livekit/agents-plugin-runway Minor
@livekit/agents-plugin-sarvam Minor
@livekit/agents-plugin-silero Minor
@livekit/agents-plugin-soniox Minor
@livekit/agents-plugin-tavus Minor
@livekit/agents-plugins-test Minor
@livekit/agents-plugin-trugen Minor
@livekit/agents-plugin-xai Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

…failure after first frame instead of retrying
devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant