Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/soft-voices-speak.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tanstack/ai-cartesia': minor
---

Add Cartesia text-to-speech and batch transcription adapters.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,7 @@ Official adapters include:
| [`@tanstack/ai-grok`](https://tanstack.com/ai/latest/docs/adapters/grok) | xAI Grok chat, images, and realtime |
| [`@tanstack/ai-groq`](https://tanstack.com/ai/latest/docs/adapters/groq) | Groq low-latency inference |
| [`@tanstack/ai-elevenlabs`](https://tanstack.com/ai/latest/docs/adapters/elevenlabs) | ElevenLabs realtime voice, speech, transcription, music, and sound effects |
| [`@tanstack/ai-cartesia`](https://tanstack.com/ai/latest/docs/adapters/cartesia) | Cartesia Sonic speech and Ink Whisper transcription |
| [`@tanstack/ai-byteplus`](https://tanstack.com/ai/latest/docs/adapters/byteplus) | BytePlus Seed chat, Seedance video, Seedream image, and Seed Speech TTS/ASR |
| [`@tanstack/ai-fal`](https://tanstack.com/ai/latest/docs/adapters/fal) | fal.ai image, video, audio, speech, and transcription models |
| [`@tanstack/ai-reactor`](https://tanstack.com/ai/latest/docs/adapters/reactor) | Reactor live world and video generation (Orbis, Happy Oyster, LingBot, Helios, FastH3) |
Expand Down
92 changes: 92 additions & 0 deletions docs/adapters/cartesia.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
title: Cartesia
id: cartesia-adapter
order: 10
description: "Generate speech with Sonic 3.6 and transcribe recordings with Ink Whisper in TanStack AI."
keywords:
- tanstack ai
- cartesia
- sonic
- ink whisper
- text to speech
- transcription
---

Call `cartesiaSpeech()` to generate audio with Sonic, or call `cartesiaTranscription()` to transcribe a recording with Ink Whisper.

## 1. Install

<!-- ::start:tabs variant="package-manager" mode="install" -->

react: @tanstack/ai @tanstack/ai-cartesia
vue: @tanstack/ai @tanstack/ai-cartesia
solid: @tanstack/ai @tanstack/ai-cartesia
svelte: @tanstack/ai @tanstack/ai-cartesia
preact: @tanstack/ai @tanstack/ai-cartesia
angular: @tanstack/ai @tanstack/ai-cartesia
vanilla: @tanstack/ai @tanstack/ai-cartesia
octane: @tanstack/ai @tanstack/ai-cartesia

<!-- ::end:tabs -->

## 2. Set the API key

Put your Cartesia key in the environment of your server:

```bash
CARTESIA_API_KEY=sk_car_...
```

## 3. Generate speech

Pass a [Cartesia voice ID](https://docs.cartesia.ai/build-with-cartesia/tts-models/latest#voice-selection) to `generateSpeech()`. The result contains base64 audio in MP3 format by default.

```typescript
import { generateSpeech } from '@tanstack/ai'
import { cartesiaSpeech } from '@tanstack/ai-cartesia'

const result = await generateSpeech({
adapter: cartesiaSpeech('sonic-3.6'),
text: 'Hello from Cartesia.',
voice: 'db6b0ed5-d5d3-463d-ae85-518a07d3c2b4',
threadId: 'speech-demo',
})

console.log(result.format, result.contentType)
```

Set `format` to `mp3`, `wav`, or `pcm`. `pcm` audio is mono `pcm_s16le` at 44100 Hz and has no header.

`speed` on `generateSpeech()` sets the speaking rate. Valid values are from 0.6 to 1.5. Values outside that range throw before the request.

`modelOptions.volume` sets the loudness. Valid values are from 0.5 to 2. Values outside that range throw before the request.

Use `modelOptions` for `language`, `normalization`, and `pronunciationDictId`.

## 4. Transcribe a recording

`cartesiaTranscription()` accepts an audio file, a Blob, an ArrayBuffer, or a base64 string. Use `responseFormat: 'verbose_json'` to request word timestamps.

If you omit `language`, Cartesia uses English and translates other speech into English. When the recording is not English, pass `language`.

```typescript
import { readFile } from 'node:fs/promises'
import { generateTranscription } from '@tanstack/ai'
import { cartesiaTranscription } from '@tanstack/ai-cartesia'

const bytes = await readFile('./recording.mp3')
const audio = new File([bytes], 'recording.mp3', { type: 'audio/mpeg' })

const result = await generateTranscription({
adapter: cartesiaTranscription('ink-whisper'),
audio,
responseFormat: 'verbose_json',
threadId: 'transcription-demo',
})

console.log(result.text, result.words)
```

For browser input and server routes, see [Text-to-Speech](../media/text-to-speech) and [Transcription](../media/transcription). The Text-to-Speech page includes a Cartesia route. The main Transcription sample uses OpenAI.

For Cartesia, call `cartesiaSpeech` or `cartesiaTranscription`. Set `CARTESIA_API_KEY` on the server. Pass a Cartesia voice id to `generateSpeech()`.
10 changes: 8 additions & 2 deletions docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -942,13 +942,13 @@
"label": "Text-to-Speech",
"to": "media/text-to-speech",
"addedAt": "2026-04-15",
"updatedAt": "2026-10-22"
"updatedAt": "2026-11-21"
},
{
"label": "Transcription",
"to": "media/transcription",
"addedAt": "2026-04-15",
"updatedAt": "2026-10-22"
"updatedAt": "2026-11-21"
},
{
"label": "Transcription: Advanced",
Expand Down Expand Up @@ -1880,6 +1880,12 @@
"addedAt": "2026-04-15",
"updatedAt": "2026-10-22"
},
{
"label": "Cartesia",
"to": "adapters/cartesia",
"addedAt": "2026-11-05",
"updatedAt": "2026-11-21"
},
{
"label": "fal.ai",
"to": "adapters/fal",
Expand Down
1 change: 1 addition & 0 deletions docs/getting-started/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ Each adapter is its own package. Install only the ones you use.
| `@tanstack/ai-grok` | xAI Grok. |
| `@tanstack/ai-bedrock` | Amazon Bedrock (Claude, Nova, Llama, and more through AWS). |
| `@tanstack/ai-byteplus` | BytePlus (Seed chat, Seedance video, Seedream image, Seed Speech). |
| `@tanstack/ai-cartesia` | Cartesia (Sonic speech and Ink Whisper transcription). |
| `@tanstack/ai-fal` | fal (image and video generation). |
| `@tanstack/ai-reactor` | Reactor (live world and video generation: Orbis, Happy Oyster, LingBot, Helios, FastH3). |
| `@tanstack/ai-worldlabs` | World Labs Marble (persistent 3D world generation). |
Expand Down
31 changes: 29 additions & 2 deletions docs/media/text-to-speech.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Text-to-Speech
id: text-to-speech
order: 3
description: "Turn text into spoken audio with generateSpeech() and play it in the browser. Works with OpenAI, Gemini, ElevenLabs, BytePlus, fal.ai, Grok, and Groq."
description: "Turn text into spoken audio with generateSpeech() and play it in the browser. Works with OpenAI, Gemini, ElevenLabs, Cartesia, BytePlus, fal.ai, Grok, and Groq."
keywords:
- tanstack ai
- text-to-speech
Expand Down Expand Up @@ -103,13 +103,40 @@ Each provider has a fixed set of voices. If none of them fit, [create your own v

## Pick a provider

Swap the adapter to change the provider. The rest of the route stays the same.
Swap the adapter to change the provider.

For Cartesia, use this route. `voice` is a Cartesia voice id. Set `CARTESIA_API_KEY` on the server.

```typescript
import {
generateSpeech,
generationParamsFromRequest,
toServerSentEventsResponse,
} from '@tanstack/ai'
import { cartesiaSpeech } from '@tanstack/ai-cartesia'

export async function POST(request: Request) {
const { input, threadId } = await generationParamsFromRequest('tts', request)

const stream = generateSpeech({
adapter: cartesiaSpeech('sonic-3.6'),
text: input.text,
threadId,
voice: input.voice ?? 'db6b0ed5-d5d3-463d-ae85-518a07d3c2b4',
stream: true,
})
return toServerSentEventsResponse(stream)
}
```

`alloy` is an OpenAI voice. Cartesia rejects it.

| Provider | Adapter | Models | API key |
| --- | --- | --- | --- |
| OpenAI | `openaiSpeech` from `@tanstack/ai-openai` | `tts-1-hd`, `tts-1`, `gpt-4o-mini-tts` | `OPENAI_API_KEY` |
| Gemini (experimental) | `geminiSpeech` from `@tanstack/ai-gemini` | `gemini-3.1-flash-tts-preview`, `gemini-2.5-flash-preview-tts`, `gemini-2.5-pro-preview-tts` | `GOOGLE_API_KEY` or `GEMINI_API_KEY` |
| ElevenLabs | `elevenlabsSpeech` from `@tanstack/ai-elevenlabs` | `eleven_v4`, `eleven_v4_turbo`, `eleven_v3`, and more | `ELEVENLABS_API_KEY` |
| Cartesia | `cartesiaSpeech` from `@tanstack/ai-cartesia` | `sonic-3.6`, `sonic-3.5`, `sonic-3`, `sonic-latest` | `CARTESIA_API_KEY` |
| BytePlus | `byteplusSpeech` from `@tanstack/ai-byteplus` | `seed-audio-1.0` | `BYTEPLUS_VOICE_API_KEY` |
| fal.ai | `falSpeech` from `@tanstack/ai-fal` | Kokoro, ElevenLabs, MiniMax, Gemini TTS, Chatterbox, and more | `FAL_KEY` |
| Grok | `grokSpeech` from `@tanstack/ai-grok` | `grok-tts` | `XAI_API_KEY` |
Expand Down
11 changes: 7 additions & 4 deletions docs/media/transcription.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Transcription
id: transcription
order: 4
description: "Turn recorded speech into text with generateTranscription() and the useTranscription hook. Works with OpenAI, ElevenLabs, Groq, xAI, BytePlus, fal.ai, Cloudflare, and Lovable."
description: "Turn recorded speech into text with generateTranscription() and the useTranscription hook. Works with OpenAI, ElevenLabs, Cartesia, Groq, xAI, BytePlus, fal.ai, Cloudflare, and Lovable."
keywords:
- tanstack ai
- transcription
Expand Down Expand Up @@ -145,7 +145,7 @@ console.log(result.text)

- A `File` or a `Blob`.
- An `ArrayBuffer`.
- A base64 string. The OpenAI and Groq adapters treat plain base64 as `audio/mpeg`.
- A base64 string. The OpenAI, Groq, and Cartesia adapters treat plain base64 as `audio/mpeg`.
- A data URL, such as `data:audio/webm;base64,...`. The type in the URL goes to the provider.

## Providers
Expand All @@ -156,6 +156,7 @@ Swap the adapter to change the provider. Each adapter reads its key from the env
| --- | --- | --- | --- |
| OpenAI | `openaiTranscription` | `gpt-4o-transcribe`, `gpt-4o-mini-transcribe`, `gpt-4o-transcribe-diarize`, `whisper-1` | `OPENAI_API_KEY` |
| ElevenLabs | `elevenlabsTranscription` | `scribe_v2`, `scribe_v2_medical` | `ELEVENLABS_API_KEY` |
| Cartesia | `cartesiaTranscription` | `ink-whisper` | `CARTESIA_API_KEY` |
| Groq | `groqTranscription` | `whisper-large-v3-turbo`, `whisper-large-v3` | `GROQ_API_KEY` |
| xAI | `grokTranscription` | `grok-stt` | `XAI_API_KEY` |
| BytePlus | `byteplusTranscription` | `seed-asr` | `BYTEPLUS_VOICE_API_KEY` |
Expand All @@ -165,14 +166,16 @@ Swap the adapter to change the provider. Each adapter reads its key from the env

Need speaker labels? Use `gpt-4o-transcribe-diarize`, or BytePlus with `enable_speaker_info`. Examples for each provider are in [Transcription: Advanced](./transcription-advanced).

If you omit `language` for Cartesia, the request uses English and translates other speech into English. When the recording is not English, pass `language`.

## Common options

| Option | Type | Description |
| --- | --- | --- |
| `audio` | `string \| File \| Blob \| ArrayBuffer` | The audio to transcribe (required) |
| `language` | `string` | The language in ISO-639-1 format, for example `"en"`, `"es"`, or `"ja"`. A correct language makes the result more accurate and faster. |
| `prompt` | `string` | Words or names the model must expect, for example `"Technical terms: API, SDK, CLI"`. `gpt-4o-transcribe-diarize` does not support it. |
| `responseFormat` | `'json' \| 'text' \| 'srt' \| 'verbose_json' \| 'vtt'` | The output format. `verbose_json` adds timestamps and segments. |
| `prompt` | `string` | Words or names the model must expect, for example `"Technical terms: API, SDK, CLI"`. `gpt-4o-transcribe-diarize` and Cartesia do not support `prompt`. |
| `responseFormat` | `'json' \| 'text' \| 'srt' \| 'verbose_json' \| 'vtt'` | The output format. `verbose_json` adds timestamps and segments. Cartesia accepts only `json` and `verbose_json`. For Cartesia, `verbose_json` returns word timestamps and does not return segments. |
| `modelOptions` | `object` | Provider options. They are typed for the model you pass. |
| `debug` | `DebugOption` | Log the request and every raw provider chunk. See [Debug Logging](../advanced/debug-logging). |

Expand Down
21 changes: 21 additions & 0 deletions packages/ai-cartesia/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2025 Tanner Linsley

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
33 changes: 33 additions & 0 deletions packages/ai-cartesia/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
<div align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://tanstack.com/api/readme/ai.png?theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://tanstack.com/api/readme/ai.png" />
<img src="https://tanstack.com/api/readme/ai.png" alt="TanStack AI" width="900" />
</picture>
</div>

<br />

# @tanstack/ai-cartesia

Cartesia text-to-speech and batch transcription adapters for TanStack AI.

```bash
pnpm add @tanstack/ai @tanstack/ai-cartesia
```

Set `CARTESIA_API_KEY` on your server. Then generate speech:

```ts
import { generateSpeech } from '@tanstack/ai'
import { cartesiaSpeech } from '@tanstack/ai-cartesia'

const result = await generateSpeech({
adapter: cartesiaSpeech('sonic-3.6'),
text: 'Hello from Cartesia.',
voice: 'db6b0ed5-d5d3-463d-ae85-518a07d3c2b4',
threadId: 'speech-demo',
})
```

For transcription, use `cartesiaTranscription('ink-whisper')` with `generateTranscription()`. To pass a key directly, use `createCartesiaSpeech()` or `createCartesiaTranscription()`.
55 changes: 55 additions & 0 deletions packages/ai-cartesia/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"name": "@tanstack/ai-cartesia",
"version": "0.0.0",
"description": "Cartesia speech and transcription adapters for TanStack AI",
"author": "Tanner Linsley",
"license": "MIT",
"homepage": "https://tanstack.com/ai",
"repository": {
"type": "git",
"url": "git+https://github.com/TanStack/ai.git",
"directory": "packages/ai-cartesia"
},
"type": "module",
"module": "./dist/esm/index.js",
"types": "./dist/esm/index.d.ts",
"exports": {
".": {
"types": "./dist/esm/index.d.ts",
"import": "./dist/esm/index.js"
}
},
"sideEffects": false,
"files": [
"dist",
"src"
],
"scripts": {
"build": "vite build",
"clean": "premove ./build ./dist",
"test:build": "publint --strict",
"test:coverage": "vitest run --coverage --coverage.include='src/**' --coverage.reporter=text-summary --coverage.reporter=json-summary",
"test:lib": "vitest run",
"test:types": "tsc",
"test:oxlint": "oxlint src --type-aware"
},
"keywords": [
"ai",
"tanstack",
"cartesia",
"text-to-speech",
"transcription"
],
"devDependencies": {
"@tanstack/ai": "workspace:*",
"@vitest/coverage-v8": "4.1.12",
"vite": "^8.3.8"
},
"peerDependencies": {
"@tanstack/ai": "workspace:^"
},
"dependencies": {
"@cartesia/cartesia-js": "^5.1.0",
"@tanstack/ai-utils": "workspace:^"
}
}
Loading
Loading