Skip to content

feat(alerts): calls speak in an agent's own Kokoro voice, not a vendor's - #136

Merged
ralyodio merged 1 commit into
masterfrom
worktree-kokoro-voice-calls
Nov 5, 2026
Merged

ralyodio merged 1 commit into
masterfrom
worktree-kokoro-voice-calls

Conversation

@ralyodio

@ralyodio ralyodio commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Anthony: "give my agents a voice using this instead of elevenlabs" →
"can't we use c0mpute.com?" → "use smshub.dev".

Alert calls read the line in a generic Telnyx female voice. Our agents have
voices of their own (riotcoder is kokoro / am_michael in its OpenProfile),
and since c0mpute 0.2.58 every c0mpute node provisions Kokoro itself — so the
network can speak for us, with no per-character bill.

Three tiers, best first

  1. Our own network. POST /v1/tts on the c0mpute speech gateway auctions a
    tts.synthesize job to a node running Kokoro and hands back a WAV; the call
    plays it with playback_start.
  2. Telnyx's copy of the same model (Telnyx.KokoroTTS.<voice>) when no
    gateway is in reach — same voice, vendor round trip, no hosting.
  3. The old generic voice, only if Telnyx ever rejects a Kokoro voice id.

Every step fails soft — no gateway configured, a node that cannot be reached, a
200 that is not audio — because an alert is worth more than the voice it
arrives in.

The audio URL

The rendered WAV waits in a disk cache under an HMAC-named path, and
/api/v1/alerts/audio/<token> serves it unauthenticated, because the only
fetcher is Telnyx's media loader. The token is an HMAC of the call id under a
server secret, so a path cannot be guessed or walked; the URL stops working
after ten minutes, and the sweep runs on each write. Not /tmp — that is
quota-bound on dev2.

call.playback.ended now hangs up on the same h = 1 rule as
call.speak.ended, so a played line ends a call exactly like a spoken one.

Config

Variable Meaning
C0MPUTE_SPEECH_URL, C0MPUTE_SPEECH_TOKEN the c0mpute speech gateway (tier 1). Unset → tier 2
ALERT_KOKORO_VOICE af_sarah, af_bella, am_onyx, am_michael (default)
ALERT_AUDIO_DIR where rendered lines wait (default .cache/alert-audio)
ALERT_AUDIO_SECRET signs the path token; falls back to the service-role key

Testing

  • Full suite on the pre-commit hook: 380 tests / 62 files pass, typecheck
    and lint clean. 8 of those are new, covering the gateway being off, an empty
    line, a non-absolute base URL, a refusing gateway, a throwing fetch, a 200
    that is not audio, token path-walking (../../etc/passwd), tokens being tied
    to the secret, and the TTL sweep.
  • The updated alert-flow test now asserts the call speaks
    Telnyx.KokoroTTS.am_michael rather than a generic voice.
  • Synthesis itself verified against the live gateway on dev2 before writing
    this: 15.56 s of 24 kHz audio in am_michael, auctioned to a Kokoro node.

🤖 Generated with Claude Code

Alert calls read the line with a generic Telnyx "female" voice. Our agents
have voices of their own — riotcoder is kokoro/am_michael in its OpenProfile
— and as of c0mpute 0.2.58 every c0mpute node provisions Kokoro itself, so
the network can speak for us.

Three tiers, best first:

1. Our own network. POST /v1/tts on the c0mpute speech gateway auctions a
   tts.synthesize job to a node running Kokoro and hands back a WAV; the call
   plays it with playback_start. No per-character bill, and the voice is the
   agent's.
2. Telnyx's copy of the same model (Telnyx.KokoroTTS.<voice>) when we have no
   gateway in reach — same voice, vendor round trip.
3. The old generic voice, only if Telnyx ever rejects a Kokoro voice id.

An alert is worth more than the voice it arrives in, so every step fails
soft: no gateway configured, a node that cannot be reached, a 200 that is not
audio — each returns null and the call speaks the next way down.

The rendered WAV waits in a disk cache under an HMAC-named path and
/api/v1/alerts/audio/<token> serves it unauthenticated, because the only
fetcher is Telnyx's media loader: the token is an HMAC of the call id under a
server secret, the URL stops working after ten minutes, and the sweep runs on
each write. Not /tmp — that is quota-bound on dev2.

call.playback.ended now hangs up on the same h = 1 rule as call.speak.ended,
so a played line ends the call exactly like a spoken one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

ThreatCrush Security Scan

14 finding(s)

HIGH/CRITICAL: 2 | MEDIUM: 4 | LOW: 8

Severity Rule Location
HIGH sh-unquoted-expansion-destructive public/install.sh:67
HIGH sh-remote-script-execution public/install.sh:89
MEDIUM sh-remote-script-execution public/install.sh:133
MEDIUM js-open-redirect src/app/numbers/page.tsx:180
MEDIUM js-open-redirect src/components/inbox-client.tsx:35
MEDIUM js-unescaped-html-sink src/components/pit-footer.tsx:17
LOW secret-generic-credential src/lib/providers/__tests__/twilio.test.ts:24
LOW secret-generic-credential src/lib/providers/__tests__/twilio.test.ts:46
LOW secret-generic-credential src/lib/providers/__tests__/twilio.test.ts:60
LOW secret-generic-credential src/lib/providers/__tests__/twilio.test.ts:78
LOW secret-generic-credential src/lib/vault/__tests__/vault.test.ts:117
LOW secret-generic-credential src/lib/webhooks/__tests__/supabase-store.test.ts:11
LOW secret-generic-credential supabase/config.toml:287
LOW secret-generic-credential supabase/config.toml:319

Snippets are redacted; ThreatCrush never prints matched credential material.

Comment thread src/lib/voice/kokoro.ts
const token = audioToken(options.id, env);
try {
await mkdir(dir, { recursive: true });
await writeFile(join(dir, `${token}.wav`), wav);
@ralyodio
ralyodio merged commit 9beb82c into master Nov 5, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants