feat: add voice dictation beta

[t3-code]

summary

• add opt-in voice dictation beta with waveform, timer, and stop controls in the composer
• support simple cloud-first BYOK transcription with OpenAI first/default and Groq
• proxy authenticated, size-limited audio through fixed provider endpoints
• use portable MediaRecorder APIs for Linux, macOS, and Windows desktop builds

providers

• OpenAI: gpt-4o-mini-transcribe
• Groq: whisper-large-v3-turbo
• Claude has no native speech-to-text API, so it is not included

demo

https://t3bot-production.up.railway.app/files/jnX8qH7ZPKYJaOkK-_lBmwYO/t3code-voice-dictation-beta-demo.mp4

testing

• transcription and client settings tests
• server, web, and contracts typechecks
• formatting and lint
• full server suite: transcription tests pass; 3 unrelated permission-sensitive tests fail under this root runner

---

Note

Medium Risk
New authenticated proxy sends user audio and API keys to third-party providers; scope is bounded by auth, size limits, and fixed endpoints, but misconfiguration or key handling still matters.

Overview
Adds an opt-in voice dictation beta: record in the chat composer, transcribe via the connected T3 server, and append text to the draft.

Client: New client settings (voiceTranscriptionEnabled, provider OpenAI/Groq, optional local API key, model). Beta settings loads models from the provider and shows whether OPENAI_API_KEY / GROQ_API_KEY exist on the server (booleans only). useVoiceTranscription uses MediaRecorder (5‑minute cap, waveform UI); ChatComposer adds mic/stop and VoiceTranscriptionPanel. macOS desktop builds add NSMicrophoneUsageDescription.

Server: Authenticated routes GET/POST /api/transcription and GET /api/transcription/models proxy to fixed OpenAI/Groq endpoints with a 25 MB limit, client key or env fallback, and structured errors. CORS allows the x-t3-transcription-* headers.

^{Reviewed by Cursor Bugbot for commit 14315f5. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add voice dictation beta to the chat composer

• Adds a mic button to ChatComposer that lets users record audio and append the transcription to the prompt; recording is capped at 5 minutes and 25 MB.
• Introduces server endpoints GET/POST /api/transcription and GET /api/transcription/models that proxy audio to OpenAI or Groq and return transcribed text or available models.
• Adds a Beta Settings panel section where users configure the provider, API key, and model; the panel dynamically loads available models and reflects server-side environment key availability.
• Adds useVoiceTranscription hook managing MediaRecorder lifecycle, AudioContext level sampling, elapsed time, and error reporting with full cleanup on unmount.
• macOS desktop build now declares NSMicrophoneUsageDescription in Info.plist for the microphone permission prompt.
• Risk: microphone access and transcription provider calls are new runtime dependencies; missing model selection or unsupported browser capabilities surface as user-facing errors.

^{Macroscope summarized 14315f5.}

[macroscopeapp]

Effect service conventions review of the voice dictation changes. Three findings, all in the new server-side transcription path (apps/server/src/transcription.ts and its route in apps/server/src/http.ts). The web, contracts, and desktop changes look consistent with the conventions.

^{Posted via Macroscope — Effect Service Conventions}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR introduces a complete voice dictation feature with new server endpoints, external API integrations (OpenAI/Groq), microphone recording capabilities, and API key handling. New features of this scope warrant human review. Additionally, there's an open bug report about state management blocking recording restarts.

^{You can customize Macroscope's approvability policy. Learn more.}

[macroscopeapp]

Effect service conventions review: one finding in apps/server/src/transcription.ts. The earlier issues (environment-acquired HttpClient, tagged errors with structured attributes, no raw provider payloads in messages, catchTags at the route) are resolved.

^{Posted via Macroscope — Effect Service Conventions}

[macroscopeapp]

One convention issue found in the new server transcription module. Everything else (tagged errors with structured attributes, HttpClient acquired from the environment, route-level catchTags) looks consistent with the service conventions.

^{Posted via Macroscope — Effect Service Conventions}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 14315f5. Configure here.}

[cursor]

Restart blocked after cancel start

Medium Severity

stop during startup sets cancelStartingRef and returns UI to idle, but leaves startingRef true. start bails out while that flag is set, so tapping start again does nothing until the still-pending getUserMedia call settles.

Additional Locations (1)

• apps/web/src/hooks/useVoiceTranscription.ts#L96-L112

 

^{Reviewed by Cursor Bugbot for commit 14315f5. Configure here.}