feat: Add content_audio_file() for audio input, and fix a Gemini audio-response crash

[cpsievert]

You can now hand an audio file to a chat the same way you already hand it an image or a PDF:

import chatlas as ctl

chat = ctl.ChatGoogle()
chat.chat(
    ctl.content_audio_file("standup.mp3"),
    "What action items were assigned, and to whom?",
)

Previously there was no inline audio path at all. The closest workaround was the Files API:

chat = ctl.ChatGoogle()
chat.chat(chat.files.upload("standup.mp3"), "What action items were assigned?")

That works on Gemini, but it's the wrong tool for a short clip. It's a two-step round trip that returns a provider-scoped id rather than data — so it expires, it isn't replayable on another provider, and a turn containing one can't be handed to a different chat. It's also undiscoverable as an audio feature, and it does nothing for ChatOpenAICompletions(), where audio has to arrive as an input_audio message part rather than a file reference.

Meeting summarization, call analysis, and voice notes now work directly. Gemini accepts up to 9.5 hours of audio per prompt at 32 tokens/second.

What each provider accepts

Audio support is narrower and more uneven than image support, so content_audio_file() sends what the provider can take and raises a specific, actionable error otherwise — rather than letting the API reject the request.

Input	ChatGoogle() / ChatVertex()	ChatOpenAICompletions()	ChatOpenAI()	ChatAnthropic()
.wav, .mp3	✅	✅ (on audio models, e.g. gpt-audio-mini)	❌ error: use Completions or Gemini	❌ error: use Completions or Gemini
.aiff, .aac, .ogg, .flac	✅	❌ error: wav/mp3 only	❌ error	❌ error
Audio in the response	✅ as ContentAudio	⚠️ transcript only, as text	—	—

The wav/mp3 ceiling on Chat Completions is OpenAI's, not ours: its input_audio part accepts exactly those two formats, against Gemini's six.

It also fixes a crash

ChatGoogle() raised pydantic.ValidationError on any response containing audio. _as_turn() passed whatever inline_data.mime_type Gemini returned straight into ContentImageInline.image_content_type, which is an image-only Literal. Gemini's TTS and native-audio responses return things like audio/pcm;rate=24000, so construction failed outright. A # type: ignore hid it from the type checker while pydantic still enforced the literal at runtime — which is exactly how it shipped. Reachable today via kwargs=, no new feature required.

That fix is isolated as the first commit (379b22a) and stands on its own if you'd rather take it separately.

Notes for review

ContentAudio.mime_type is a plain str, not a Literal. This is deliberate: a Literal is what caused the crash above, because a provider's output MIME types aren't drawn from the same set as its accepted input formats, and Gemini's carry parameters (;rate=24000) that would break exact matching anyway. Input validation lives in content_audio_file()'s signature, where typos are caught statically, while the model field stays honest about what a provider actually sent.

Audio output is represented for Gemini only. OpenAI's ChatCompletionAudio carries {id, data, expires_at, transcript} with no format field — the format is only ever specified on the request, which chatlas doesn't expose yet, so there's no reliable MIME type to attach to the bytes. Those are deferred (noted in the CHANGELOG). The transcript is surfaced as text, though, since ChatCompletionMessage.content is None when audio is requested and the turn would otherwise come back completely empty.

Requesting audio output is intentionally not here. The modalities/voice/format submit params are a parameter surface, not a content-model change, with their own questions about streaming and turn rendering.

Round-trip was checked against the live API. Gemini accepts a replayed audio/pcm;rate=24000 as input, so a conversation that produced audio survives its next turn.

Verified after merging main: pyright clean, ruff clean, 203 tests passing across the content and provider-dispatch suites, plus two live-recorded VCR cassettes (Gemini inline audio, and gpt-audio-mini audio input). The fixture is a generated 3.2 KB / 200 ms tone rather than a real recording, so the assertion is just "tone, not speech".

ellmer parity

New capability, not a port — ellmer has no audio content of any kind today:

	ellmer today
Audio input	No constructor exists. content_image_* and content_pdf_* are the only file inputs; the Content classes are Text, Image, ToolRequest, ToolResult, Json, Uploaded, Thinking, PDF.
Audio output	Not represented. Open issue #984 ("Support extracting non-text content from chat responses") asks for exactly this; closed #937 is a hand-rolled Gemini TTS workaround.
Gemini Files API	google_upload() has an audio MIME table (R/provider-google-upload.R:202-208 — mp3, wav, ogg, m4a, flac, aac). That's the analogue of chat.files.upload(), not inline audio.

If ellmer wants parity:

1. Add ContentAudio + content_audio_file(). An S7 class alongside ContentPDF carrying data and mime_type, with the same per-provider as_json() dispatch: inlineData for Gemini, an input_audio part for OpenAI, and a "use Gemini or OpenAI" error for Anthropic. Note the OpenAI half is easier there than here: ellmer::chat_openai() is already the Completions API, which is the one that takes audio, whereas chatlas.ChatOpenAI() defaults to Responses and can't. google_upload()'s MIME table is a starting point, but it lacks aiff and includes m4a, which Gemini's inline-audio docs don't list.
2. Keep the MIME type a free-form string, not a match.arg() enum — same reasoning as the crash above. Validate the input extension in content_audio_file(); leave the field itself permissive so a provider-generated audio/pcm;rate=24000 round-trips.
3. Represent Gemini audio output. This is the concrete first slice of open issue #984, and the R side has the same latent bug waiting if it ever coerces response MIME types into an image-only set.

No ellmer issue tracks audio input (searched audio, wav, mp3), so that part would be new.

Landing order

Part of a three-PR set (documents, audio, video). All three touch ContentTypeEnum, ContentUnion, and create_content(). #345 has since landed and this branch is merged up to current main, so there's nothing left to sequence here — #347 (video) is the remaining sibling.