Skip to content

fix: skip 16kHz mono WAV conversion for cloud transcription adapters - #476

Open
Swahjak wants to merge 2 commits into
rishikanthc:mainfrom
Swahjak:fix/skip-normalization-for-cloud-transcription
Open

fix: skip 16kHz mono WAV conversion for cloud transcription adapters#476
Swahjak wants to merge 2 commits into
rishikanthc:mainfrom
Swahjak:fix/skip-normalization-for-cloud-transcription

Conversation

@Swahjak

@Swahjak Swahjak commented Aug 20, 2026

Copy link
Copy Markdown

Closes #475

The bug

AudioFormatPreprocessor.AppliesTo always returned true, so every upload went through an ffmpeg pass to 16kHz mono 16-bit PCM WAV before transcription, no matter which model was selected.

That's correct for the local models (WhisperX/Parakeet/Canary/Voxtral), which expect exactly that input. It's harmful for the OpenAI-compatible adapter: those APIs accept the original compressed file (flac/mp3/mp4/mpeg/mpga/m4a/ogg/wav/webm, as the adapter already declares) and enforce a hard 25MB request-size cap. Uncompressed PCM is several times larger than typical compressed source audio, so a recording that would have sailed under the limit in its original format gets inflated past it and the job dies with a 413 — for example a ~36 minute m4a of ~13MB becomes ~26MB of WAV. Splitting the source file isn't a reliable workaround either, because it's the transcoded size that decides, and users have no visibility into that.

The fix

  • New field ModelCapabilities.SkipAudioNormalization. The zero value is false, so every adapter that doesn't say anything keeps normalizing exactly as before — no behaviour change for any local model, and no need to touch their capability declarations.
  • The OpenAI adapter sets it to true.
  • AudioFormatPreprocessor.AppliesTo now returns !capabilities.SkipAudioNormalization instead of a hardcoded true.

One extra detail worth flagging: a separate diarization pass in unified_service.go reuses the same preprocessed audio, but the capabilities handed to the pipeline come from the transcription adapter. Without a guard, choosing OpenAI transcription plus local diarization would have fed the un-normalized file to pyannote/sortformer. So the skip is only honoured when the diarization adapter that will consume the file also opts out; otherwise normalization stays on.

The adapter's Transcribe needed no changes — it just opens input.FilePath and multipart-uploads it, with no WAV-specific logic, and ValidateAudioInput already checks the incoming format against the adapter's own supported-formats list.

Testing

  • go build ./... — clean
  • go vet ./... — clean
  • go test ./internal/... ./tests/... — passing, except the pre-existing TestListTranscriptionJobsDeltaSync failure that also fails on unmodified main
  • Added internal/transcription/pipeline/pipeline_test.go covering both directions of the new gate (default normalizes, opt-out skips)

AudioFormatPreprocessor.AppliesTo unconditionally returned true, so every
upload was transcoded to 16-bit PCM WAV before transcription. Local models
need that, but the OpenAI-compatible adapter accepts the original compressed
file and rejects requests over 25MB - so a small compressed recording could
fail with a 413 purely because of the preprocessing pass.

Add ModelCapabilities.SkipAudioNormalization (zero value false, so every
existing adapter keeps normalizing), set it on the OpenAI adapter, and gate
AppliesTo on it. A separate diarization pass reuses the same preprocessed
audio, so the skip is only honoured when the diarization adapter agrees.

Closes rishikanthc#475
… file

Transcription and diarization consumed the same preprocessed file, so a job
combining a cloud transcription model with a local diarization model had to
normalize for both — re-enabling the 16kHz mono WAV conversion for the cloud
call and reintroducing the request-size failure the skip flag was meant to fix.

Each consumer now runs the preprocessing pipeline with its own adapter's
capabilities. Identical preprocessing needs still share a single conversion
(matched via ProcessingPipeline.PreprocessorSignature), and every temp file
produced is tracked for cleanup.
@Swahjak

Swahjak commented Aug 20, 2026

Copy link
Copy Markdown
Author

Follow-up commit pushed to this branch: the first version of the fix was still incomplete.

It computed one shared preprocessed file for the whole job, and because a separate diarization pass reused that same file, the code had to fall back to normalizing whenever diarize was set — including for the cloud transcription call. So a job with model_family: openai + diarize: true (sortformer) still sent a 16kHz mono WAV to OpenAI and still failed with HTTP 413 over the 25MB request cap. Confirmed live: the identical job with diarize: false succeeded.

The follow-up removes that all-or-nothing compromise. processSingleTrackJob now preprocesses per consumer: the transcription input is produced with the transcription adapter's capabilities and the diarization input with the diarization adapter's, so OpenAI gets the original compressed file while sortformer gets its normalized WAV. To avoid a second ffmpeg pass when both consumers want the same thing (two local models), results are reused when the set of applicable preprocessors matches — via a new ProcessingPipeline.PreprocessorSignature. All temp files created are tracked for cleanup. Composite transcription+diarization models (WhisperX with its own diarization) are unaffected: they still only preprocess once.

Multi-track jobs route each track back through ProcessJobprocessSingleTrackJob, so they inherit the fix; processMultiTrackJob itself does no preprocessing and needed no change.

go build ./..., go vet ./... and go test ./internal/... ./tests/... all pass, apart from the pre-existing unrelated TestListTranscriptionJobsDeltaSync failure. Added pipeline tests covering the divergent and identical preprocessing cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AudioFormatPreprocessor unconditionally transcodes to 16kHz mono WAV, even for cloud API transcription providers

1 participant