fix: skip 16kHz mono WAV conversion for cloud transcription adapters - #476
fix: skip 16kHz mono WAV conversion for cloud transcription adapters#476Swahjak wants to merge 2 commits into
Conversation
AudioFormatPreprocessor.AppliesTo unconditionally returned true, so every upload was transcoded to 16-bit PCM WAV before transcription. Local models need that, but the OpenAI-compatible adapter accepts the original compressed file and rejects requests over 25MB - so a small compressed recording could fail with a 413 purely because of the preprocessing pass. Add ModelCapabilities.SkipAudioNormalization (zero value false, so every existing adapter keeps normalizing), set it on the OpenAI adapter, and gate AppliesTo on it. A separate diarization pass reuses the same preprocessed audio, so the skip is only honoured when the diarization adapter agrees. Closes rishikanthc#475
… file Transcription and diarization consumed the same preprocessed file, so a job combining a cloud transcription model with a local diarization model had to normalize for both — re-enabling the 16kHz mono WAV conversion for the cloud call and reintroducing the request-size failure the skip flag was meant to fix. Each consumer now runs the preprocessing pipeline with its own adapter's capabilities. Identical preprocessing needs still share a single conversion (matched via ProcessingPipeline.PreprocessorSignature), and every temp file produced is tracked for cleanup.
|
Follow-up commit pushed to this branch: the first version of the fix was still incomplete. It computed one shared preprocessed file for the whole job, and because a separate diarization pass reused that same file, the code had to fall back to normalizing whenever The follow-up removes that all-or-nothing compromise. Multi-track jobs route each track back through
|
Closes #475
The bug
AudioFormatPreprocessor.AppliesToalways returnedtrue, so every upload went through an ffmpeg pass to 16kHz mono 16-bit PCM WAV before transcription, no matter which model was selected.That's correct for the local models (WhisperX/Parakeet/Canary/Voxtral), which expect exactly that input. It's harmful for the OpenAI-compatible adapter: those APIs accept the original compressed file (
flac/mp3/mp4/mpeg/mpga/m4a/ogg/wav/webm, as the adapter already declares) and enforce a hard 25MB request-size cap. Uncompressed PCM is several times larger than typical compressed source audio, so a recording that would have sailed under the limit in its original format gets inflated past it and the job dies with a 413 — for example a ~36 minute m4a of ~13MB becomes ~26MB of WAV. Splitting the source file isn't a reliable workaround either, because it's the transcoded size that decides, and users have no visibility into that.The fix
ModelCapabilities.SkipAudioNormalization. The zero value isfalse, so every adapter that doesn't say anything keeps normalizing exactly as before — no behaviour change for any local model, and no need to touch their capability declarations.true.AudioFormatPreprocessor.AppliesTonow returns!capabilities.SkipAudioNormalizationinstead of a hardcodedtrue.One extra detail worth flagging: a separate diarization pass in
unified_service.goreuses the same preprocessed audio, but the capabilities handed to the pipeline come from the transcription adapter. Without a guard, choosing OpenAI transcription plus local diarization would have fed the un-normalized file to pyannote/sortformer. So the skip is only honoured when the diarization adapter that will consume the file also opts out; otherwise normalization stays on.The adapter's
Transcribeneeded no changes — it just opensinput.FilePathand multipart-uploads it, with no WAV-specific logic, andValidateAudioInputalready checks the incoming format against the adapter's own supported-formats list.Testing
go build ./...— cleango vet ./...— cleango test ./internal/... ./tests/...— passing, except the pre-existingTestListTranscriptionJobsDeltaSyncfailure that also fails on unmodifiedmaininternal/transcription/pipeline/pipeline_test.gocovering both directions of the new gate (default normalizes, opt-out skips)