Skip to content

feat: chunk long audio for duration-capped OpenAI models (gpt-4o-transcribe) - #478

Open
Swahjak wants to merge 5 commits into
rishikanthc:mainfrom
Swahjak:feat/openai-chunking-for-duration-capped-models
Open

feat: chunk long audio for duration-capped OpenAI models (gpt-4o-transcribe)#478
Swahjak wants to merge 5 commits into
rishikanthc:mainfrom
Swahjak:feat/openai-chunking-for-duration-capped-models

Conversation

@Swahjak

@Swahjak Swahjak commented Aug 20, 2026

Copy link
Copy Markdown

Closes #477.

Problem

gpt-4o-transcribe and gpt-4o-mini-transcribe reject audio longer than 1400 seconds (~23 minutes) with an HTTP 400:

"audio duration 2158.5493333333334 seconds is longer than 1400 seconds which is the maximum for this model"

whisper-1 has no duration cap (only the 25MB request-size limit, addressed separately in #476), so today it is the only OpenAI model usable for meeting-length recordings — even though gpt-4o-mini-transcribe is half its price and gpt-4o-transcribe is materially more accurate at the same price. Anything past ~23 minutes simply fails.

Fix

Client-side chunking in the OpenAI adapter, transparent to callers.

  • Duration caps are declared, not guessed. openAIModelDurationCaps maps model ID → cap (gpt-4o-transcribe: 1400s, gpt-4o-mini-transcribe: 1400s). The API does not expose these limits, so they are hardcoded with a comment saying so. A model absent from the map is uncapped.
  • Chunking only when needed. If the model has a cap and input.Duration exceeds it, the file is split with ffmpeg into sequential chunks of cap - 10s (a small safety margin, since cutting on frame boundaries can make a chunk marginally longer than requested), each overlapping the previous by 3s so a word spoken across a cut point is not lost. ffmpeg stream-copies the chunk (exact for audio-only streams, no re-encode of what may be hours of audio) and falls back to a re-encode if the container cannot be cut that way.
  • One request per chunk. The existing multipart/HTTP/response-parsing path was extracted into transcribeFile, which both the single-shot and the chunked path call — the request is built identically either way, no duplicated code path.
  • Reassembly. Each chunk's segment and word timestamps are shifted by that chunk's start offset, so everything is relative to the original recording. For the region two chunks share, the merge switches chunks at the midpoint of the overlap: content before it is more central to the earlier chunk, content after it more central to the later one, so overlapping text is kept exactly once. Models that answer without segment timestamps (the gpt-4o family returns plain json, one segment per chunk) cannot be de-duplicated that way, so the repeated leading words of the next chunk are dropped instead — a bounded word-run comparison ignoring case and punctuation, not a diff algorithm. Segment starts are clamped to stay monotonic across a boundary.
  • Cleanup. Chunking is self-contained in the adapter, so chunk files are removed by a defer there rather than being threaded through unified_service.go's tempFilesToCleanup.

No regression for uncapped models

whisper-1 (and any model not in the cap map) takes the exact same single-request path as before: no ffmpeg invocation, no chunk planning, no merge. The only change on that path is that ProcessingTime/ModelUsed/Metadata are now set by the caller instead of inside the request helper, and the whole-text fallback segment falls back to the known audio duration when the response reports none.

Why overlap rather than silence/VAD cut points

The issue floats splitting on silence boundaries as the cleaner alternative. The pipeline's VoiceActivityDetectionPreprocessor is currently a placeholder that returns its input unchanged, so there is no silence detection to wire in — building it is out of scope for this change. Overlap plus de-duplication is the approach that fits what is there today.

Base branch

This branch is main merged with #474 (feat/enabled-models-env) and #476 (fix/skip-normalization-for-cloud-transcription), since it builds on the OpenAI adapter changes in #476 and the model-capability plumbing in #474. The one merge conflict (in unified_service.go, where both branches touched the diarization call site) was resolved by keeping both sides: EnsureModelReady from #474 plus per-adapter preprocessing from #476.

It should be reviewed/merged after those two land — or rebased if they merge with a different final shape.

Build and tests

  • go build ./... — clean
  • go vet ./... — clean
  • go test ./internal/... ./tests/... — 230 passed, 1 pre-existing unrelated failure (TestListTranscriptionJobsDeltaSync, fails on main too). No new failures.

New tests in internal/transcription/adapters/openai_chunking_test.go cover the chunking logic in isolation (no network or API key needed):

  • the cap lookup, including whisper-1 and unknown models resolving to uncapped
  • audio under the cap producing a single chunk covering the whole file (the no-chunking case)
  • a one-hour file producing chunks that stay under the limit, leave no gaps, overlap by exactly the configured amount, and end on the end of the file
  • an overlap too large to make progress being ignored rather than looping
  • timestamp shifting onto the original timeline for both segments and words
  • the overlap de-duplication keeping shared speech exactly once, with monotonic segment starts
  • the untimed (gpt-4o-shaped) case, where the repeated leading words of the next chunk are dropped
  • a single chunk passing through with its timings untouched
  • the word-run trim itself: repeated tails, case/punctuation differences, unrelated text left alone, fully-repeated text, empty input

InitializeModels prepared every registered adapter on startup, so a
cloud-only setup still installed the local Python environments and
downloaded weights for WhisperX, Parakeet, Canary, Voxtral, PyAnnote and
Sortformer.

SCRIBERR_ENABLED_MODELS takes a comma separated list of model IDs and
limits startup preparation to those models. Unset or empty keeps the
current behaviour of preparing everything. Skipped models stay
registered and are prepared on demand the first time a job uses them.
AudioFormatPreprocessor.AppliesTo unconditionally returned true, so every
upload was transcoded to 16-bit PCM WAV before transcription. Local models
need that, but the OpenAI-compatible adapter accepts the original compressed
file and rejects requests over 25MB - so a small compressed recording could
fail with a 413 purely because of the preprocessing pass.

Add ModelCapabilities.SkipAudioNormalization (zero value false, so every
existing adapter keeps normalizing), set it on the OpenAI adapter, and gate
AppliesTo on it. A separate diarization pass reuses the same preprocessed
audio, so the skip is only honoured when the diarization adapter agrees.

Closes rishikanthc#475
… file

Transcription and diarization consumed the same preprocessed file, so a job
combining a cloud transcription model with a local diarization model had to
normalize for both — re-enabling the 16kHz mono WAV conversion for the cloud
call and reintroducing the request-size failure the skip flag was meant to fix.

Each consumer now runs the preprocessing pipeline with its own adapter's
capabilities. Identical preprocessing needs still share a single conversion
(matched via ProcessingPipeline.PreprocessorSignature), and every temp file
produced is tracked for cleanup.
…-transcription' into feat/openai-chunking-for-duration-capped-models

# Conflicts:
#	internal/transcription/unified_service.go
gpt-4o-transcribe and gpt-4o-mini-transcribe reject audio longer than
1400 seconds with an HTTP 400, which makes them unusable for anything
past ~23 minutes — the length of a typical meeting recording. whisper-1
has no such limit, so it stayed the only workable choice despite being
more expensive and less accurate.

Audio longer than a model's known cap is now split with ffmpeg into
sequential chunks, transcribed one request at a time, and reassembled
into a single transcript: chunk timestamps are shifted onto the original
timeline, and the seconds two chunks share are taken from whichever
chunk covers them more centrally. Models answering without timestamps
get the repeated leading words of a chunk dropped instead. Chunk files
are removed when the job finishes.

Models without a known cap keep the exact single-request path they had,
so whisper-1 gains no ffmpeg work and no behaviour change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: client-side chunking for OpenAI-compatible adapter to support duration-capped models (gpt-4o-transcribe / gpt-4o-mini-transcribe)

1 participant