Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ of tests.
| `TRANSCRIBE_DUMP_DIR=<dir>` | Enable the per-stage tensor dumper; writes `<name>.f32` + `<name>.json` per dumped tensor into `<dir>`. The basis for the numerical-comparison harness (`scripts/compare_tensors.py`). |
| `TRANSCRIBE_PERF_DEBUG` | Print a per-stage timing breakdown to stderr (DEBUG log) on the families that profile (`cohere`, `granite`, `canary`, `canary_qwen`, `moonshine`, `moonshine_streaming`, `moss`, `qwen3_asr`, `whisper`). For whisper, a value containing `cpu` or `all` additionally prints the CPU sub-section breakdown. |
| `TRANSCRIBE_VOXTRAL_REALTIME_STREAM_TIMING` | Print a per-component streaming wall-time breakdown at stream finalize (voxtral_realtime). |
| `TRANSCRIBE_MOONSHINE_STREAMING_ENC_WINDOW=<frames>` | Encoder frames emitted per window by the moonshine_streaming one-shot / batch encoder (default 1024, ~20 s). Windowing bounds encoder memory; it is mathematically exact (each window carries the full sliding-window context) and float-level differences stay at the backend's reduction-order noise. Set small to exercise the windowed path on short clips. Read at each run. |
| `TRANSCRIBE_TEST_DEV_INIT_THROW=<match>` | Fault injection: backend device init (`ggml_backend_dev_init`) throws for devices whose name contains `<match>` (`*` matches every device). Exercises throw → skip → CPU-fallback in backend probing; an explicit backend request fails with `TRANSCRIBE_ERR_BACKEND`. Used by `backend_init_throw_unit` and `scripts/ci/vulkan_degradation_check.py`. |
| `TRANSCRIBE_TEST_TEARDOWN_THROW` | Fault injection: any non-empty value injects a throw after each real free inside the `transcribe::safe_*` teardown wrappers, proving containment without leaking the handle. Used by `teardown_safety_unit`. |

Expand Down
18 changes: 10 additions & 8 deletions docs/input-limits.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ need and do not have a length gate.

| Families | Limit source | Behavior |
| --- | --- | --- |
| qwen3_asr, canary_qwen, funasr_nano, granite, granite_nar, voxtral, cohere, canary | decoder context window (`dec_max_position_embeddings` / `dec_max_seq`), or the encoder positional table (`enc_pos_emb_max_len`, for cohere/canary) — all from GGUF | KV cache grows to fit, clamped to the model's true max. Over-length input is **rejected before the decode** (or before the encoder, where the encoder table is the binding limit) with `TRANSCRIBE_ERR_INPUT_TOO_LONG`. |
| qwen3_asr, canary_qwen, funasr_nano, granite, granite_nar, voxtral, cohere, canary, moonshine_streaming | decoder context window (`dec_max_position_embeddings` / `dec_max_seq`), or an encoder-side positional table (`enc_pos_emb_max_len` for cohere/canary; the adapter `pos_emb` table, 4096 frames = 81.9 s, for moonshine_streaming) — all from GGUF | KV cache grows to fit, clamped to the model's true max. Over-length input is **rejected before the decode** (or before the encoder, where the encoder table is the binding limit) with `TRANSCRIBE_ERR_INPUT_TOO_LONG`. A moonshine_streaming *stream* that reaches its cap is truncated instead (see "Streaming is the exception"). |

A clip's sample count deterministically fixes its decoder prefill size or
encoder frame count, so the library checks the relevant bound before running
Expand Down Expand Up @@ -107,7 +107,7 @@ on sequences which are identical and are obviously looping

| Families | Window | Behavior |
| --- | --- | --- |
| sensevoice (~30 s), medasr (~400 s), moonshine (output-bound, ~48 s), moonshine_streaming (output-bound, ~17 min) | training / positional window | Any length is accepted; past the window the library emits a `WARN` (degraded accuracy is possible) and proceeds. `max_audio_ms` reports the window as advisory. |
| sensevoice (~30 s), medasr (~400 s), moonshine (output-bound, ~48 s) | training / positional window | Any length is accepted; past the window the library emits a `WARN` (degraded accuracy is possible) and proceeds. `max_audio_ms` reports the window as advisory. |

These families have no hard architectural wall but were trained on a bounded
window; beyond it, accuracy degrades rather than failing. The library does not
Expand All @@ -119,11 +119,12 @@ Moonshine is the honest edge case in this bucket: its cap is on *output*
audio length — a dense short clip can hit it too. It is reported via
`transcribe_was_truncated()` and a `WARN` (and, offline, the hard
`TRANSCRIBE_ERR_OUTPUT_TRUNCATED` status) when the cap is reached.
`moonshine_streaming` has the same output-bound shape with a much larger window
(`dec_max_position_embeddings = 4096`, ≈ 17 min); because it also streams, its
truncation follows the streaming rule below — `stream_finalize` still returns
`TRANSCRIBE_OK` and the truncation surfaces only through
`transcribe_was_truncated()`.
`moonshine_streaming` used to be listed here as output-bound (≈ 17 min). It is
not: its encoder adapter adds a position embedding per encoder frame from a
4096-entry table, so input past 81.9 s has no defined output (the reference
raises an index error, and the port used to read past the table). It is now a
hard-cap family (bucket 2) for offline input, and a stream that reaches the cap
follows the streaming rule below.

## Context sizing and the `n_ctx` knob

Expand Down Expand Up @@ -194,7 +195,8 @@ IDLE/ACTIVE/FINISHED/FAILED), and `stream_feed` / `stream_finalize` return the
status of *that step*, not a verdict on the whole transcript. So when a
streaming decode reaches its context cap (e.g. `voxtral_realtime` at its
absolute position limit — hours of continuous audio, or `moonshine_streaming`
at its output window), the stream does **not** fail and `stream_finalize`
at its 81.9 s input cap, past which it ignores further audio), the stream does
**not** fail and `stream_finalize`
returns `TRANSCRIBE_OK`; the truncation is surfaced through
`transcribe_was_truncated(session)` and a `WARN`. This is deliberate:
forcing a stream into a failed terminal state on truncation would discard the
Expand Down
23 changes: 17 additions & 6 deletions docs/models/moonshine-streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,12 +64,23 @@ each per-variant doc has direct download links.

## Input limits

No input-length limit, but the decoder is capped at its output window — about
**17 minutes** of typical speech (a 4,096-token decode window, shared across
variants). Offline, a transcript that reaches the cap is returned with
`TRANSCRIBE_ERR_OUTPUT_TRUNCATED` (partial text retained); when streaming, the
stream keeps its committed text and sets `transcribe_was_truncated()` while
finalize still returns OK. See the [input-length contract](../input-limits.md).
**81.9 s of audio per decode** (all variants). The encoder adapter adds a
learned position embedding to every encoder frame from a 4,096-entry table, and
one encoder frame is 20 ms; the reference implementation has no defined output
past it either. `transcribe_capabilities::max_audio_ms` reports 81920.

- **Offline** (`transcribe_run`, `transcribe_run_batch`): longer input is
rejected up front with `TRANSCRIBE_ERR_INPUT_TOO_LONG` (per utterance in a
batch). Split long recordings before transcribing them.
- **Streaming**: a stream that runs past 81.9 s keeps its transcript up to the
cap, ignores later audio, sets `transcribe_was_truncated()` and logs one
`WARN`; feeds and finalize still return `TRANSCRIBE_OK`. Start a new stream
(for example at a pause) to keep transcribing.

Within that window the decoder can still stop early: a transcript that reaches
the generation budget is returned with `TRANSCRIBE_ERR_OUTPUT_TRUNCATED`
offline (partial text retained), or flagged via `transcribe_was_truncated()`
when streaming. See the [input-length contract](../input-limits.md).

## Quick start

Expand Down
Loading
Loading