Skip to content

parakeet: honor run_params.vocabulary with phrase boosting - #221

Open
mikwiseman wants to merge 1 commit into
handy-computer:mainfrom
mikwiseman:parakeet-phrase-boosting
Open

mikwiseman wants to merge 1 commit into
handy-computer:mainfrom
mikwiseman:parakeet-phrase-boosting

Conversation

@mikwiseman

Copy link
Copy Markdown

transcribe_run_params::vocabulary has existed since 0.3, but Parakeet ignores it with a WARN because it has no prompt slot. This change makes the TDT and RNN-T greedy decoders take the terms as phrase boosting. Every Parakeet user (v3, Ultra and the rest of the family) can then bias recognition toward names and jargon through the existing API, with no new public symbol.

How it works

  • Each term is tokenized the way the model writes it, starting at a word boundary, with the existing SentencePiece encoder. A term with characters the tokenizer cannot write is skipped with a WARN.
  • The terms form an Aho-Corasick automaton over token ids (src/transcribe-phrase-boost.{h,cpp}). Every transition is resolved through the failure links once, at build time.
  • The model's own argmax decides between blank and a token. Only when it emits a token does the vocabulary re-rank the non-blank tokens. Biasing therefore changes which word is written and never whether one is, so it cannot invent speech on silence or drop a word the model emitted.
  • A term earns no bonus for its first piece. Once the model has begun a term on its own, each piece that extends the match earns a logit bonus. A piece that breaks the match gives back what the unfinished match earned, and a completed term keeps its bonus. Many terms begin with a short, common word piece, and a bonus there rewrote ordinary words.
  • The reported token confidence stays the model's own.
  • With no terms the decoder never calls into the automaton, so output is bit-exact with today's.
  • The bonus defaults to 8.0, chosen on a held-out half of the term set, and TRANSCRIBE_VOCABULARY_BOOST overrides it. Streaming and CTC heads ignore terms for now, and docs/prompting.md says so.
  • The parakeet loader sets TRANSCRIBE_FEATURE_VOCABULARY for TDT and RNN-T heads.

Tests

  • New host-only unit test tests/phrase_boost_unit.cpp. It checks that a completed term keeps its bonus, that a broken match gives back exactly what it earned, that a suffix completing another term keeps that term through the failure link, and that apply matches delta for every token while leaving blank alone.
  • ctest: 60 of 60 pass on macOS (Apple M4 Pro, Metal); transcribe_cli_diarize_smoke is skipped as usual.
  • On this branch, Parakeet v3 Q8 through transcribe-cli with no terms gives byte-identical text to v0.4.0 on 12 synthetic term sentences. With 60 terms, 7 of the 12 change.

Measurements

These come from OpenRamble's report, which used the same change on v0.4.0 with Parakeet v3 Q8, Metal and one decoder thread. The rules were preregistered before the code existed.

No terms With 60 terms
English term recall, held-out half of a 70-sentence synthetic set 22/37 31/37
Pooled WER, all 1,422 FLEURS English and Russian test utterances +0.047 pp (95% interval +0.014 to +0.091)
FLEURS utterances changed 16, each toward a vocabulary term
Words written on 11 clips of silence and noise 0 0
Warm p50, idle / under a 14-process CPU load +0.5% / +1.5%

Without terms, transcripts match v0.4.0 on all 1,422 utterances. Russian speech that says English terms the Russian way is not helped: the model writes them in Cyrillic, and biasing toward the Latin spelling found at most 2 of 34. The two earlier designs, and why they were dropped, are in the same report. One gave a bonus that could outscore blank and wrote "Postgres." on two seconds of silence. The other boosted first pieces and rewrote ordinary words.

Credit

Phrase boosting with a context graph follows icefall's ContextGraph and NeMo's phrase boosting work; this is the single-hypothesis greedy reduction. Thank you, CJ, for transcribe.cpp; OpenRamble's dictation runs on it.

🤖 Generated with Claude Code

Parakeet has no prompt slot, so it ignored transcribe_run_params::vocabulary
with a WARN. The TDT and RNN-T greedy decoders now take the terms as phrase
boosting over an Aho-Corasick automaton of token ids.

The model's own argmax decides between blank and a token; only when it emits
a token does the vocabulary re-rank the non-blank tokens, so biasing changes
which word is written and never whether one is. A term earns no bonus for its
first piece, a broken match gives back what it earned, and a completed term
keeps its bonus. Token confidences stay the model's own, and with no terms the
decoder never calls into the automaton, so output is bit-exact.

The bonus defaults to 8.0 and can be overridden with
TRANSCRIBE_VOCABULARY_BOOST. Streaming and CTC heads ignore terms for now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mikwiseman
mikwiseman requested a review from cjpais as a code owner October 20, 2026 06:28
@cjpais

cjpais commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Thanks. I'll take a look. I will check the implementation and see if this is reasonable to bring in or not as something that's supported by these family of models

@bunmi-oye

Copy link
Copy Markdown

+1, would love to see this merged. We transcribe legal recordings with Parakeet, and names and case terms are what it gets wrong most often ("Bates stamped" came out as "bait stamped"). Being able to pass a word list per job would fix a lot of that for us.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants