Repository navigation
parakeet: honor run_params.vocabulary with phrase boosting - #221
Open
mikwiseman wants to merge 1 commit into
Open
mikwiseman wants to merge 1 commit into
mikwiseman wants to merge 1 commit into
Conversation
Parakeet has no prompt slot, so it ignored transcribe_run_params::vocabulary with a WARN. The TDT and RNN-T greedy decoders now take the terms as phrase boosting over an Aho-Corasick automaton of token ids. The model's own argmax decides between blank and a token; only when it emits a token does the vocabulary re-rank the non-blank tokens, so biasing changes which word is written and never whether one is. A term earns no bonus for its first piece, a broken match gives back what it earned, and a completed term keeps its bonus. Token confidences stay the model's own, and with no terms the decoder never calls into the automaton, so output is bit-exact. The bonus defaults to 8.0 and can be overridden with TRANSCRIBE_VOCABULARY_BOOST. Streaming and CTC heads ignore terms for now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Thanks. I'll take a look. I will check the implementation and see if this is reasonable to bring in or not as something that's supported by these family of models |
|
+1, would love to see this merged. We transcribe legal recordings with Parakeet, and names and case terms are what it gets wrong most often ("Bates stamped" came out as "bait stamped"). Being able to pass a word list per job would fix a lot of that for us. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
transcribe_run_params::vocabularyhas existed since 0.3, but Parakeet ignores it with a WARN because it has no prompt slot. This change makes the TDT and RNN-T greedy decoders take the terms as phrase boosting. Every Parakeet user (v3, Ultra and the rest of the family) can then bias recognition toward names and jargon through the existing API, with no new public symbol.How it works
src/transcribe-phrase-boost.{h,cpp}). Every transition is resolved through the failure links once, at build time.TRANSCRIBE_VOCABULARY_BOOSToverrides it. Streaming and CTC heads ignore terms for now, anddocs/prompting.mdsays so.TRANSCRIBE_FEATURE_VOCABULARYfor TDT and RNN-T heads.Tests
tests/phrase_boost_unit.cpp. It checks that a completed term keeps its bonus, that a broken match gives back exactly what it earned, that a suffix completing another term keeps that term through the failure link, and thatapplymatchesdeltafor every token while leaving blank alone.ctest: 60 of 60 pass on macOS (Apple M4 Pro, Metal);transcribe_cli_diarize_smokeis skipped as usual.transcribe-cliwith no terms gives byte-identical text to v0.4.0 on 12 synthetic term sentences. With 60 terms, 7 of the 12 change.Measurements
These come from OpenRamble's report, which used the same change on v0.4.0 with Parakeet v3 Q8, Metal and one decoder thread. The rules were preregistered before the code existed.
Without terms, transcripts match v0.4.0 on all 1,422 utterances. Russian speech that says English terms the Russian way is not helped: the model writes them in Cyrillic, and biasing toward the Latin spelling found at most 2 of 34. The two earlier designs, and why they were dropped, are in the same report. One gave a bonus that could outscore blank and wrote "Postgres." on two seconds of silence. The other boosted first pieces and rewrote ordinary words.
Credit
Phrase boosting with a context graph follows icefall's
ContextGraphand NeMo's phrase boosting work; this is the single-hypothesis greedy reduction. Thank you, CJ, for transcribe.cpp; OpenRamble's dictation runs on it.🤖 Generated with Claude Code