Repository navigation
fix(personaplex): avoid timing discarded audio decode - #1783
Closed
chaofengw-nv wants to merge 2 commits into
Closed
chaofengw-nv wants to merge 2 commits into
chaofengw-nv wants to merge 2 commits into
Conversation
Decode each returned speech frame once in the Native benchmark. The auxiliary codec output was discarded but its decode was included in task latency; keep the existing official warmup and quality oracle unchanged. Add a CPU regression that verifies the returned waveform and counts work for a bounded generation request. Refs: NVIDIA#1782 Signed-off-by: chaofengw <chaofengw@nvidia.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
Keep this change on the active AIPerf reference path. The legacy benchmark entry point depends on the retired qualification harness and is outside this fix. Refs: NVIDIA#1782 Signed-off-by: chaofengw <chaofengw@nvidia.com>
This was referenced Oct 17, 2026
chaofengw-nv
added a commit
to chaofengw-nv/TensorRT-Model-Connect-fork
that referenced
this pull request
Oct 20, 2026
Decode each returned audio frame once while retaining the official warmup and accuracy oracle. Closes NVIDIA#1782 Refs NVIDIA#1783 Signed-off-by: chaofengw <chaofengw@nvidia.com>
Collaborator
Author
chaofengw-nv
added a commit
that referenced
this pull request
Oct 21, 2026
* fix(personaplex): avoid timing discarded audio decode Decode each returned audio frame once while retaining the official warmup and accuracy oracle. Closes #1782 Refs #1783 Signed-off-by: chaofengw <chaofengw@nvidia.com> * feat(personaplex): score spoken QA with VoiceBench Collect spoken QA accuracy and task timing from the same inference outputs using the pinned VoiceBench questions, official voice prefix and offline transcription. Preserve explicit unanswered and ambiguous responses and the paired statistical verdict. Signed-off-by: chaofengw <chaofengw@nvidia.com> * fix(personaplex): feed both sides the same 24 kHz audio VoiceBench and LibriSpeech questions are 16 kHz. Native resampled them with librosa (soxr HQ) while TRTMC used its own windowed-sinc resampler, so the two sides answered slightly different inputs. Resample each suite once with SciPy polyphase (audio_sample_rate: 24000) and make the Native reference reject audio it would have to resample. librosa no longer touches the model input; the speech-parity suite moves to version 2. Refs #1749 Signed-off-by: chaofengw <chaofengw@nvidia.com> --------- Signed-off-by: chaofengw <chaofengw@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Fixes #1782. PersonaPlex's Native benchmark decodes each emitted audio frame twice inside the measured task. Only the first result is returned; the auxiliary decode is discarded. This adds codec work to Native Task P50 that the TRTMC task does not perform.
Exit Criteria
Perform one acoustic decode per returned frame in the active AIPerf Native benchmark, preserve the official warmup and quality oracle, and verify the returned audio plus operation count with a CPU regression. Real GPU latency must be remeasured before using a new Native P50.
Implementation
Remove the unused auxiliary decode from the AIPerf reference adapter. Keep the auxiliary codec for the existing pinned official warmup, avoiding a separate warmup implementation. The official accuracy oracle, generation configuration, requests and quality gates are unchanged.
Change categories
Validation
Commands and Results
CUDA_VISIBLE_DEVICES='' OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 python -m pytest -c pyproject.toml -q -p no:cacheprovider families/personaplex/tests/test_reference_decode_work.py: 1 passed. The test runs the real adapter invocation with deterministic codec/generator stand-ins, checks the returned waveform and two-frame budget, and requires two primary decodes and zero discarded decodes.2 != 0). Returned audio assertions already pass on the original path.model_call_ms, n=20 each): FP16 2106.749 ms, FP32 3278.766 ms. Each run emitted 2405 frames in total. Historical controls are retained separately; these timings retain the original numerical policy.ruff check --config ruff.tomlon both changed files,python -m tools.model_ci validate,python -m tools.test_impact --base github/main --validate, andgit diff --check: passed.Hardware, Environment, and Revisions
0e0ea393ee9649597da6bef04c76285acf0af6ee; base9643800b3b5a951382c87ae1ebfbbeb24b1a8090.fdaf4090a61cb315c138a1faee287ffd6c716309. The exercised adapter SHA2565cf0414991d59e94eea0c47edff1b72d20a0554a4b55ec1e23f67f9918667fedmatches this head.Not Run / Remaining Gaps
Protected exact-head premerge, other platforms, and repeated latency runs are not qualified. The fresh timings are measurements from one full 20-request run at each precision; no confidence interval or full-model TRTMC accuracy claim is made. Historical Native P50 values include the extra decode and remain labeled separately. Numerical-precision alignment is separate work.
Contributor Self-Review
Reviewed the measured call site, dataflow of the discarded value, the unchanged official warmup and accuracy oracle, the two-file family-local scope, CPU negative control, test assertions, and DCO.
Notes For Future Readers
This corrects benchmark work accounting rather than optimizing the model. Compare newly measured Native timings with TRTMC only after confirming the original requests and real output behavior. Do not rewrite historical measurements or interpret the CPU stand-ins as full-model accuracy proof.
Risk level
The removed result has no consumer, and the auxiliary codec does not feed the generator or returned waveform. Original-request GPU replay confirms identical PCM for both exercised precisions.