Skip to content

fix(personaplex): match Native HQ audio resampling - #1798

Closed
chaofengw-nv wants to merge 1 commit into
NVIDIA:mainfrom
chaofengw-nv:fix/personaplex-exact-audio-resampling-20261018
Closed

chaofengw-nv wants to merge 1 commit into
NVIDIA:mainfrom
chaofengw-nv:fix/personaplex-exact-audio-resampling-20261018

Conversation

@chaofengw-nv

@chaofengw-nv chaofengw-nv commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Background

Fixes #1749. Approximate audio resampling changes Mimi input codes and can redirect PersonaPlex generation. The bounded Kaiser filter in #1751's earlier revision 8f6eddc02 changes clip 18's codes at frame 23: the same engine passes the original speech gate with Native's resampled input (1.08 dB) and fails with the Kaiser input (3.78 dB).

Exit Criteria

  • Follow Native's HQ resampling recipe and librosa output length, including zero padding.
  • Keep the dependency within PersonaPlex without requiring development headers or a link-time libsoxr dependency for every model family.
  • Preserve the original requests, Native baseline, FP16 temporal/depth core and 3 dB speech gate.

Implementation

Load the system libsoxr.so.0 when resampling is needed. Its opaque C API uses soxr_create defaults (HQ, float32 I/O, one thread), consumes the input and flushes until drained or the output is full. Already-matching sample rates copy the input. A missing library produces an actionable error.

The family CI dependency image installs libsoxr0; the family README documents this separate LGPL-2.1-or-later runtime dependency. The wheel does not vendor it. There is no global CMake dependency search or global image change.

Change categories

  • Model or runtime behavior
  • Public API
  • ABI
  • Bundle or artifact format
  • Dependencies
  • Documentation only
  • CI or developer tooling

Validation

Commands and Results

  • python -m pytest -c /dev/null -o addopts= -p no:cacheprovider -q families/personaplex/tests/test_resampler_precision.py: same final test on both CPU controls. Kaiser revision 8f6eddc02 (before splitting fix(personaplex): align FP16 inference with Native #1751): 1 numerical failure, 5 passes, zero errors/skips. Final implementation: 6 passes, zero errors/skips. Local execution without libsoxr: identity/invalid-rate/missing-library assertions pass; 5 waveform cases explicitly skip.
  • Original 20 input waveforms: output lengths match Native; maximum absolute error is 7.152557e-7, not bit-exact. Final flush-loop outputs are bit-identical to the GPU-tested prototype on all 20 clips.
  • One-engine injection control: Native waveform passes clip 18 at 1.08 dB; Kaiser waveform fails at 3.78 dB and reproduces the earlier output digest. The engine SHA is identical in both runs.
  • Final published-code integration: 18/20, with clip 18 passing. Clip 7 fails at 5.58 dB and clip 11 at 6.42 dB under the unchanged gate, against both frozen and fresh same-batch Native FP16. Fresh Native PCM and output-frame counts match the frozen reference on all 20 requests. Exact input re-exposes clip 7, which had passed with altered Kaiser input codes. This is not whole-model acceptance.
  • Final same-batch Task P50: Native 2125.46 ms / TRTMC 1474.17 ms, 20 requests each; output sample counts and rates match on all requests (4,617,600 samples per side). The clean integration passes 7 family CTests, 6 resampler CPU tests and 7 RMSNorm-gain GPU tests. readelf -d confirms no libsoxr DT_NEEDED entry.
  • ruff check, clang-format --dry-run --Werror, git diff --check, python -m tools.model_ci validate, and python tools/test_impact.py --validate: passed.

Hardware, Environment, and Revisions

Base 9a8b545f5e8c22c653c8029c95a999dc070b4f3f; head b3b12608132d5935c264f4c0615e1dce25e1e470. GB300, Linux ARM64, CUDA 13, TensorRT 11.2.0.113, PyTorch 2.12.0+cu130. Native uses librosa 0.11.0, python-soxr 1.1.0 and bundled libsoxr 0.1.3-14-ga66f3ee. Checkpoint nvidia/personaplex-7b-v1@fdaf4090a61cb315c138a1faee287ffd6c716309; original 20 LibriSpeech requests.

Final runtime source SHA 723604cd...; final test SHA 583ad9a5.... Final GPU integration uses this runtime from a clean composition of the published PR heads, including #1751 decoder-only and #1800 gain-only. Its 1,528 production Python/CLI files match the builder source exactly, so the same engine bundle is explicitly reused by SHA. The temporal engine and all 16 depth engines remain Half; Mimi remains FP32.

Not Run / Remaining Gaps

Three-build model acceptance, current-head premerge, a rebuilt installed wheel and other hardware are unqualified. The system libsoxr runtime must be installed for resampling. The Native codec-policy decision and clip-7 core-numerics investigation remain separate; neither the Native baseline nor the speech tolerance was changed.

Contributor Self-Review

  • I have completed a self-review of this change.

Reviewed the C API signatures/defaults, loader lifetime, flush, length bounds, all PersonaPlex callers, dependency ownership and the same-test original/fixed controls. Independent review found no blockers after the flush and missing-library test improvements.

Notes For Future Readers

The PR scopes are now split: #1751 (head 8ae0f442) carries the Mimi FP32 decoder fix for #1750; this PR carries the resampler fix for #1749. They can merge independently. This branch starts from current main and contains no decoder-precision change. The GPU integration above used both fixes alongside the other pending PersonaPlex arithmetic fixes.

The public libsoxr header documents soxr_create's HQ defaults. soxr_oneshot defaults to LQ, so substituting it without an explicit HQ specification changes the signal.

Risk level

  • Low
  • Medium
  • High

Resampling changes conditioning for non-24 kHz audio and adds a family runtime dependency. Full PersonaPlex speech parity still fails on two requests.

Approximate input filters change Mimi codes and can redirect speech generation.
Load the system libsoxr runtime when resampling is needed and use its HQ recipe
with librosa output sizing. Keep the dependency and CI declaration family-local;
no soxr development headers or unconditional link dependency are required.

Add CPU waveform and length regressions and document the runtime requirement.
The original speech benchmark gate and Native baseline remain unchanged.

Fixes: NVIDIA#1749
Signed-off-by: chaofengw <chaofengw@nvidia.com>
@coderabbitai

coderabbitai Bot commented Oct 18, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@chaofengw-nv chaofengw-nv self-assigned this Oct 18, 2026
chaofengw-nv added a commit to chaofengw-nv/TensorRT-Model-Connect-fork that referenced this pull request Oct 18, 2026
Revert f8320af. The input resampler is now handled separately by NVIDIA#1798
(issue NVIDIA#1749), which loads the system libsoxr HQ resampler at run time.
This PR keeps only the FP32 Mimi decoder change for NVIDIA#1750, so the two
pull requests no longer carry competing resampler implementations.

This reverts commit f8320af.

Signed-off-by: chaofengw <chaofengw@nvidia.com>
chaofengw-nv added a commit to chaofengw-nv/TensorRT-Model-Connect-fork that referenced this pull request Oct 20, 2026
Use dynamically loaded libsoxr HQ with librosa-compatible output lengths for the Native-aligned input path.

Closes NVIDIA#1749
Refs NVIDIA#1798

Signed-off-by: chaofengw <chaofengw@nvidia.com>
@chaofengw-nv

Copy link
Copy Markdown
Collaborator Author

Superseded by #1751. This change and its regression coverage are preserved in commit 258c706 of the consolidated PR. Original issue links and acceptance criteria remain tracked there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Accuracy] PersonaPlex resamples input audio differently from the Native reference loader and changes the Mimi input codes

1 participant