[chaofengw-nv]

Background

PersonaPlex's FP16 execution differs from the Native reference at several input and arithmetic boundaries. These related fixes previously required separate PRs with overlapping decoder and runtime changes. This PR consolidates nine of them into nine DCO-signed commits, with one issue mapping per defect.

Exit Criteria

• Keep the temporal/depth core, weights and KV caches in FP16 while preserving the reference's explicit FP32 operations.
• Match Native embedding, normalization, RoPE, gating, projection and residual rounding boundaries.
• Use FP32 Mimi decoding.
• Preserve existing model-validation thresholds and the existing E2E reference configuration.

Implementation

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Original PR	Issue	Preserved fix
#1751	#1750	Build the Mimi decoder in FP32.
#1780	#1779	Keep RMSNorm epsilon in FP32.
#1781	#1778	Preserve Half embedding addition order and rounding.
#1789	#1788	Preserve Half depth-input projection and addition boundaries.
#1791	#1790	Compute RoPE in FP32 before restoring the input dtype.
#1794	#1793	Round the SwiGLU gate product once.
#1795	#1786	Remove FP32 component promotion from the declared FP16 profile.
#1797	#1796	Preserve projection-output rounding before residual addition.
#1800	#1799	Round RMSNorm scale weights to the model dtype.

Changes stay within the PersonaPlex family. The only patch-composition conflict was adjacent additive CMake test declarations; both the Half embedding include path and depth-kernel test target are retained.

The input-resampling fix from #1798 is no longer part of this PR. It made the runtime dlopen the system libsoxr (LGPL-2.1-or-later) only to reproduce the reference harness's librosa resampling, which the official PersonaPlex loader does not use. Instead, #1809 resamples each AIPerf speech suite once, so Native and TRTMC read the same 24 kHz samples. The runtime keeps its existing windowed-sinc resampler for other input rates and adds no dependency.

Validation

Commands and Results

Current head f2cf1ae596cc660005932538b357d912fe074933, rebased onto 682d84862:

• git range-diff 06debb09e..93d1acb1b 682d84862..f2cf1ae59: all nine retained commits are unchanged; only the libsoxr resampling commit was dropped. The PersonaPlex tree equals the previous head minus that commit's six files.
• CPU, CUDA_VISIBLE_DEVICES='': python -m pytest tools/tests/test_architecture.py tools/tests/test_family_impact.py -q: 77 passed. python -m tools.model_ci validate and python tools/test_impact.py --validate: valid; test_impact selects personaplex only, with core tests.
• git diff --check: passed.

Previous head 93d1acb1b (the same nine commits plus the dropped one):

• PYTHONPATH=core/builder:apps/benchmark:. python -m pytest families/personaplex/tests/test_fp16_decoder_norm_epsilon.py families/personaplex/tests/test_rmsnorm_weight_dtype.py families/personaplex/tests/test_rotary_precision.py families/personaplex/tests/test_swiglu_precision.py families/personaplex/tests/test_residual_precision.py families/personaplex/tests/test_e2e.py -k 'not official_checkpoint_e2e' -q, run together with the now-removed resampler test: 41 passed, one of them a resampler check. Three full-model E2E cases were deselected.
• Repository CMake configure and build: the actual PersonaPlex runtime and all seven family C++ test targets built successfully. ctest -R '^test_personaplex_' --output-on-failure: 7/7 passed, including the CUDA depth-input regression, on the combined model/benchmark tree.
• ruff check --config ruff.toml on all changed Python files, clang-format 22.1.8 on changed native files, independent runtime syntax checks on both PR heads, and git diff --check: passed.

Hardware, Environment, and Revisions

• Current head f2cf1ae596cc660005932538b357d912fe074933; base 682d84862. CPU checks only.
• Previous head 93d1acb1b049bb5e7a9d9177c48646328f715972; base 06debb09eda99887580ac6264da4c31ceec8f559. Local NVIDIA A30; Python 3.12, PyTorch 2.12.0+cu130, TensorRT 10.16.1.11; CMake used CUDA 13.1 and SM80. Focused tests exercise the original FP16 rounding boundaries; this is not checkpoint-level parity evidence.

Not Run / Remaining Gaps

The GPU/TRT precision tests have not been rerun on the rebased head, and the repository pins a newer TensorRT than the local run. Three full-model E2E cases were not run. Full-model FP16 acceptance, current-head premerge CI and GB300 accuracy/performance remain pending.

Contributor Self-Review

• I have completed a self-review of this change.

Reviewed the composed diff, family boundaries, DCO history, issue mappings and recorded validation. Full-model acceptance remains open.

Notes For Future Readers

Risk level

• Low
• Medium
• High

The arithmetic and input fixes affect generated tokens and speech; full-model parity has not yet been established.

• Merge this PR before feat(personaplex): qualify spoken QA with AIPerf #1809. The two PRs target main with separate commit sets; feat(personaplex): qualify spoken QA with AIPerf #1809 contains benchmark plumbing and Native timing only.
• Each superseded PR's implementation and regression coverage remain traceable through the table and individual commits. Historical test results are not presented as exact-head CI.
• The existing full E2E testcase still declares a BF16 oracle. The separate benchmark baseline uses FP16; its acceptance remains under investigation.
• Full-model validation and current-head premerge CI remain pending. Keep this PR in draft during that work.

Closes #1750
Closes #1779
Closes #1778
Closes #1788
Closes #1790
Closes #1793
Closes #1796
Closes #1799
Refs #1786

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/TensorRT-Model-Connect/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 206b0355-53eb-43f3-baa2-3da007f7050c

📥 Commits

Reviewing files that changed from the base of the PR and between 682d848 and f2cf1ae.

📒 Files selected for processing: 21

• families/personaplex/decoder_tp_builder.py
• families/personaplex/default_decoder.py
• families/personaplex/default_dual_profile_decoder.py
• families/personaplex/graph_blocks.py
• families/personaplex/graph_ops.py
• families/personaplex/model.py
• families/personaplex/runtime/CMakeLists.txt
• families/personaplex/runtime/pipeline.cpp
• families/personaplex/runtime/plugin.cpp
• families/personaplex/runtime/sampling_kernels.cu
• families/personaplex/runtime/sampling_kernels.h
• families/personaplex/runtime/speech_config.h
• families/personaplex/runtime/speech_temporal_embed_plan.h
• families/personaplex/tests/cpp/test_depth_embedding_kernel.cpp
• families/personaplex/tests/cpp/test_speech_temporal_embed_plan.cpp
• families/personaplex/tests/manifests/personaplex-7b.json
• families/personaplex/tests/test_fp16_decoder_norm_epsilon.py
• families/personaplex/tests/test_residual_precision.py
• families/personaplex/tests/test_rmsnorm_weight_dtype.py
• families/personaplex/tests/test_rotary_precision.py
• families/personaplex/tests/test_swiglu_precision.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

📝 Summary

Summary

Aligns PersonaPlex FP16 inference with Native precision boundaries:

• Builds Mimi decoding in FP32.
• Preserves FP32 RMSNorm epsilon and model-dtype RMSNorm weights.
• Restores Half rounding for temporal and depth embeddings.
• Performs RoPE and SwiGLU intermediate calculations in FP32.
• Rounds projection outputs before residual addition.
• Removes FP32 component promotion from the FP16 profile.
• Adds targeted CPU, C++, and GPU/TensorRT regression tests.
• Adds no external dependency and keeps the existing resampler.

Architecture impact

All changes remain within families/personaplex. The PR changes PersonaPlex decoder builders, graph helpers, runtime embedding and depth kernels, configuration, manifests, and tests.

New or changed family-owned surfaces include:

• SpeechConfig.temporal_embed_half
• personaplex_prepare_depth_embedding(..., DType depth_dtype, ...)
• add_swiglu_activation
• add_residual_sum
• Half-precision temporal embedding helpers

No cross-family implementation dependency or central registry change is shown. CUDA runtime linkage is added for the PersonaPlex GPU test target only.

Validation

• PASS — 77 architecture and family-impact tests passed on the rebased CPU head.
• PASS — Historical focused validation reported 41 Python tests and 7/7 PersonaPlex C++ tests passing.
• HUMAN REVIEW REQUIRED — GPU/TensorRT precision tests were not rerun on the rebased head.
• HUMAN REVIEW REQUIRED — Full-model FP16 acceptance, current-head premerge CI, GB300 accuracy/performance, and three deselected E2E cases remain pending.

Review outcome

HUMAN REVIEW REQUIRED

Component-level validation passed, but full-model and current-head GPU validation remain unresolved.

This PR updates precision handling in PersonaPlex decoder graphs and runtime embedding preparation. It changes RoPE, RMSNorm, SwiGLU, residual, temporal-embedding, and depth-input calculations, and sets Mimi decoder precision to FP32. It adds regression tests and updates the 7B manifest.

Changes: PersonaPlex precision alignment

Layer / File(s)	Summary
Shared graph precision operations families/personaplex/graph_ops.py, families/personaplex/graph_blocks.py	The shared helpers perform residual, RMSNorm, and RoPE calculations in FP32 where described, then restore the relevant dtype. The new SwiGLU helper computes the gated activation in FP32 for non-FP32 inputs and returns the result in the original dtype.
Decoder integration and precision tests families/personaplex/decoder_tp_builder.py, families/personaplex/default_decoder.py, families/personaplex/default_dual_profile_decoder.py, families/personaplex/model.py, families/personaplex/tests/manifests/personaplex-7b.json, families/personaplex/tests/test_*.py	The decoder builders use shared residual and SwiGLU helpers, and create RoPE and RMSNorm epsilon constants as FP32. Mimi decoder precision is set to FP32, and the 7B manifest’s fp32_layers list is empty. New GPU tests cover normalization, residual, RoPE, and SwiGLU precision.
Temporal embedding precision families/personaplex/runtime/speech_config.h, families/personaplex/runtime/speech_temporal_embed_plan.h, families/personaplex/runtime/plugin.cpp, families/personaplex/runtime/pipeline.cpp, families/personaplex/tests/cpp/test_speech_temporal_embed_plan.cpp, families/personaplex/runtime/CMakeLists.txt	The runtime sets temporal_embed_half from the temporal cache dtype. When enabled, embedding values and each audio-then-text addition are rounded to FP16. Tests cover half rounding and accumulation order.
Depth embedding precision families/personaplex/runtime/sampling_kernels.h, families/personaplex/runtime/sampling_kernels.cu, families/personaplex/runtime/pipeline.cpp, families/personaplex/tests/cpp/test_depth_embedding_kernel.cpp, families/personaplex/runtime/CMakeLists.txt	Depth input preparation receives the depth cache dtype and rounds projection operands, accumulated projection, seed, and output to that dtype. The added CUDA test checks FP16 and FP32 results.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to f2cf1

No identified defect currently blocks merging; GPU and full-model validation remain pending.

Pre-merge checks: 8 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 38.98% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 59 functions across 19 files. (2 skipped:…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 8 passed

Check name	Status	Explanation
Title check		The title clearly summarizes the primary change: aligning PersonaPlex FP16 inference behavior with the Native reference.
Description check		The description covers the required background, exit criteria, implementation, change category, validation results, environment, remaining gaps, self-review, risk, and rollout notes. It also identifie…
Linked Issues check		The PR implements the coding objectives in [#1751], [#1750], [#1779], [#1778], [#1788], [#1790], [#1793], [#1796], and [#1799]. The Mimi decoder uses FP32. RMSNorm preserves FP32 epsilon and rounds no…
Out of Scope Changes check		The changes stay within the PersonaPlex family and support the linked precision fixes. Added runtime helpers, CMake declarations, manifests, and regression tests directly support the temporal embeddin…
Family Ownership Boundary		No cross-family dependency was introduced. The authoritative diff changes 21 files, and every changed file is under families/personaplex. New Python dependencies are PersonaPlex-local modules, such …
Shared Semantic Neutrality		The authoritative diff contains no changed shared file outside the excluded scopes. All changed Python files are under families/personaplex and implement PersonaPlex-owned graph/model behavior. All …
Benchmark Validation Integrity		No benchmark accounting or validation contract changed. The PR changes PersonaPlex execution precision and adds component correctness tests; it does not change timing, task units, aggregation, seriali…
Shared Change Blast Radius		The authoritative diff changes only families/personaplex/**. It does not modify shared framework code, cross-family contracts, catalogs, examples, benchmarks, or repository-wide validation infrastru…

Full details: Description check

Explanation

The description covers the required background, exit criteria, implementation, change category, validation results, environment, remaining gaps, self-review, risk, and rollout notes. It also identifies linked issues and clearly states pending validation.

Full details: Linked Issues check

Explanation

The PR implements the coding objectives in [#1751], [#1750], [#1779], [#1778], [#1788], [#1790], [#1793], [#1796], and [#1799]. The Mimi decoder uses FP32. RMSNorm preserves FP32 epsilon and rounds non-FP32 weights to the model dtype. Temporal and depth embedding paths apply the required Half rounding boundaries. RoPE uses FP32 tables and rotation before restoring the input dtype. SwiGLU and residual paths preserve the required rounding boundaries. The FP16 profile removes FP32 component promotion. The reported focused regressions, architecture tests, and C++ tests cover these changes. The PR leaves thresholds and E2E settings unchanged. The linked component issues do not require full-model parity, and the PR records the remaining GPU and full-model validation gaps.

Full details: Out of Scope Changes check

Explanation

The changes stay within the PersonaPlex family and support the linked precision fixes. Added runtime helpers, CMake declarations, manifests, and regression tests directly support the temporal embedding, depth input, decoder precision, RoPE, SwiGLU, RMSNorm, and residual objectives. The PR does not include the excluded runtime resampling or libsoxr change. No unrelated change is demonstrated.

Full details: Family Ownership Boundary

Explanation

No cross-family dependency was introduced. The authoritative diff changes 21 files, and every changed file is under families/personaplex. New Python dependencies are PersonaPlex-local modules, such as graph_blocks, graph_ops, and default_decoder (for example, families/personaplex/decoder_tp_builder.py:20-31 and default_decoder.py:27-38). The new test target uses PersonaPlex sources plus shared CUDA and core runtime headers (families/personaplex/runtime/CMakeLists.txt:89-102), not another family. No central family registry, switch, strategy map, or file outside the PersonaPlex family changed.

Full details: Shared Semantic Neutrality

Explanation

The authoritative diff contains no changed shared file outside the excluded scopes. All changed Python files are under families/personaplex and implement PersonaPlex-owned graph/model behavior. All other changes are under families/personaplex/runtime or families/personaplex/tests, which this check excludes. The new add_swiglu_activation and add_residual_sum helpers remain in the PersonaPlex-owned graph_blocks.py and graph_ops.py; no shared code receives model-specific semantics.

Full details: Benchmark Validation Integrity

Explanation

No benchmark accounting or validation contract changed. The PR changes PersonaPlex execution precision and adds component correctness tests; it does not change timing, task units, aggregation, serialization, metrics, or thresholds. families/personaplex/tests/test_e2e.py, reporting.py, tests/thresholds/personaplex-7b.json, and tests/benchmark/prepare_environment.py are unchanged. The only manifest change removes FP32 layers, while the existing E2E path still consumes the manifest and applies the same thresholds. New GPU checks synchronize TRT execution before host-side exact output validation; they do not report benchmark timings or compare different timed regions.

Full details: Shared Change Blast Radius

Explanation

The authoritative diff changes only families/personaplex/**. It does not modify shared framework code, cross-family contracts, catalogs, examples, benchmarks, or repository-wide validation infrastructure. Searches found no references to the changed PersonaPlex symbols outside that family. The changed helpers, SpeechConfig field, and depth-kernel signature are consumed only by PersonaPlex code and tests, so the shared-change blast-radius condition does not apply.

Full details: Docstring Coverage

Explanation

Docstring coverage is 38.98% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 59 functions across 19 files. (2 skipped: 2 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}