[chaofengw-nv]

Background

Fixes #1778. Native FP16 PersonaPlex adds audio codebook embeddings in order, then text, rounding each addition to Half. The runtime instead adds text first in FP32 and casts only the final sum. Identical token histories therefore produce different temporal inputs before any transformer computation.

Exit Criteria

• Preserve Native Half rounding and audio-before-text order for FP16 temporal engines, including text-prompt frames.
• Keep the existing FP32 path and bundle interface unchanged; retain all audio-parity gates.

Implementation

Select Half summation from the temporal engine's base cache dtype. Per-layer FP32 promotion retains this base cache contract. Use CUDA's host-only round-to-nearest-even Half conversions for each embedding value and addition; no GPU kernel or CUDA-runtime call is introduced. Both dual-stream and text-prompt frames use the same family-owned helper. Add the CUDA header include path only to the affected CPU test target.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

• CPU, with CUDA_VISIBLE_DEVICES empty: compile families/personaplex/tests/cpp/test_speech_temporal_embed_plan.cpp using g++ -std=c++17 -Wall -Wextra -Wpedantic -I. -isystem /usr/local/cuda/include, then execute the resulting binary: passed without linking cudart or executing GPU work. Covers Half ties/subnormals/overflow, rounding after every addition, audio-before-text cancellation, the preserved FP32 sum, and existing embedding/logit cases.
• CPU product control: sequential Half additions of [1, 0.000400066375732421875, -1] return zero after this change; the original FP32-sum path returns 0.000400066375732421875 before its final cast, reproducing the defect.
• GB300 component capture in the diagnosis pipeline: originally 1737/4096 effective Half input elements differ at step 0. With this helper, Native and TRTMC inputs are bit-identical at steps 0 through 28. A generated token still diverges at step 28 and whole-clip audio parity still fails (5.58 dB against a 3 dB limit). This is input-boundary evidence, not full-model acceptance.
• python -m tools.model_ci validate, python -m tools.test_impact --base github/main --validate, git diff --check, and clang-format 22 on changed C++ files: passed.

Hardware, Environment, and Revisions

• Head a8b1160451353a47a66883eae832d8ed4e05dc87; base 9643800b3b5a951382c87ae1ebfbbeb24b1a8090.
• CPU regression: x86-64 with CUDA 13 headers, GPU access disabled.
• Component capture: NVIDIA GB300, ARM64, CUDA 13, TensorRT 11.2.0.113; checkpoint nvidia/personaplex-7b-v1 at revision fdaf4090. The summation helper and C++ regression are byte-identical to the ones used in the diagnosis pipeline. The focused PR uses the first base-cache dtype instead of checking every identically typed base cache.

Not Run / Remaining Gaps

This PR alone has not passed full audio-parity acceptance, multi-GPU execution, or protected exact-head premerge. The capture pipeline includes separate resampler/Mimi fixes and does not qualify this entire PR head as an end-to-end pass. Remaining temporal/depth differences are being investigated without changing quality gates.

Contributor Self-Review

• I have completed a self-review of this change.

Reviewed the six-file family-local diff, prompt and ordinary-frame call sites, base-cache dtype contract, host-only Half conversions, unchanged FP32 path, test assertions, DCO, and the distinction between component proof and audio acceptance.

Notes For Future Readers

• This runtime fix applies to existing FP16 temporal bundles; no bundle-format migration is required. FP32 temporal bundles retain the existing path.
• fix(personaplex): align FP16 inference with Native #1751 tracks separate audio preprocessing/Mimi work; fix(personaplex): keep RMSNorm epsilon in FP32 #1780 preserves the decoder RMSNorm epsilon. Neither is included here or claimed to establish whole-model acceptance.

Risk level

• Low
• Medium
• High

Every FP16 temporal input now follows Native's arithmetic boundaries. Token trajectories can change even though token IDs and checkpoint weights are unchanged.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[chaofengw-nv]

Superseded by #1751. This change and its regression coverage are preserved in commit acca115 of the consolidated PR. Original issue links and acceptance criteria remain tracked there.