[chaofengw-nv]

Background

Fixes #1788. When the PersonaPlex depth transformer runs in FP16, the reference builds each depth step's input as
depformer_in[cb](hidden) with a Half F.linear, rounds that output to Half, and then adds the Half previous-token
embedding. The runtime's depth-input helper (prepare_depth_embedding_kernel) instead summed FP32 copies of the BF16
tables in FP32, and the depth engine applied a single Half rounding afterwards.

As a result, the FP16 depth input differed from the reference in every depth step, even when the temporal hidden state
and the previous token were identical.

Exit Criteria

• For FP16 depth engines, the helper rounds at the reference's boundaries: the weights and hidden values are rounded to
  Half, the projection output is rounded to Half, and the embedding plus the sum are rounded to Half.
• For FP32 depth engines, the old FP32 formula is unchanged.
• A GPU regression test checks a predeclared binary counterexample with exact equality.
• Non-goal: this PR does not claim that the full-model audio-parity gate passes; see the remaining gaps below.

Implementation

• sampling_kernels.cu gains round_to_depth_dtype, which applies Half or BF16 rounding, or none for FP32. The kernel
  applies it to each projection operand, to the reduced projection, to the seed embedding, and to their sum. The launcher
  takes the depth dtype as a new parameter.
• SpeechPipeline::prepare_depth_input passes each codebook engine's own cache_k_0 dtype. The depth engine computes in
  its KV-cache dtype, so this is the dtype of the reference's depth F.linear and add.
• New test test_personaplex_depth_embedding_kernel (CTest label gpu; it exits with skip code 77 when no GPU is
  present). It uses hidden [1, 1], projection row [1, 2^-12] and embedding -1, all exact in BF16 and Half. The
  reference result is 0, while the old formula gives 2^-12. The test also asserts that FP32 depth still returns 2^-12.
• This change touches an existing family helper kernel, which is migration debt under the architecture rules. It is the
  minimal correction to that kernel. No new kernels, tactic choices or GPU-specific branches are added.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

• Isolated tree of this head's sources on main 88ceb9f: cmake -DTRTMC_BUILD_TESTS=ON, built
  trtmc_model_personaplex and the 7 PersonaPlex C++ tests, then ran ctest -R ^test_personaplex_. Result: 7/7 passed,
  including the new GPU test (0.62 s). The tested source hashes equal this commit's files.
• Product kernel control: the helper source from main 88ceb9f and from this change, each compiled as-is, called on
  the same inputs.
  • Counterexample: original 2^-12 for codebook 0 and codebook 1, so it fails; fixed 0 and 0, so it passes; FP32 depth
    with the fix gives 2^-12, unchanged.
  • Actual reference FP16 capture of audio-parity clip 7, steps 0-39 and all 16 codebooks (640 depth steps), using the
    reference's own temporal hidden state and previous token (recovered uniquely in all 640 steps).
    • The original helper's Half depth input differs from the reference depformer input in 640/640 steps: 147,457 of
      655,360 elements.
    • With this change: 2,382 elements, 1-11 per step.
  • The remaining elements are not bitwise. The reference projection is a cuBLAS Half GEMM, which torch reproduces
    bitwise, while the helper keeps its own FP32 reduction order. That order is the likely cause, but it is unverified.
• git diff --check: clean. clang-format --dry-run on the changed C++ and CUDA files: clean.

Hardware, Environment, and Revisions

GB300 (SBSA), CUDA 13.0, TensorRT 11.2. Checkpoint nvidia/personaplex-7b-v1 revision
fdaf4090a61cb315c138a1faee287ffd6c716309 (BF16 weights). The reference ran FP16 with the pinned PersonaPlex reference
environment.

Not Run / Remaining Gaps

• Full model: with this change plus an FP32 RMSNorm epsilon in a true-FP16 build, the audio-parity clip 7 still fails
  against the reference FP16 baseline (5.58 dB). In that run the generated model audio tokens follow the reference FP32
  run through all 197 steps. The remaining difference is under investigation in the temporal and depth paths. This PR
  fixes one confirmed component boundary only.
• The full-model runs used the earlier form of this change, which read one depth dtype (the first codebook engine's)
  for all codebooks. That form is equivalent to this head only when every depth engine's cache is Half; this has been
  recorded for the first two depth engines, and recording all 16 is pending. The per-codebook form in this head was
  validated by the CTest run above.
• No CI results yet for this head.

Contributor Self-Review

• I have completed a self-review of this change.

Notes For Future Readers

• The depth dtype comes from each codebook engine's cache_k_0 input, not from the temporal hidden dtype. The temporal
  output can be FP32 while the depth runs in Half.
• If the helper projection moves into the depth engine later, it should keep these Half boundaries.

Risk level

• Low
• Medium
• High

Medium: this changes FP16 depth numerics for every generated frame toward the reference semantics. FP32 builds are
unchanged.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[chaofengw-nv]

Superseded by #1751. This change and its regression coverage are preserved in commit ef71a18 of the consolidated PR. Original issue links and acceptance criteria remain tracked there.