[wOvAN]

What

Moves the Piper duration encoder (text encoder + duration discriminator) onto the NPU, next to the already-merged RKNN decoder. The voice's own enc_p.proj/dp.proj seam is sliced, padded to fixed token widths, constant-folded, compiled per-bin, and served from a sidecar (rknn_duration_encoder.{h,cpp}) selected through the existing piper_tts.exec_plan option:

{"decoder":{"impl":"rknn","artifacts":{"main":"/root/tts-rknn/piper-lessac-w93.rknn"}},
 "duration_encoder":{"impl":"rknn","artifacts":{"main":"enc_n128.rknn,enc_n192.rknn"}}}

The sidecar walks the sorted ladder to the smallest engine >= token_count; chunks above the widest engine fall back to the ggml encoder. Default ({}) is unchanged, no model_spec or catalog changes.

Why the graph rewrite is the core of this PR

The first ladder (slice + fold, no rewrite) compiled and ran — and computed garbage: 9.9–16 dB against fp32 on the real NPU for every bin, while the PC simulator said 55.5–61.4 dB. Bisecting per-layer: one attn_layers.0 alone lands at 0.4 dB on the NPU and 56 dB in the simulator; a probe over just that layer's rel-value chain returns 506 NaN and fp16-saturated ±65504 values exactly in the last row's window (the flatten-Pad tails are filled with uninitialized memory by the kernel lowering), and the rel-logit chain probe segfaults the runtime (rc=139). Standard Conv/MatMul/Softmax probes are fine; the breakage is exactly the unrolled per-row shift chains (Pad + Reshape + Slice, N copies) that the padded fixed-width fold leaves in every attention layer.

So the converter now rewrites the two chains into exact constant-index gathers (matrix_form_encoder), verified identical (maxdiff 0.0 numerically, 121–131 dB fp32 gate against the raw dynamic slice):

bias[b,h,w,t] = rel_logits[b,h,w,(N-1)+t-w]                    (GatherElements, axis=3)
rel_out[w]    = Σ_j probs[b,h,w,j] · rel_v[(N-1)+j-w]    (Gather + Mul + ReduceSum)

The pad-safety self-check runs on the rewritten graph, so the existing gate protects the surgery too.

Measured, RK3588 board (en_US-lessac-medium, 557-char RU/EN text, --text-chunk-size 60, 8 chunks of 109–459 tokens)

Engine fidelity vs fp32 (stats / conditioning, n ∈ {17, 61, N−1, N}):

bin	SNR dB	engine size
128	45.5–48.6 / 40.1–44.8	137 MB
192	45.1–48.0 / 39.5–44.3	402 MB
384	47.1–48.4 / 43.0–44.5	2.9 GB
512	—	converter OOMs at 31 GB RAM

The gathers' index initializers are constants, so the toolkit folds them into per-row Toeplitz expansions — O(N³) weight bytes. Narrow bins only; the docs say so.

Runtime, duration stage:

chunk tokens	ggml	NPU bin
109/115/111	84/89/85 ms	71/63/65 ms (bin 128: 1.25–1.35x faster)
285/323/373	440/561/758 ms	688/773/688 ms (bin 384: 0.6–1.1x, slower)
435/459	1081/1311 ms	fallback (above ladder)

Quality: the fp16 duration noise sums across a chunk, so drift grows with width — latent frames move +7/+8/−6 % on bin-128 chunks (the scale of the flow's own seeded sampling: ggml-vs-ggml reruns at the same seed are byte-identical, so this is the engine) against −21…−23 % on the 384-bin chunks — a systematic ~25 % tempo shift. Whisper (small) transcribes the baseline and ladder renders to the same text. That is why the recommended ladder is {128, 192} with the ggml fallback above 192 tokens: the fallback is a quality gate, not a convenience.

Reproduce

# PC, converter venv (onnx + onnxruntime for the slice/fold/check; rknn-toolkit2 for the build)
python tools/rknn/prepare_piper_tts_rknn_vocoder.py \
  --voice-onnx en_US-lessac-medium.onnx --encoder --frames 128   # ...192 for the second bin

Board matrix used above: run b = RKNN decoder only, run c = decoder + duration_encoder ladder; per-chunk duration_ms/latent_frames come from --log TIMING lines. The board probe (librknnrt directly, float pass_through inputs) is what gates this export — the PC simulator is explicitly not a gate for graphs containing these chains, per the docs section added here.

Notes

• Slice determinism: all three piper scales scalars feed sampling, the duration ceiling and the discriminator's flow noise — strictly downstream of the seam.
• Engines are per voice (validated against the loaded voice's channel count at init, same contract as the decoder engine).
• No defaults move: exec_plan stays opt-in; ctest suites are unaffected.

[0xShug0]

@wOvAN Thanks! All three PRs merged.