piper_tts: RKNN duration-encoder ladder

CMakeLists.txt

@@ -1119,6 +1119,7 @@ audiocpp_add_model(piper_tts
         src/community_models/piper_tts/frontend.cpp
         src/community_models/piper_tts/vits_runtime.cpp
         src/community_models/piper_tts/rknn_decoder.cpp
+        src/community_models/piper_tts/rknn_duration_encoder.cpp
         src/community_models/piper_tts/session.cpp
     INCLUDES
         engine/community_models/piper_tts/session.h

docs/build/rockchip-npu.md

@@ -184,6 +184,41 @@ board — `Resize` does not lower to a target the board runtime accepts, so the
 `ConvTranspose1d` of all ones (exact nearest, and the same operator the decoder already runs on the
 NPU).
 
+### Piper duration encoder
+
+The same script with `--encoder` compiles the other half — the text encoder plus the duration
+discriminator — for a fixed phoneme-token width: it slices at `enc_p.proj` (the 2x inter_channels
+prior stats) and `dp.proj` (the flow conditioning), the two arrays the runtime's flow graphs and
+length regulator consume, right-pads tokens to `--frames` and compiles one engine per width. The
+slice is deterministic (all three piper `scales` scalars feed sampling, the duration ceiling and
+the discriminator's flow noise, strictly downstream of the seam), and piper's exported masks are
+pad-safe: a padded run reproduces the exact-length run on the real frames at 117-130 dB fp32,
+checked by the self-check on the graph the engine is compiled from. `scales` is not an input.
+
+```bash
+<rknn-venv>/bin/python tools/rknn/prepare_piper_tts_rknn_vocoder.py \
+  --voice-onnx en_US-lessac-medium.onnx --encoder --frames 128
+```
+
+With the frame axis pinned the graph constant-folds (992 to 463 nodes at the BASIC level), and
+what remains in every attention layer is a relative-shift chain: `Pad + Reshape + Slice` shifting
+the [N, 2N-1] relative-position slabs by one step per query row, unrolled N times. The RKNN
+kernels compute exactly those chains as garbage on the board — probes show the flatten-Pad tails
+filled with uninitialized memory (NaN, ±65504), one attention layer alone drags the engine to
+0.4 dB end to end, and a chain probe segfaults the runtime — while the PC simulator computes the
+same graph at 56 dB. The simulator is therefore not a gate for this export; a board probe is. The
+converter rewrites the chains into exact constant-index gathers (rel-logit bias via
+`GatherElements`, rel-value sum via `Gather + Mul + ReduceSum`, verified 121-131 dB fp32 against
+the raw slice), the engine computes 40-48 dB on the NPU, and the fold self-checks run on the
+rewritten graph.
+
+The gathers cost O(N^3) weight bytes: the index initializers are constants, so the toolkit folds
+them into per-row Toeplitz expansions — 137 MB at 128 tokens, 402 MB at 192, 2.9 GB at 384, and
+512 exhausts 31 GB of converter RAM. Compile narrow bins only. The sidecar walks the sorted ladder
+to the smallest engine that fits the token count and falls back to the ggml encoder above the
+widest: the wide widths are where the fp16 duration drift hurts most (see the measurements
+below), so the fallback is a quality gate, not a convenience.
+
 ### Inflect v2 (Micro/Nano) decoder
 
 `tools/rknn/prepare_inflect_v2_rknn_decoder.py` slices the official `decode.onnx` at the same kind
@@ -381,7 +416,8 @@ the configurable components:
 | Component | Option | Implementations | Artifact roles |
 |---|---|---|---|
 | Piper `decoder` | `piper_tts.exec_plan` | `ggml`, `rknn` | RKNN: required `main` |
-| Piper `duration_encoder` / `duration_flows` | `piper_tts.exec_plan` | `ggml` | None (`backend` may name `cpu`) |
+| Piper `duration_encoder` | `piper_tts.exec_plan` | `ggml`, `rknn` | RKNN: required `main` (comma-separated ladder of per-width engines) |
+| Piper `duration_flows` | `piper_tts.exec_plan` | `ggml` | None (`backend` may name `cpu`) |
 | Kokoro `generator` | `kokoro_tts.exec_plan` | `ggml`, `rknn` | RKNN: required `main` |
 | Qwen3 `speech_decoder` | `qwen3_tts.exec_plan` | `ggml`, `rknn` | RKNN: required `main`, optional `first` |
 | sanoTTS `decoder` | `sanotts.exec_plan` | `default`, `ggml`, `neon` | None |
@@ -458,6 +494,14 @@ The same option pins the duration stages to the host, with the decoder untouched
   --session-option 'piper_tts.exec_plan={"duration_encoder":{"backend":"cpu"},"duration_flows":{"backend":"cpu"}}'
 ```
 
+and it moves the duration encoder to the NPU, over a comma-separated ladder of per-width engines
+(the same producer as the decoder, `--encoder`; short chunks pick the narrow engine, chunks wider
+than the ladder stay on the ggml encoder):
+
+```bash
+  --session-option 'piper_tts.exec_plan={"decoder":{"impl":"rknn","artifacts":{"main":"/root/tts-rknn/piper-lessac-w93.rknn"}},"duration_encoder":{"impl":"rknn","artifacts":{"main":"/root/tts-rknn/enc/encoder_n128.rknn,/root/tts-rknn/enc/encoder_n192.rknn"}}}'
+```
+
 The engine is per voice: a `.rknn` compiled from lessac's latent is validated against the loaded
 voice's channel count at init and fails with a message naming the producer if it does not match.
 
@@ -549,6 +593,19 @@ drift (best lag 0 samples in every window), and the max sample-to-sample step is
 (13384 vs 13412), so no seam clicks. The dips are near-silence (LSB noise) and one loud plosive
 onset, i.e. the fp16 noise floor, not the stitching.
 
+The duration encoder ladder is a small, width-limited win, and the width limit is the quality
+story. On a 557-character mixed RU/EN text (8 chunks, 109-459 tokens each), the narrow bins are
+faster than the ggml encoder (63-71 ms vs 84-89 ms at 109-115 tokens, 1.25-1.35x) and the wide
+bin is slower (688-773 ms vs 440-758 ms at 285-373 tokens: 2.9 GB of Toeplitz weights saturate
+DRAM bandwidth), so a `{128,192}` ladder with a ggml fallback above 192 is the shape worth
+shipping: it moves 3 of the 8 chunks and costs nothing on the rest. Per-bin engines score 40-48 dB
+against fp32 on the board (stats and conditioning, n in {17, N-1, N}). Quality: the fp16 engine
+shifts each chunk's duration sum by the sum of its per-token sampling errors, so the drift grows
+with width — ±7% latent frames on the 128-bin chunks (the scale of the flow's own seeded sampling
+noise) against a one-sided −21…−23% on the 384-bin chunks, a 25% tempo shift, audible. GigaAM/
+Whisper transcribe the narrow-bin render and the ggml render to the same text. The wide bins are
+fp16-exact per frame and systematically bad per utterance: keep the ladder narrow.
+
 ### Kokoro, measured on the same board
 
 `kokoro-82m-q8_0.gguf`, 185 Russian characters, seed 1234, `--threads 4` (14.85 s of audio,

docs/models/piper_tts.md

@@ -39,13 +39,17 @@ audiocpp_cli --task tts --family piper_tts \
 |---|---|
 | `piper_tts.espeak_library_path` | Explicit path to the eSpeak-ng shared library. |
 | `piper_tts.espeak_data_path` | Explicit path to `espeak-ng-data`. |
-| `piper_tts.exec_plan` | JSON object of component execution plans. Default `{}` keeps existing execution. Select the RKNN decoder with `{"decoder":{"impl":"rknn","artifacts":{"main":"/models/decoder.rknn"}}}`, and pin the duration stages to the host with `{"duration_encoder":{"backend":"cpu"},"duration_flows":{"backend":"cpu"}}`. No params. RKNN requires `ENGINE_ENABLE_RKNN=ON`; see [Rockchip NPU](../build/rockchip-npu.md#component-execution-plans). |
+| `piper_tts.exec_plan` | JSON object of component execution plans. Default `{}` keeps existing execution. Select the RKNN decoder with `{"decoder":{"impl":"rknn","artifacts":{"main":"/models/decoder.rknn"}}}`, move the duration encoder onto the NPU over a comma-separated ladder of per-width engines with `{"duration_encoder":{"impl":"rknn","artifacts":{"main":"/models/enc_n128.rknn,/models/enc_n192.rknn"}}}` (narrow bins only: wide widths drift the duration sum; chunks above the widest engine stay on ggml), and pin the duration stages to the host with `{"duration_encoder":{"backend":"cpu"},"duration_flows":{"backend":"cpu"}}`. No params. RKNN requires `ENGINE_ENABLE_RKNN=ON`; see [Rockchip NPU](../build/rockchip-npu.md#component-execution-plans). |
 
 On RK3588 the decoder is 75% of the runtime, so the NPU path is worth the extra file: measured
 RTF 0.198 → 0.0696 and peak RSS 299 → 175 MB on `en_US-lessac-medium`.
 
 The two duration stages are components of the same option, so each can run on the host while the
-decoder stays on the device. `duration_encoder` already runs on the host on CUDA by default: host
+decoder stays on the device. `duration_encoder` can also run on the NPU
+(`impl: rknn`, a comma-separated ladder of per-width engines from the decoder's producer script
+with `--encoder`): the narrow bins measure 1.25-1.35x over the host encoder, wide bins are slower
+and shift the duration sum, so keep the ladder narrow and let the fallback cover the rest.
+`duration_encoder` already runs on the host on CUDA by default: host
 duration alignment keeps small TF32 differences from being amplified by `expand_prior` while the
 expensive decoder stays on the device. `duration_flows` runs on the device by default, and it is
 the stage worth watching on a device backend, because the flows are the sampler: a rounding

include/engine/community_models/piper_tts/rknn_duration_encoder.h

@@ -0,0 +1,49 @@
+#pragma once
+
+#include <cstdint>
+#include <memory>
+#include <string>
+#include <vector>
+
+namespace engine::models::piper_tts {
+
+// Rockchip NPU (RKNN) backend for the VITS duration encoder (text tokens to prior
+// stats and flow conditioning).
+//
+// The graph is the voice's own encoder + duration discriminator sliced at their
+// projections (see tools/prepare_piper_tts_rknn_vocoder.py --encoder): tokens and
+// their length in, the prior stats (2*inter_channels rows) and the flow conditioning
+// (inter_channels rows) out. The export carries the frame mask inside the encoder,
+// so a right-padded fixed-width run reproduces the exact-length run on the real
+// frames (measured 117-130 dB fp32); each compiled engine covers one token width,
+// so the sidecar keeps a ladder of bins and an utterance longer than the widest bin
+// runs the ggml encoder for that step.
+class PiperTtsRknnDurationEncoder {
+public:
+    // `model_paths` is a comma-separated list of compiled engines (the exec_plan
+    // artifact list); each is validated against the voice at load.
+    PiperTtsRknnDurationEncoder(
+        const std::string & model_paths,
+        int64_t inter_channels);
+    ~PiperTtsRknnDurationEncoder();
+
+    PiperTtsRknnDurationEncoder(const PiperTtsRknnDurationEncoder &) = delete;
+    PiperTtsRknnDurationEncoder & operator=(const PiperTtsRknnDurationEncoder &) = delete;
+
+    // Widest token count any loaded engine covers.
+    int64_t max_tokens() const;
+
+    // `token_ids` (length n <= max_tokens()) in; stats [2*inter_channels][n] and
+    // conditioning [inter_channels][n], channel-major, in the layout the flow
+    // graphs and prior expansion consume.
+    void encode(
+        const std::vector<int32_t> & token_ids,
+        std::vector<float> & stats,
+        std::vector<float> & conditioning);
+
+private:
+    struct Impl;
+    std::unique_ptr<Impl> impl_;
+};
+
+}  // namespace engine::models::piper_tts

include/engine/community_models/piper_tts/vits_runtime.h

@@ -21,14 +21,18 @@ class PiperVitsRuntime {
 public:
     // `rknn_decoder_path` (optional) routes the decoder stage to a pre-compiled RKNN
     // engine on a Rockchip NPU; empty keeps the ggml decoder.
+    // `rknn_encoder_path` (optional) routes the duration encoder to a comma-separated
+    // ladder of pre-compiled RKNN engines; utterances longer than the widest bin fall
+    // back to the ggml encoder for that step.
     // `duration_encoder_backend` / `duration_flows_backend` (optional) pin the two duration
     // stages to a backend of their own, independently of each other and of the decoder.
     PiperVitsRuntime(
         std::shared_ptr<const PiperTtsAssets> assets,
         core::BackendConfig backend_config,
         std::string rknn_decoder_path = {},
         std::optional<core::BackendType> duration_encoder_backend = {},
-        std::optional<core::BackendType> duration_flows_backend = {});
+        std::optional<core::BackendType> duration_flows_backend = {},
+        std::string rknn_encoder_path = {});
     ~PiperVitsRuntime();
 
     runtime::AudioBuffer synthesize(