[wOvAN]

What

Streaming on the RKNN vocoder and streaming on ggml compute two different functions, and the shipped numbers show it: the stateful engine voices each token once through a carried cache (57.7 s wall for 307 characters), while the ggml streaming graph re-voices the 25-frame left context inside every window and drops its audio (106 s). The renders agree at 16.4 dB SNR — that gap is the conditioning semantics, not fp16 noise, and no engine retraining closes it.

This adds the second function to the NPU path as an opt-in plan artifact:

• qwen3_tts.exec_plan → speech_decoder.rknn gains an optional window artifact. When present, every streaming event runs a full decode of its whole [left context + new] window: the same stateful graph with cache_in zeroed, cache_valid 0 and RoPE restarting at 0, so the step is the full decode over the window; the context's audio is dropped and flush voices nothing (the engine is stateless in this mode). Offline decoding keeps using main, and stateful streaming stays the default when no window is named.
• A window that names the same file as main aliases it — one resident context voices groups offline and windows streaming, and engine_resources() already dedupes by canonical path, so the residency guard prices the three-role plan exactly as the pair (verified). A dedicated smaller export (--codes-length 25 + stream_frames_per_event) is the per-event optimization; the aliased B=64 engine covers the default 25 + 12 window.
• loader.cpp: model_resources counts window alongside first.
• The converter gets a small fix: with --output *.rknn the export stage now writes the ONNX beside the engine path instead of over it, so the one-command export→compile chain works with an engine-named --output.

Validation

RK3588 board (native aarch64 build, ENGINE_ENABLE_RKNN=ON, rebuilt after the change, 0 errors), 307 Russian characters, voice-clone from assets/resources/b.wav, seed 42, greedy, conv_weight_type=f16, --threads 4:

• Streaming, ggml decoder (reference function): 106 s wall.
• Streaming, RKNN 16+64, stateful (default, unchanged): 44 s wall, decoder 12.9 s, first audio 0.77 s.
• Streaming, RKNN 16+64 with "window":"/root/tts-rknn/vocoder/vocoder_stateful_64b_fp16.rknn": 93.2 s wall, decoder 61.1 s (20 × 3.07 s events), qwen3_tts.speech_decoder.rknn.window_group 64 in the log.
• Quality against the ggml streaming render, same length (382080 samples): stateful 16.4 dB, window 47.4 dB SNR with max sample delta 731/32767 — the windowed run reproduces the CPU streaming function to engine-precision noise, at the re-voicing price (+48 s decoder, the 25 context frames per event).
• Residency guard A/B with the aliased plan at --min-free-memory-mb 27300: refused with estimated 3.49 GiB + 27300 MiB headroom exceeds available host memory (29.88 GiB) — identical to the main+first pair estimate, so the aliased file is priced once.
• ctest -R "rknn|qwen3|server_config": rknn_engine_test, server_config_test, rknn_sidecar_test all pass.
• Offline decode and default (window-less) streaming are byte-comparable to the pre-change behavior on the same runs: the stateful streaming render measured in this matrix reproduces the documented 16.4 dB row exactly.

[0xShug0]

@wOvAN One issue: Engine alias detection compares path strings, whereas memory accounting canonicalizes paths. Setting main and window to the same file using different spellings can load two contexts but count only one.

[wOvAN]

@wOvAN Fixed — the alias test is now file-level, matching the accounting: std::filesystem::equivalent(window_path, model_path, ec) replaces the string comparison (src/models/qwen3_tts/rknn_speech_decoder.cpp, commit c2524de8; the header doc now says "naming the same file", not "equal"). A missing window path still takes the load path and fails with its own named cannot open model message, so the ec-swallowing form is deliberate.

Measured on the RK3588 board with the same file spelled two ways (window a symlink to the main engine):

• pre-fix binary, strace -e trace=openat: engine image opened twice (vocoder_stateful_64b_fp16.rknn + window_alias.rknn) — two contexts, one counted, exactly the skew you named;
• post-fix: opened once, the symlink never reaches the loader (one context, one counted);
• the aliased render is byte-identical to the pre-fix aliased render (261164 B, cmp clean), so the alias path behaves as before once the spelling is canonicalized away.