fix(qwen35) span the whole KV pool for slot-mapped verify attention

[davide221]

The bug

With --kvflash, a speculative request whose prompt is short enough returns decode_failed. On a Radeon AI PRO R9700 running Qwen3.8-27B with --kvflash auto (16,384-token pool) and the block-16 DFlash2 drafter:

prompt tokens	before	after
1,556	decode_failed	59.5 tok/s
6,208	decode_failed	43.3 tok/s
13,148	40.5 tok/s	39.0 tok/s
26,728	38.6 tok/s	38.6 tok/s

The server log shows:

[spec-decode] invalid draft seed -1 after 2 emitted tokens; switching to AR
[server] chat DONE ... ok=false ... error=decode_failed

Root cause

Under kvflash the KV cache lives at pool slots, and verify_batch builds its attention mask in slot space over the entire pool. The flash-attention view was still sized from the logical context length (kv_start + n_tokens, rounded up to the 256 stride).

Those two disagree. Slot indices are not ordered by logical position, so the view can end below slots the mask still marks visible. Attention then reads rows that were never written, the softmax row degenerates, and ggml_argmax returns -1 for every verify row past the first.

Instrumenting the failing case makes it concrete: at kv_start=1556 the view is 1,792 rows of a 16,384-row pool while the mask is 16,416 columns wide. Row 0 lands inside the view and returns a valid token; row 1 does not.

That invalid token then propagates. do_spec_decode sees the bad seed and falls back to plain decode, but the fallback inherits the same state and fails too, turning a recoverable condition into a failed request. Larger prompts survived only by accident: their logical extent happened to cover the slots in use.

The fix

Span the whole pool. The mask is sized from that same pool and is what restricts which slots are readable, so it is the bound that matches the caller's contract.

The condition is scoped to the slot-mapped path. A set_rows KV write together with an explicit mask is a pair only kvflash verify produces: the non-kvflash step-invariant write requires !with_mask, and the paged path never reaches this branch.

Validation

All on one R9700, gfx1201, ROCm 7.2, Qwen3.8-27B IQ4_XS with the DFlash2 q8_0 drafter, greedy.

• Sweep above passes with zero invalid-seed events; a label planted at the top of the context is recalled at every length.
• No change to the default path. Without --kvflash, HumanEval-10 measures 144.61 tok/s with output sha a4467e9d, identical to before.

Note for reviewers: this fixes correctness, not speed. kvflash still measures slower than plain full attention on this hardware at long context (38.6 vs 44.4 tok/s at 27K), which is a separate question.

[cubic-dev-ai]

All reported issues were addressed across 56 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 7 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>