fix(qwen35): warn when a prompt outgrows --fa-window

[davide221]

The bug

A finite --fa-window caps the full-attention layers to a sliding window, so content earlier than the window is invisible to them. Nothing reports this. The model still answers, it just cannot see the head of a long prompt, so it reads as a model quality problem rather than a configuration one.

Found while benchmarking long context on a Radeon AI PRO R9700 with Qwen3.8-27B. A label was planted at the very top of the prompt and the model asked to repeat it:

prompt tokens	--fa-window 2048	full attention
1,556	recalled	recalled
6,208	missed	recalled
13,148	missed	recalled
26,728	missed	recalled

No warning was emitted in any of the failing cases.

Worth noting: on this hardware the flag was not even buying speed. At a 27K prompt it measured 44.4 tok/s with the window against 45.7 without, and full attention recalled the label at every length tested up to 81K.

The fix

Warn once, the first time a prompt actually outgrows the window, naming both numbers so the cause is unambiguous. No behaviour change otherwise.

Validation

On the same box with --fa-window 2048: the warning fires exactly once, on the 6,208-token request that does lose the label, and does not fire for the 1,556-token request that retrieves it.

[qwen35] WARNING: prompt is 6208 tokens but --fa-window is 2048: full-attention
layers see only the last 2048 tokens, so content before that cannot be
retrieved. Drop --fa-window for long-context work.

[cubic-dev-ai]

1 issue found across 1 file

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/qwen35/qwen35_backend.cpp">

<violation number="1" location="server/src/qwen35/qwen35_backend.cpp:1634">
P3: The warning only checks the prefill-time length (tokens + kv_offset) and emits once, so it never fires when the prompt fits inside the window at prefill but a long generation then pushes the context past it. The decode-time sliding window at layer_split_forward.cpp:335 covers only [kv_start - fa_window, kv_start], so as generation grows kv_start the prompt head is dropped mid-generation with no warning, exactly the silent-retrieval problem the PR targets. Consider also warning from the decode path the first time kv_start exceeds fa_window with a finite window.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P3: The warning only checks the prefill-time length (tokens + kv_offset) and emits once, so it never fires when the prompt fits inside the window at prefill but a long generation then pushes the context past it. The decode-time sliding window at layer_split_forward.cpp:335 covers only [kv_start - fa_window, kv_start], so as generation grows kv_start the prompt head is dropped mid-generation with no warning, exactly the silent-retrieval problem the PR targets. Consider also warning from the decode path the first time kv_start exceeds fa_window with a finite window.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/qwen35/qwen35_backend.cpp, line 1634:

<comment>The warning only checks the prefill-time length (tokens + kv_offset) and emits once, so it never fires when the prompt fits inside the window at prefill but a long generation then pushes the context past it. The decode-time sliding window at layer_split_forward.cpp:335 covers only [kv_start - fa_window, kv_start], so as generation grows kv_start the prompt head is dropped mid-generation with no warning, exactly the silent-retrieval problem the PR targets. Consider also warning from the decode path the first time kv_start exceeds fa_window with a finite window.</comment>

<file context>
@@ -1624,6 +1624,24 @@ int Qwen35Backend::do_prefill(const std::vector<int32_t> & tokens,
+    // configuration one. Say so once, the first time a prompt actually
+    // outgrows the window.
+    if (cfg_.fa_window > 0 &&
+        (int)tokens.size() + kv_offset > cfg_.fa_window) {
+        static std::atomic<bool> s_fa_window_warned{false};
+        if (!s_fa_window_warned.exchange(true)) {
</file context>