perf(qwen35): batch draft projections across lanes for concurrency

[Graffioh]

Stack

This PR contains the exact current #642 history through 308f630f, followed by two separate commits:

1. bb65ab93 — perf(qwen35): batch draft projections across lanes for concurrency
2. f1c85e05 — refactor(qwen35): rename GDN journal to replay log

The projection commit is patch-equivalent to the original 98896bf8; its object ID changed only because the stack moved onto the current #625 and #642 heads. The replay-log rename stays separate, so the performance change is not folded together with unrelated cleanup.

Summary

• Batch shared draft dense projections across active lanes.
• Keep convolution, cache updates, RoPE, masks, positions, attention, and draft KV state lane-local.
• Use one builder for C1 through C6, without a runtime switch or duplicate execution path.
• Include the backend in the cached graph replay key.
• Inherit qwen35: batch DFlash2 decode for Qwen3.8-27B concurrency #642's metadata-preserving DFlash2 sliding-window handling. The tested Qwen3.8 draft reports SWA layers: 5/5 (window=2048).

The previous path ran each lane's dense draft backbone in a separate graph. The packed path combines projection columns and removes repeated MMQ launches without sharing lane state.

Projection A/B

This matched A/B isolates the unchanged #651 projection patch against its #642 parent. Warm medians exclude the first speculative round; each run generated a fixed 64-token output.

GPU	C	Draft compute, ms	Complete round, ms
R9700	1	5.864 -> 5.864 (-0.0%)	62.161 -> 62.107 (-0.1%)
R9700	2	11.622 -> 8.498 (-26.9%)	74.153 -> 70.951 (-4.3%)
R9700	3	17.424 -> 11.392 (-34.6%)	96.811 -> 90.871 (-6.1%)
Strix Halo	1	12.129 -> 12.120 (-0.1%)	137.892 -> 138.107 (+0.2%)
Strix Halo	2	24.039 -> 15.341 (-36.2%)	166.877 -> 158.065 (-5.3%)
Strix Halo	3	36.378 -> 19.665 (-45.9%)	224.446 -> 209.150 (-6.8%)

C1 is neutral. C2 and C3 reduce complete-round latency by 4.3% to 6.8%. The R9700 C3 rocprof trace reduced MMQ launches from 336 to 196 (-41.7%) and MMQ device time from 11.938 ms to 6.103 ms (-48.9%).

Verification

• git range-diff reports both perf(qwen35): batch draft projections across lanes for concurrency #651 commits patch-equivalent across the final restack.
• Confirmed that perf(qwen35): batch draft projections across lanes for concurrency #651's parent is exactly 308f630f, the final qwen35: batch DFlash2 decode for Qwen3.8-27B concurrency #642 head.
• Built and tested the patch-equivalent perf(qwen35): batch draft projections across lanes for concurrency #651 source before the final test-only qwen35: DSpark speculative decoding (Qwen3.8-27B drafters) #625 parent move; the exact final feat(qwen35): support sampled concurrent speculation #654 descendant also builds the complete perf(qwen35): batch draft projections across lanes for concurrency #651 source at e0647848.
• Passed the disk-cache version-2 test inherited from qwen35: DSpark speculative decoding (Qwen3.8-27B drafters) #625 on the exact final descendant.
• Passed five host SWA policy cases, batched-GDN CPU/GPU checks, all replay-log modes, and the real three-lane SWA qualification before the final test-only parent update.
• Passed git diff --check.

The refreshed Qwen3.8 concurrency matrix is reported on #642. #654 is stacked on this PR and adds sampled concurrent speculation.

[cubic-dev-ai]

1 issue found across 64 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/test/test_batched_gdn.cpp">

<violation number="1" location="server/test/test_batched_gdn.cpp:73">
P2: In test_cpu_gdn_support_matrix the active op is built with n_tokens=2 yet active_slots has only 1 element. GGML construction asserts the active-slot ids tensor matches n_tokens*n_seqs (as run_gdn does with n_seqs for n_tokens==1, and as the sibling tree/tree_persist helpers assert), so on the --cpu path this GGML_ASSERT aborts the test instead of printing the support-matrix result. Size active_slots to n_tokens.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>