perf(ds4): overlap heterogeneous owner prefill

[cheese-cakee]

Important

Stacking: GitHub shows main as this PR's base branch, but the commit history is stacked directly on the open PR #640 head (51d2929). The #647-specific commits are cdcd4ac4, 42bd9ab2, 538b3041, and 2eab093e. PR #640 merges first and becomes the baseline; PR #647 is the incremental scheduling optimization on top of it.

Benchmark baseline: Every #647 feature-off and feature-on measurement below already includes #640. “Feature off” means #640 with only DFLASH_MOE_COLD_INPUT_FIRST=0; it does not mean the pre-#640 baseline.

Result

This removes a host-side serialization point from heterogeneous DeepSeek-V4 expert prefill while preserving the existing device-side copy dependency.

Performance stack

The merge and measurement story has two stages:

Stage	Matched comparison	2048-token result	401-token result
#640	main at #640's qualification (ac22a3ed) -> #640	292.42 -> 304.69 tok/s: +4.196%	54.16 -> 184.11 tok/s: 3.399x (+239.937%)
#640 + #647	#640 with #647 disabled -> #640 with #647 enabled	308.67 -> 321.86 tok/s: +4.273% incremental	179.62 -> 187.87 tok/s: +4.593% incremental

The headline for #647 is therefore +4.27% at 2048 tokens and +4.59% at 401 tokens, incrementally on top of #640, with exact output parity. The #640 absolute values differ between its own qualification (304.69 tok/s at 2048 tokens) and the later #647 qualification baseline (308.67 tok/s) because they were separate sessions; each percentage above is calculated only within its matched experiment.

Full qualification

Qualified workload	Feature off	Feature on	Improvement
2048-token prefill throughput	308.67 tok/s	321.86 tok/s	+4.27%
2048-token prefill wall time	6.635 s	6.363 s	-4.10%
401-token, chunk-512 throughput	179.62 tok/s	187.87 tok/s	+4.59%
401-token, chunk-512 wall time	2.233 s	2.135 s	-4.39%

The two independent 2048 brackets were +4.45% and +3.84%. The 401 regression brackets were +4.84% and +4.73%. Every timed response matched the exact oracle.

Review-fix validation

The final head was rebuilt and re-tested after the review fixes:

• focused readiness test: PASS
• test_deepseek4_unit: PASS on gfx1201 and gfx1151, including the 63=false, 64=true safety boundary

The exact live head 2eab093e was not run through another complete performance qualification. The performance headline above remains tied to the preserved complete matched qualification; no incomplete or cross-session post-review result is used as headline evidence.

What changed

• launch the cold expert owner first and release hot-owner submission once the common-input peer-copy dependency has been enqueued
• remove the redundant host-wide ggml_backend_synchronize() from that readiness point
• retain the HIP stream ordering already established by the peer-copy path: record an event on the source stream, then wait for it on the destination stream
• make readiness single-shot and exception-safe so a worker failure before signaling cannot deadlock the caller
• preserve the original worker/call shape when DFLASH_MOE_COLD_INPUT_FIRST is disabled
• fall back to the original scheduling when GGML peer-copy batching is enabled, because that mode defers publication of the destination-stream wait

The option remains default-off and is an immediate escape hatch: leave DFLASH_MOE_COLD_INPUT_FIRST unset or set it to 0.

Why it works

The D2D transfer itself was not made smaller or removed. Both traces contain the same 43 hot-to-cold copies with effectively unchanged median copy duration.

Causal trace metric	Disabled	Enabled
Expert-owner layers with overlapping MMQ	2 / 43	40 / 43
Simultaneous gfx1201/gfx1151 busy time	47.1 ms	535.7 ms
Simultaneous share of gfx1201 busy time	2.45%	28.90%
Median cold-minus-hot owner start gap	23.50 ms	0.57 ms
D2D copies	43	43
Median D2D duration	9.680 ms	9.720 ms

This is a scheduling/overlap improvement: the host can submit independent hot work earlier while the destination stream still waits on the copy event before consuming the input.

Qualification

The preserved complete matched performance qualification used two warmups and five measured requests per block in off/on/on/off order. Values above are pooled medians; no outliers were removed.

• 2048-token primary acceptance: PASS (+4.27%, brackets +4.45%, +3.84%)
• 401-token regression acceptance: PASS (+4.59%, brackets +4.84%, +4.73%)
• default-off equivalence: PASS (-0.24% pooled; brackets +2.06%, -0.50%)
• exact differential oracle: PASS on all 52 timed qualification requests
• profiled off/on requests: exact oracle PASS
• test_moe_input_ready: PASS, including bounded pre-ready exception propagation and concurrent single-shot signaling
• test_deepseek4_unit: PASS on gfx1201 and gfx1151.

Reproducibility

• stack base: 51d2929bd0b133f7b284ce544be56327312b8737
• final PR head: 2eab093e
• final build binary SHA-256: 3d88a1c184c8e6579cbd048eb843631cdc57dac68602908e1f51f10c98df6740
• measured post-review candidate binary SHA-256: ab64b72401058b04a18404ae3630254fcd8b9c74e54f0578b68e7f35abae22f1
• full-qualification candidate binary SHA-256: fca26a03d0b9e5b79bd4103c26b4add4391f0656fe16f86b7c4373be006cf79b
• model SHA-256: 8fa6c30d9badd8e72f83c62952fe24b2e8cd647e103aff086bece328cb9c9208
• exact response SHA-256: cd5cb9fb5ac3c4f4007e8b41d117da21622439cd05c1728f3e82f90e4f869dad
• qualified source patch SHA-256: 8494bc2a54ee32d5b99eab0b76a99c7060974e41c472b9027ab50fb24fd87743
• review fix patch SHA-256: c5d09cff0e30baed33f2087f91313dfb82f1daeabdf5666d57d5d4fa26f977f1
• cutoff regression patch SHA-256: d8230ed1f8d0e59688ac09e0226a6e6d192c9b4fd21c8630007da54ec9c5ff66

Preserved evidence roots

• /home/cheese/perf-ds4-owner-input-first-final-qual5-20260823T203000IST
• /home/cheese/perf-ds4-owner-input-first-final-trace-20260823T220000IST
• /home/cheese/perf-ds4-owner-input-first-final-static-20260823T214500IST

Each final evidence root includes checksums; the qualification and causal-trace manifests were sealed and verified.

[cubic-dev-ai]

All reported issues were addressed across 4 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 7 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 2 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>