Skip to content

Port MI355X DeepSeek-R1 FP8 disaggregation to srt-slurm - #2631

Draft
cquil11 wants to merge 7 commits into
mainfrom
agent/port-dsr1-fp8-mi355x-srt-slurm
Draft

Port MI355X DeepSeek-R1 FP8 disaggregation to srt-slurm#2631
cquil11 wants to merge 7 commits into
mainfrom
agent/port-dsr1-fp8-mi355x-srt-slurm

Conversation

@cquil11

@cquil11 cquil11 commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • replace the legacy MI355X DeepSeek-R1 FP8 disaggregated launcher with one native srt-slurm override recipe
  • preserve all six STP/MTP topology families and every original 8k/1k concurrency point
  • move to the current MI355X SGLang v0.5.17 runtime, SGLang Router, and MoRI KV transfer
  • reuse the complete 688.6 GB legacy model cache through a zero-copy canonical Hugging Face cache link
  • remove the retired model-specific wrapper and keep benchmark_serving.py unchanged via the custom benchmark interface

Topologies

  • STP and MTP1: 2P DEP8/DPA + 1D DEP8/DPA at c1024/c2048
  • STP: 1P TP8 + 2D TP8 at c4-c256
  • MTP2: 1P TP8 + 2D TP8 at c2-c256
  • STP and MTP2: 1P TP4 + 1D TP8 at c1-c64

Validation

  • all six resolved recipe selectors pass srt-slurm dry-run at c609754b5622f96d5c12a93149e245308d4f1e9b
  • resolved topology, parallelism, MTP, MoRI dispatch, and benchmark environment assertions pass
  • 231 matrix generation and validation tests pass
  • exact six-job InferenceX matrix generation passes
  • launcher bash syntax and repository diff checks pass
  • cluster audit confirms 163/163 indexed model shards with zero missing files

@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

@cquil11
cquil11 force-pushed the agent/port-qwen35-fp8-mi355x-srt-slurm branch from dda2f3a to 32d3ee2 Compare August 26, 2026 23:54
@cquil11
cquil11 force-pushed the agent/port-dsr1-fp8-mi355x-srt-slurm branch from 273b39f to 6e8e30d Compare August 27, 2026 00:11
@cquil11
cquil11 changed the base branch from agent/port-qwen35-fp8-mi355x-srt-slurm to main August 27, 2026 00:11
@cquil11
cquil11 force-pushed the agent/port-dsr1-fp8-mi355x-srt-slurm branch from c3fc5ab to e7294ca Compare August 27, 2026 01:56
@cquil11
cquil11 force-pushed the agent/port-dsr1-fp8-mi355x-srt-slurm branch 2 times, most recently from b4bec48 to 1cbb32e Compare August 27, 2026 06:26
@cquil11
cquil11 force-pushed the agent/port-dsr1-fp8-mi355x-srt-slurm branch from 1cbb32e to 7b8a470 Compare August 27, 2026 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant