[WIP] Test DeepSeek-V4 GB300 Dynamo AgentX recipes / 测试 DeepSeek-V4 GB300 Dynamo AgentX 配置 - #2302
[WIP] Test DeepSeek-V4 GB300 Dynamo AgentX recipes / 测试 DeepSeek-V4 GB300 Dynamo AgentX 配置#2302ivanium wants to merge 3 commits into
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Claude finished @ivanium's task in 6m 24s —— View job PR Review
SummaryFound 1 blocking issue and 2 warnings (details in inline comments):
The rest looks good: new recipe YAMLs are well-documented, the |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
ivanium
changed the title
There was a problem hiding this comment.
This PR adds new multi-node GB200 AgentX recipes and modifies shared launcher/routing logic, including a change to configs/nvidia-master.yaml (a CODEOWNER-protected file), so it's worth a human look before merging.
What was reviewed: the two new srt-slurm recipe YAMLs (aggregate TP8 and disaggregated 3P1D DEP8/DEP16), the launch_gb200-nv.sh model-path routing and srt-slurm source switch for agentic runs, the benchmark_lib.sh aiperf conv-aware-routing opt-out, and the new nvidia-master.yaml/perf-changelog.yaml entries.
Extended reasoning...
Overview
The PR adds two new DeepSeek-V4 GB200 Dynamo AgentX recipes (aggregate TP8 and disaggregated 3P1D DEP8/DEP16), registers them in configs/nvidia-master.yaml and perf-changelog.yaml, and modifies two shared scripts: benchmarks/benchmark_lib.sh (adds an opt-out env var for aiperf's dynamo conv-aware-routing flag) and runners/launch_gb200-nv.sh (switches the agentic srt-slurm clone from a pinned fork to NVIDIA/srt-slurm:main, and branches MODEL_PATH resolution for DSV4 FP4 based on IS_AGENTIC).
Security risks
No security-sensitive surface — this is internal benchmark/CI infrastructure (Slurm job launching, recipe YAML, shell env plumbing) with no auth, crypto, or externally-reachable inputs. The git clone URLs and container images are pinned to specific orgs/tags rather than user-controlled input.
Level of scrutiny
Higher than typical for this repo: the diff touches shared launcher logic (launch_gb200-nv.sh) and the aiperf routing default in benchmark_lib.sh, both of which affect existing non-agentic GB200 runs in addition to the new recipes. It also modifies configs/nvidia-master.yaml, which is CODEOWNER-protected (@ankur-singh @kedarpotdar-nv @InferenceX/core). Combined with the size and multi-file nature of the change (new multi-node recipes with many tuned env vars, a repo-source switch for the agentic srt-slurm path, and conditional model-path logic), this warrants a human pass rather than auto-approval.
Other factors
The bug-hunting system found no bugs, and several candidate issues (leftover NCCL symm-mem env var, an odd tokenizer string, --no-preflight usage, combining kv-cache-memory with gpu-memory-utilization) were raised and ruled out as not real bugs. The PR description/comments show no CODEOWNER sign-off yet per the standard recipe-PR checklist, which is itself a signal that human review is expected before merge.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29960434846 |
baca859 to
99a28ab
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29976615750 |
ivanium
changed the title
Add four GB300 MTP3 AgentX topologies using the bedf79e image: aggregate TP8, 1P/1D DEP4/DEP8, 1P/1D DEP8/DEP8, and 2P/1D DEP8/DEP16. Follow PR #2157's aggregate/disaggregated organization, PR #2269's CI rack networking, and PR #2302's upstream srt-slurm per-node launch path with automatic hybrid load balancing. Configure each disaggregated decode-side MooncakeStore connector as a synchronous KV consumer with lookup disabled while preserving the prefill-side configuration. 中文：新增四个使用 bedf79e 镜像的 GB300 MTP3 AgentX 拓扑：聚合式 TP8、1P/1D DEP4/DEP8、1P/1D DEP8/DEP8 和 2P/1D DEP8/DEP16。配置组织遵循 PR #2157 的聚合式/分离式结构，网络设置对齐 PR #2269 的 CI 机架环境，并依据 PR #2302 使用上游 srt-slurm 的单节点单进程启动模式及其自动启用的混合负载均衡。仅将各分离式配置解码侧的 MooncakeStore 连接器设为同步 KV 消费端并关闭查找功能，同时保留预填充侧原有配置。
99a28ab to
985c29e
Compare
Add four GB300 MTP3 AgentX topologies using the bedf79e image: aggregate TP8, 1P/1D DEP4/DEP8, 1P/1D DEP8/DEP8, and 2P/1D DEP8/DEP16. Follow PR #2157's aggregate/disaggregated organization, PR #2269's CI rack networking, and PR #2302's upstream srt-slurm per-node launch path with automatic hybrid load balancing. Align all disaggregated recipes on Dynamo 1.3.0.dev20260720, preserve prefill-only connector prefetch and breakable CUDA graph settings, and configure decode-side MooncakeStore as a synchronous KV consumer with lookup disabled. 中文：新增四个使用 bedf79e 镜像的 GB300 MTP3 AgentX 拓扑：聚合式 TP8、1P/1D DEP4/DEP8、1P/1D DEP8/DEP8 和 2P/1D DEP8/DEP16。配置组织遵循 PR #2157 的聚合式/分离式结构，网络设置对齐 PR #2269 的 CI 机架环境，并依据 PR #2302 使用上游 srt-slurm 的单节点单进程启动模式及其自动启用的混合负载均衡。所有分离式配置统一使用 Dynamo 1.3.0.dev20260720，连接器预取和可中断 CUDA Graph 设置仅保留在预填充侧；解码侧 MooncakeStore 设为同步 KV 消费端并关闭查找功能。
985c29e to
dfc774f
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30154534054 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30170531787 |
Add four GB300 MTP3 AgentX topologies using the bedf79e image: aggregate TP8, 1P/1D DEP4/DEP8, 1P/1D DEP8/DEP8, and 2P/1D DEP8/DEP16. Follow PR #2157's aggregate/disaggregated organization, PR #2269's CI rack networking, and PR #2302's upstream srt-slurm per-node launch path with automatic hybrid load balancing. Align all disaggregated recipes on Dynamo 1.3.0.dev20260720, preserve prefill-only connector prefetch and breakable CUDA graph settings, and configure decode-side MooncakeStore as a synchronous KV consumer with lookup disabled. 中文：新增四个使用 bedf79e 镜像的 GB300 MTP3 AgentX 拓扑：聚合式 TP8、1P/1D DEP4/DEP8、1P/1D DEP8/DEP8 和 2P/1D DEP8/DEP16。配置组织遵循 PR #2157 的聚合式/分离式结构，网络设置对齐 PR #2269 的 CI 机架环境，并依据 PR #2302 使用上游 srt-slurm 的单节点单进程启动模式及其自动启用的混合负载均衡。所有分离式配置统一使用 Dynamo 1.3.0.dev20260720，连接器预取和可中断 CUDA Graph 设置仅保留在预填充侧；解码侧 MooncakeStore 设为同步 KV 消费端并关闭查找功能。
8a83d53 to
64f87af
Compare
Add four GB300 MTP3 AgentX topologies using the bedf79e image: aggregate TP8, 1P/1D DEP4/DEP8, 1P/1D DEP8/DEP8, and 2P/1D DEP8/DEP16. Follow PR #2157's aggregate/disaggregated organization, PR #2269's CI rack networking, and PR #2302's upstream srt-slurm per-node launch path with automatic hybrid load balancing. Align all disaggregated recipes on Dynamo 1.3.0.dev20260720, preserve prefill-only connector prefetch and breakable CUDA graph settings, and configure decode-side MooncakeStore as a synchronous KV consumer with lookup disabled. 中文：新增四个使用 bedf79e 镜像的 GB300 MTP3 AgentX 拓扑：聚合式 TP8、1P/1D DEP4/DEP8、1P/1D DEP8/DEP8 和 2P/1D DEP8/DEP16。配置组织遵循 PR #2157 的聚合式/分离式结构，网络设置对齐 PR #2269 的 CI 机架环境，并依据 PR #2302 使用上游 srt-slurm 的单节点单进程启动模式及其自动启用的混合负载均衡。所有分离式配置统一使用 Dynamo 1.3.0.dev20260720，连接器预取和可中断 CUDA Graph 设置仅保留在预填充侧；解码侧 MooncakeStore 设为同步 KV 消费端并关闭查找功能。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30217145037 |
Add three GB300 MTP3 AgentX topologies using the bedf79e image: aggregate TP8, 1P/1D DEP4/DEP8, and 1P/1D DEP8/DEP8. Follow PR #2157's aggregate/disaggregated organization, PR #2269's CI rack networking, and PR #2302's upstream srt-slurm per-node launch path with automatic hybrid load balancing. Align both disaggregated recipes on Dynamo 1.3.0.dev20260720, preserve prefill-only connector prefetch and breakable CUDA graph settings, and configure decode-side MooncakeStore as a synchronous KV consumer with lookup disabled. 中文：新增三个使用 bedf79e 镜像的 GB300 MTP3 AgentX 拓扑：聚合式 TP8、1P/1D DEP4/DEP8 和 1P/1D DEP8/DEP8。配置组织遵循 PR #2157 的聚合式/分离式结构，网络设置对齐 PR #2269 的 CI 机架环境，并依据 PR #2302 使用上游 srt-slurm 的单节点单进程启动模式及其自动启用的混合负载均衡。两个分离式配置统一使用 Dynamo 1.3.0.dev20260720，连接器预取和可中断 CUDA Graph 设置仅保留在预填充侧；解码侧 MooncakeStore 设为同步 KV 消费端并关闭查找功能。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31276735958 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31302825977 |
|
/stage-results 31302825977 |
|
@ivanium staged run 31302825977: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-09~r31302825977 This run remains available across future @ivanium 已将运行 31302825977 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-09~r31302825977 后续的 |
6a190ec to
5794386
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31339574966 |
Add a validated runner-node scheduling override that preserves the cluster hardware identity. Pin PR #2302 aggregate jobs to gb300-nv_0 and P/D jobs to gb300-nv_1 while gb300-nv_2 is repaired.\n\n中文：新增经过校验的 runner-node 调度覆盖字段，同时保留集群硬件标识。在 gb300-nv_2 修复期间，将 PR #2302 的聚合任务固定到 gb300-nv_0，并将预填充/解码分离任务固定到 gb300-nv_1。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31343667510 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31348412800 |
2721733 to
44721d9
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31440326390 |
44721d9 to
c9f35d6
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31470492563 |
5b14616 to
e74b093
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31527599663 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31581245372 |
e466115 to
21f1caf
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31980065264 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32006497502 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32006497502 |
新增与 TensorRT-LLM c736 点相同的 2P1D DEP8/DEP32 拓扑。vLLM 使用合成接受长度 2.97，以对应 TensorRT-LLM 的 1.97 记法。
新增 GB300 vLLM AgentX 1P1D DEP8/DEP32 c388 点，并保留现有 c736 配置与结果历史。
新增 vLLM GB300 c4、c12 和 c24 低延迟配置，并从当前生成的扫描矩阵中移除 c736 和 c388。
3332492 to
ecc6dc6
Compare
2 participants
Summary
mainand re-append this PR's changelog entry at the tailAIPerf dependency fix: SemiAnalysisAI/aiperf#38
Validation
bash -n runners/launch_gb300-nv.sh benchmarks/benchmark_lib.sh中文说明
flock；同时让 AIPerf 的自动生成预热阶段真正应用已配置的 60 秒并发爬升，防止快照预热请求压垮 Dynamo 预填充入口main，并按追加式变更日志规则将本 PR 条目重新放到文件末尾AIPerf 依赖修复：SemiAnalysisAI/aiperf#38