Update DeepSeek-V4-Pro (DSV4) FP4 GB300 Dynamo-SGLang AgentX MTP configs / 更新 DeepSeek-V4-Pro（DSV4）FP4 GB300 Dynamo-SGLang AgentX MTP 配置 - #2644
Conversation
Add TP8 concurrency 1/4 and TP4 concurrency 8 aggregate profiles with phase-specific decode CUDA graph limits and throughput-only synthetic acceptance. 中文：优化 SGLang AgentX 低并发吞吐，新增 TP8 并发 1/4 和 TP4 并发 8 的聚合配置，并使用解码阶段 CUDA Graph 上限及仅吞吐测试启用的模拟验收率。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Use SGLang nightly-dev-cu13-20260811-d59c1ddf and the Dynamo 1.4.0 release across the GB300 aggregate and disaggregated recipe family. 中文：将 GB300 聚合与分离式配置族统一更新到 SGLang nightly-dev-cu13-20260811-d59c1ddf 和 Dynamo 1.4.0 正式版。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
|
||
| - config-keys: | ||
| - dsv4-fp4-gb300-dynamo-sglang-agentic-agg | ||
| scenario-type: | ||
| - agentic-coding | ||
| description: | ||
| - "Tune the GB300 SGLang AgentX aggregate low-concurrency frontier with TP8 concurrency 1/4 and TP4 concurrency 8 profiles." | ||
| - "Update the aggregate and disaggregated configurations to SGLang nightly-dev-cu13-20260811-d59c1ddf and the Dynamo 1.4.0 release." | ||
| - "Use phase-specific decode CUDA graph limits and throughput-only synthetic MTP acceptance; eval-only runs retain real target verification." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2644 |
There was a problem hiding this comment.
🔴 The new changelog entry (perf-changelog.yaml:6053-6062) only lists config-keys: [dsv4-fp4-gb300-dynamo-sglang-agentic-agg], but this same PR also bumps the image (to nightly-dev-cu13-20260811-d59c1ddf) and router (to 1.4.0) for dsv4-fp4-gb300-dynamo-sglang-agentic-disagg in configs/nvidia-master.yaml, plus the Dynamo wheel to 1.4.0 in all 5 disagg recipe YAMLs. Since run-sweep.yml/process_changelog.py only schedules a master-config change for keys listed in the changelog entry, these disagg version bumps will merge without being validated by this PR's sweep. Please add dsv4-fp4-gb300-dynamo-sglang-agentic-disagg to config-keys.
Extended reasoning...
What the bug is: The tail changelog entry this PR appends (perf-changelog.yaml:6053-6062) declares only one config key:
- config-keys:
- dsv4-fp4-gb300-dynamo-sglang-agentic-agg
scenario-type:
- agentic-coding
description:
- "Tune the GB300 SGLang AgentX aggregate low-concurrency frontier with TP8 concurrency 1/4 and TP4 concurrency 8 profiles."
- "Update the aggregate and disaggregated configurations to SGLang nightly-dev-cu13-20260811-d59c1ddf and the Dynamo 1.4.0 release."
- ...Note the description's own second line explicitly claims both the aggregate and disaggregated configs are being updated — but the config-keys list only selects the agg key.
The code path that triggers it: Per docs/architecture.md (Stage 1, 'configuration and trigger selection'): 'A master entry is inert until selected. On the main sweep path, additions to perf-changelog.yaml select exact config keys or key patterns... Editing a master entry without a matching changelog addition does not schedule that change through run-sweep.yml, whose path trigger watches perf-changelog.yaml.' utils/process_changelog.py's get_config_keys_from_master() only expands the config-keys listed in the added changelog lines, and the matrix generator is subsequently invoked with only those expanded keys. There is no separate mechanism that detects 'this master entry byte-diffed, so schedule it regardless of changelog selection.'
Why nothing else prevents it: validate_append_only_scope (the guard that would reject a master edit outside the entry's declared append-only scope) only fires when an entry is marked append-only: true. Neither this new entry nor the historical entries touching these keys carry that flag, so nothing rejects — or even flags — an unselected master-file edit. It silently passes review and CI.
Concrete proof this actually happens in the diff:\n1. configs/nvidia-master.yaml diff shows dsv4-fp4-gb300-dynamo-sglang-agentic-disagg changes image: lmsysorg/sglang:nightly-dev-cu13-20260719-99f5a6f4 → nightly-dev-cu13-20260811-d59c1ddf and router: { ..., version: "1.3.0.dev20260718" } → { ..., version: "1.4.0" }.\n2. All five disagg-gb300-*-mtp-kvoffload.yaml recipe files bump dynamo.wheel from "1.3.0.dev20260718" to "1.4.0".\n3. The new changelog entry's config-keys is only [dsv4-fp4-gb300-dynamo-sglang-agentic-agg].\n4. When run-sweep.yml fires off this changelog diff, process_changelog.py expands only the agg key into sweep points; the disagg key is never passed to generate_sweeps, so no disagg job runs and the new image + Dynamo 1.4.0 combination for the disagg family reaches main completely unvalidated by this PR.
Impact: The disaggregated recipe family (the most complex, highest-node-count topology here — up to 12P4D across 16 nodes) is left running an SGLang nightly image and Dynamo release it has never actually been benchmarked or eval'd against by this PR's own gate — exactly the class of regression the changelog-trigger split exists to catch.
Fix: Add dsv4-fp4-gb300-dynamo-sglang-agentic-disagg to the new entry's config-keys list so run-sweep.yml schedules and validates both the agg and disagg sweeps this PR actually touches.
| - config-keys: | ||
| - dsv4-fp4-gb300-dynamo-sglang-agentic-agg | ||
| - dsv4-fp4-gb300-dynamo-sglang-agentic-disagg | ||
| - dsv4-fp4-gb300-dynamo-sglang-agentic-disagg | ||
| description: | ||
| - "Add GB300 DeepSeek-V4 dynamo-sglang agentic recipes (agg-tp4 + dep8 disagg pareto)" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2157 |
There was a problem hiding this comment.
🔴 This PR's changelog diff accidentally duplicates the dsv4-fp4-gb300-dynamo-sglang-agentic-disagg line inside the pre-existing PR #2157 entry (perf-changelog.yaml ~4885-4891), turning its config-keys into [agg, disagg, disagg]. Historical entries must stay byte-for-byte immutable except for pr-link fixes, so this will break the changelog validation gate — please drop the duplicate line and leave the #2157 entry untouched.
Extended reasoning...
The diff for perf-changelog.yaml adds a stray + - dsv4-fp4-gb300-dynamo-sglang-agentic-disagg line inside the pre-existing PR #2157 changelog entry, rather than only appending a brand-new entry at the end of the file. Before this PR, that entry's config-keys list was [dsv4-fp4-gb300-dynamo-sglang-agentic-agg, dsv4-fp4-gb300-dynamo-sglang-agentic-disagg]; after this PR it becomes [agg, disagg, disagg] — an accidental duplicate, almost certainly a copy/paste slip while adding the new tail entry for this PR (#2644). The entry's own description text ("agg-tp4 + dep8 disagg pareto") only ever described two config keys, confirming the third is unintentional.
This matters because perf-changelog.yaml historical entries are contractually immutable in this repo — only pr-link corrections are allowed. utils/validate_perf_changelog.py's compare_entries() diffs each historical entry against its base-branch counterpart via without_pr_link(); since config-keys differs here (not just pr-link), it raises \"entry N changed; existing entries are immutable except for pr-link-only corrections\". Separately, validate_raw_change() requires head_raw.startswith(base_raw) whenever new entries are appended (which they are, at EOF for PR #2644) — because the byte stream is now mutated mid-file rather than purely appended-to, that prefix check fails too, raising \"appended entries changed historical perf-changelog.yaml bytes; restore the base file byte-for-byte and append at the end\".
It's also worth noting this doesn't get filtered out by YAML parsing quirks: config-keys is a YAML sequence, not a mapping, so the UniqueKeyLoader used elsewhere in the validation pipeline (which only rejects duplicate mapping keys) does not catch or dedupe it — the duplicate list item survives untouched into the immutability comparison. Additionally, one verifier traced the PR-time check-changelog CI path specifically (validate_perf_changelog.py main() → validate_generated_config → process_changelog.py), which collects only the added + lines and feeds them to yaml.safe_load(). Because the orphaned mid-entry - dsv4-...-disagg fragment (indented 4 spaces) precedes the new top-level - config-keys: entry in that concatenated added-lines blob, the YAML parse itself fails with a ParserError (expected <document start>, but found <block sequence start>), independently breaking the same CI gate before it even reaches the immutability check.
Proof, step by step:
- Base (main)
perf-changelog.yamlentry for PR Add GB300 DeepSeek-V4 Dynamo-SGLang AgentX aggregated and disaggregated recipes / 新增 GB300 DeepSeek-V4 Dynamo-SGLang AgentX 聚合式与分离式配方 #2157 hasconfig-keys: [dsv4-fp4-gb300-dynamo-sglang-agentic-agg, dsv4-fp4-gb300-dynamo-sglang-agentic-disagg]. - This PR's diff inserts
+ - dsv4-fp4-gb300-dynamo-sglang-agentic-disaggas a third list item under that same entry (confirmed at perf-changelog.yaml lines ~4885-4891 in the current tree). compare_entries()parses both base and head versions of that entry;without_pr_link(base) != without_pr_link(head)becauseconfig-keysnow has 3 items vs. 2 — it raises the immutability error.- Independently,
validate_raw_change()checks that all bytes preceding the newly appended tail entries are unchanged (head_raw.startswith(base_raw)); since a line was inserted mid-file, this is false, raising a second, distinct error. - Either failure alone is sufficient to fail the
check-changelogCI job and block the PR from merging.
The fix is a one-line deletion: remove the erroneously duplicated - dsv4-fp4-gb300-dynamo-sglang-agentic-disagg line so the #2157 entry's config-keys reverts to its original two-item list, leaving that historical entry byte-for-byte identical to main (the new tail entry for PR #2644, appended at EOF, is unaffected and should remain).
中文：合并 main 并解决性能变更记录冲突。
完善 GB300 SGLang AgentX 配置：同步配方镜像与身份信息，将模拟验收率限定到吞吐任务，并统一 Slurm 账户与作业时限。
合并最新 main，并保留其完整变更日志后将本 PR 条目追加到文件末尾。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32154816722 |
RohitNagraj
changed the title
更新 GB300 电源通道契约测试，使其验证显式的 Slurm 账户和分区参数。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32170134901 |
4 participants
Description
nightly-dev-cu13-20260811-d59c1ddfand the released Dynamo1.4.0wheels/router.benchmarkaccount to image-validation steps.Validation
uv run python -m pytest utils/matrix_logic/ -q(231 passed)srtctl dry-runpassed for all seven affected aggregate/disaggregated recipes.runners/launch_gb300-nv.shrenders accountbenchmark, partitionbatch_1, and an eight-hour DSV4 SGLang AgentX allocation.中文说明
nightly-dev-cu13-20260811-d59c1ddf和 Dynamo1.4.0正式版 wheel/router。benchmark账户。本地验证
uv run python -m pytest utils/matrix_logic/ -q（231 项通过）srtctl dry-run。runners/launch_gb300-nv.sh生成的配置使用benchmark账户、batch_1分区，以及 DSV4 SGLang AgentX 的八小时作业时限。Related Issue / 相关问题
N/A / 不适用
Type of Change / 变更类型
Checklist / 检查清单
perf-changelog.yamland have not edited historical entries. / 对于每项可能影响基准性能的变更以及每项 recipe 新增或修改，我已在perf-changelog.yaml的物理末尾追加新条目，且未修改历史条目。/reuse-sweep-runon this PR after a final full sweep is green with evals passing. / 在通过复用方式合并前，授权维护者将在最终完整扫描和评估全部通过后评论/reuse-sweep-run。