[NV] Add B300 DeepSeek-V4-Pro AgentX with SGLang DSpark6 / 使用 SGLang DSpark6 新增 B300 DeepSeek-V4-Pro AgentX - #2624
[NV] Add B300 DeepSeek-V4-Pro AgentX with SGLang DSpark6 / 使用 SGLang DSpark6 新增 B300 DeepSeek-V4-Pro AgentX#2624weireweire wants to merge 22 commits into
Conversation
Root cause: - The GB300 DeepSeek-V4-Pro AgentX matrix still used the older EAGLE/MTP serving path and an outdated SGLang nightly. - The previous draft targeted vLLM recipes instead of the existing SGLang topology ladder. Fix: - Replace the SGLang aggregate and disaggregated GB300 recipes with DSpark block size 6 on DeepSeek-V4-Pro-DSpark. - Use nightly-dev-cu13-20260816-4a6dc267, which already includes the required DSpark performance and MegaMoE support, without a runtime patch overlay. - Force thinking-on AL 3.82 with static ragged verification, retain MegaMoE on DEP workers, and set the 8P4D throughput endpoint to concurrency 1536. Validation: - Passed srtctl dry-run for all seven recipes. - Passed 224 matrix and schema tests. - Generated the eight targeted matrix jobs at concurrency 1, 4, 8, 32, 192, 400, 640, and 1536. 中文：将 GB300 DeepSeek-V4-Pro AgentX 的 SGLang 配置从 EAGLE/MTP 更新为 DSpark6，使用已包含所需上游修复的 20260816 nightly，不再注入补丁；固定 AL 3.82 与 static ragged verify，并将 8P4D 吞吐端点设为并发 1536。七份 recipe dry-run、224 项矩阵与 schema 测试以及八个目标矩阵任务生成均已通过。
Root cause: - The InferenceX port hard-coded SGLANG_SIMULATE_ACC_* in every recipe instead of carrying NVIDIA InferenceMAX PR #214 throughput-only master-config injection. - Eval-only launch therefore skipped the injector but still inherited synthetic acceptance from the checked-in recipe. - The DSpark AL 3.82 source curve and the focused injection regression tests were also omitted. Fix: - Keep SGLANG_RAGGED_VERIFY_MODE=static in all six recipes while moving synthetic AL 3.82 into each throughput search-space entry. - Commit the probabilistic DeepSeek-V4-Pro-DSpark golden curve from Actions run 31279568355 and reference it from the bilingual index and performance changelog. - Add tests covering all six recipes, throughput injection, eval preservation, launcher ordering, DSpark6 parameters, tokenizer paths, and the committed golden value. Validation: - Passed matrix, schema, changelog, runner, and synthetic-acceptance tests. - Generated seven throughput points and four eval-only points at the intended concurrencies. - Passed YAML parsing, changelog validation, formatting checks, and git diff checks. 中文：将模拟接受长度改为仅在吞吐测试启动时注入，精度评测继续使用真实 DSpark 验证；同时补充黄金 AL 曲线、模型路径映射和回归测试，并删除表现不佳的 c32 点。
Root cause: SemiAnalysis InferenceX lacked the validated B300 Dynamo-SGLang DSpark6 AgentX configurations, including session-affinity routing and sufficient host HiCache capacity for DEP4 and DEP8.\n\nFix: add DEP4/DEP8 aggregate recipes, the B300 master sweep, DSpark checkpoint selection, srt-slurm v1.0.38 overlay, serialized image imports, workspace/tokenizer mounts, and the shared synthetic-acceptance injector used by GB300.\n\nValidation: bash syntax, YAML parsing, and git diff whitespace checks pass.\n\n根因：SemiAnalysis InferenceX 缺少已验证的 B300 Dynamo-SGLang DSpark6 AgentX 配置，包括会话亲和路由，以及 DEP4/DEP8 所需的主机端 HiCache 容量。\n\n修复：新增 DEP4/DEP8 聚合 recipe 与 B300 master sweep，并加入 DSpark checkpoint 选择、srt-slurm v1.0.38 overlay、镜像导入串行化、workspace/tokenizer 挂载，以及与 GB300 共用的合成验收率注入逻辑。\n\n验证：Bash 语法、YAML 解析及 git diff 空白检查均通过。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Root cause: the new B300 performance key was not yet registered in the append-only performance changelog.\n\nFix: append the B300 DSpark6 Dynamo-SGLang AgentX configuration and PR link.\n\nValidation: YAML parsing and git diff whitespace checks pass.\n\n根因：新的 B300 性能配置键尚未登记到只追加的性能变更日志。\n\n修复：追加 B300 DSpark6 Dynamo-SGLang AgentX 配置及 PR 链接。\n\n验证：YAML 解析及 git diff 空白检查均通过。
|
Claude finished @weireweire's task in 8m 48s —— View job Review of PR #2624 — 1 blocking issue, 1 warning
Findings🔴 BLOCKING — 🟡 WARNING — both new recipes omit Checked and OK
One note (not blocking): the DEP4 recipe uses |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| additional-settings: | ||
| - "DECODE_NODES=1" | ||
|
|
||
| dsv4-fp4-b300-dynamo-sglang-agentic-mtp-agg: |
There was a problem hiding this comment.
🔴 BLOCKING: This new config block is inserted in the middle of glm5.1-fp8-b200-tilert's fixed-seq-len scenario list, splitting it in two.
Why it matters: The tilert config's second point (- isl: 8192 / osl: 1024, now at lines 8609–8628) sits at the same 4-space indent as this config's agentic-coding: list items, so YAML re-attaches it as a third entry of dsv4-fp4-b300-dynamo-sglang-agentic-mtp-agg's agentic-coding scenario. Two concrete breakages:
glm5.1-fp8-b200-tilertsilently loses its 8k1k benchmark point from the matrix.- The new B300 AgentX config gains a bogus fixed-seq-len-shaped entry carrying tilert settings (
tp: 8, ep: 1,PREFILL_IMAGE=vllm/vllm-openai:v0.26.0,DECODE_NODES=1) and noCONFIG_FILE— when the sweep reaches it,launch_b300-nv.shexits withError: CONFIG_FILE is not set(or the matrix generator chokes onisl/oslinsideagentic-codingfirst).
The file still parses as valid YAML, which is why the PR's YAML-parse validation didn't catch it.
Fix: Move the entire dsv4-fp4-b300-dynamo-sglang-agentic-mtp-agg block (lines 8558–8608) to the end of the file, after the glm5.1 config's final - "DECODE_NODES=1" (line 8628).
|
|
||
| dsv4-fp4-b300-dynamo-sglang-agentic-mtp-agg: | ||
| image: lmsysorg/sglang:nightly-dev-cu13-20260816-4a6dc267 | ||
| model: deepseek-ai/DeepSeek-V4-Pro-DSpark | ||
| model-prefix: dsv4 | ||
| runner: cluster:b300-nv | ||
| precision: fp4 | ||
| framework: dynamo-sglang | ||
| router: { name: dynamo-router, version: "7a4e47ead90ec6b479e397996f6479ce7d251510" } | ||
| multinode: true | ||
| disagg: false | ||
| scenarios: | ||
| agentic-coding: | ||
| - dram-utilization: 0.95 | ||
| search-space: | ||
| - spec-decoding: draft_model | ||
| conc-list: [48, 64] | ||
| kv-offloading: dram | ||
| kv-offload-backend: { name: hicache } | ||
| prefill: | ||
| num-worker: 1 | ||
| tp: 4 | ||
| ep: 4 | ||
| dp-attn: true | ||
| additional-settings: | ||
| - "SYNTHETIC_ACCEPTANCE=true" |
There was a problem hiding this comment.
🔴 The new dsv4-fp4-b300-dynamo-sglang-agentic-mtp-agg block (configs/nvidia-master.yaml:8558-8583) is inserted mid-list inside glm5.1-fp8-b200-tilert's fixed-seq-len scenario, right before its pre-existing - isl: 8192 item. Since that item sits at the same 4-space indent as the new block's - dram-utilization: 0.95 item, YAML re-parents it into the new agentic-coding list — corrupting both configs: the new B300 AgentX sweep gains a bogus fixed-seq-len-style scenario carrying incompatible vllm PREFILL_IMAGE settings, and glm5.1-fp8-b200-tilert silently loses its isl:8192/osl:1024 sweep point. Fix by moving the new top-level key to after the complete glm5.1 entry (i.e. after - isl: 8192 ... DECODE_NODES=1).
Extended reasoning...
What the bug is: In configs/nvidia-master.yaml, the new top-level key dsv4-fp4-b300-dynamo-sglang-agentic-mtp-agg is inserted at line 8558, in the middle of the pre-existing glm5.1-fp8-b200-tilert entry. Specifically, it lands between the first and second items of glm5.1-fp8-b200-tilert.scenarios.fixed-seq-len, right before the pre-existing - isl: 8192 list item.
Why this corrupts the YAML structure: The new block's own scenario list item, - dram-utilization: 0.95, is indented at 4 spaces under agentic-coding:. The pre-existing - isl: 8192 item (glm5.1's second fixed-seq-len entry) happens to sit at that exact same 4-space indentation. Because YAML determines list membership purely from indentation of the - markers, the parser has no way to know - isl: 8192 was meant to continue the previous key's list — it reads as a second element of the new key's agentic-coding list instead.
Verified directly against the tree with yaml.safe_load:
dsv4 agentic-coding entries: 2
0 ['dram-utilization', 'search-space']
1 ['isl', 'osl', 'search-space'] # <- stolen from glm5.1
glm5.1 fixed-seq-len entries: 1
0 {'isl': 1024, 'osl': 1024} # <- lost its isl:8192/osl:1024 entry
This confirms both halves of the corruption in the current working tree, not just in the diff context.
Concrete proof/walkthrough:
- Before this PR,
glm5.1-fp8-b200-tilert.scenarios.fixed-seq-lenhad 2 entries:{isl:1024,osl:1024}and{isl:8192,osl:1024}. - The PR inserts the new
dsv4-fp4-b300-dynamo-sglang-agentic-mtp-aggkey directly after glm5.1's first fixed-seq-len entry ends (... DECODE_NODES=1) and directly before the second entry's- isl: 8192line. - YAML parses top-to-bottom by indentation; once it sees the new top-level key
dsv4-fp4-...:, it starts a fresh mapping. Itsscenarios.agentic-codinglist begins with- dram-utilization: 0.95at 4-space indent. - The next line at that same 4-space indent,
- isl: 8192, is indistinguishable (structurally) from a second element ofagentic-coding, so it — and everything nested under it through the eval/mtp/vllm-prefill scenario — becomesagentic-coding[1]. - Result:
glm5.1-fp8-b200-tilertsilently drops its 8k/1k benchmark point from the sweep, and the brand-new B300 dynamo-sglang AgentX config picks up a nonsensical second scenario entry carryingspec-decoding: mtp,PREFILL_IMAGE=vllm/vllm-openai:v0.26.0, and other vLLM-specific settings that don't apply to a dynamo-sglang aggregate config.
Why nothing else catches this: YAML syntax is fully valid (no parse error), so any lint/syntax check passes. The bug is purely structural/semantic — a silent mis-nesting — which won't be caught by tests unless something actually diffs scenario counts per top-level config before/after.
Impact: Two sweeps are broken: the glm5.1 config silently loses a benchmark data point (no error, just missing coverage), and the new B300 AgentX sweep would attempt to run a scenario with settings incompatible with its framework (dynamo-sglang vs. vllm PREFILL_IMAGE), likely causing that scenario to fail or produce meaningless results.
Fix: Move the new dsv4-fp4-b300-dynamo-sglang-agentic-mtp-agg: block to after the complete glm5.1-fp8-b200-tilert entry finishes (i.e., after its - isl: 8192 ... "DECODE_NODES=1" block), rather than splitting it apart.
| import_squash() { | ||
| local squash_file="$1" | ||
| local image_ref="$2" | ||
|
|
||
| srun -N 1 -A "$SLURM_ACCOUNT" -p "$SLURM_PARTITION" \ | ||
| bash -s -- "$squash_file" "$image_ref" <<'IMPORT_SQUASH' | ||
| set -euo pipefail | ||
| squash_file="$1" | ||
| image_ref="$2" | ||
| lock_dir="$(dirname "$squash_file")/.locks" | ||
| mkdir -p "$lock_dir" | ||
| lock_file="$lock_dir/$(basename "$squash_file").lock" | ||
|
|
||
| ( | ||
| flock -w 3600 9 || { | ||
| echo "Failed to acquire lock for $squash_file" >&2 | ||
| exit 1 | ||
| } | ||
| if unsquashfs -l "$squash_file" > /dev/null 2>&1; then | ||
| echo "Squash file already exists and is valid, skipping import: $squash_file" | ||
| else | ||
| rm -f "$squash_file" | ||
| enroot import -o "$squash_file" "docker://$image_ref" | ||
| unsquashfs -l "$squash_file" > /dev/null 2>&1 || { | ||
| echo "enroot import did not produce a valid squash file: $squash_file" >&2 | ||
| exit 1 | ||
| } | ||
| chmod a+r "$squash_file" || true | ||
| fi | ||
| ) 9>"$lock_file" |
There was a problem hiding this comment.
🟡 sweep:import_squash()
runners/launch_b300-nv.sh:157-188 defines its own import_squash() that duplicates the existing runners/launch_gb300-nv.sh:124-137 (flock + unsquashfs validity check + serialized enroot import); both files already source runners/slurm_utils.sh. Consider factoring a single parameterized helper there (lock path, timeout, post-import re-validation differ) so future locking/import fixes land in one place instead of two.
Extended reasoning...
What/where: runners/launch_b300-nv.sh:157-188 introduces a new import_squash() shell function implementing enroot-container-squash import under a per-file flock, with an unsquashfs -l validity check used both as a cache-hit test and post-import re-validation. runners/launch_gb300-nv.sh:124-137 already has an import_squash() doing the same core job: srun onto a compute node, flock -w N 9 on a lock file, unsquashfs -l as a cache-hit check, and rm -f + enroot import on a miss. This PR's launcher now carries a second, independently-maintained copy of that concurrency-control logic.
Why it matters: Both launchers already source runners/slurm_utils.sh (added at launch_b300-nv.sh:3 in this very PR, and already present in launch_gb300-nv.sh:8), and that file already centralizes other cross-launcher helpers such as inject_synthetic_acceptance. That makes it the natural home for this locking/import logic too. As written, any future fix to the concurrency handling — e.g. a lock-acquisition bug, a corrupted-squash edge case, or a change to how validity is checked — has to be found and applied twice, in two Bash scripts that will inevitably drift further apart over time.
Addressing the refutation: One reviewer objected that the two implementations aren't a literal copy-paste and that unifying them isn't a clean, risk-free move. That's correct as far as it goes — b300's version uses a quoted heredoc (bash -s -- ... <<'IMPORT_SQUASH') instead of gb300's inline bash -c "..." string interpolation, a dedicated .locks/ subdirectory instead of a sibling .lock file, a 3600s vs 600s flock timeout, and adds a post-import re-validation + chmod a+r step gb300 doesn't have. But none of that contradicts the underlying finding: the duplication is real regardless of the two versions not being byte-identical, and a shared helper is still very feasible — it would just need to take the differing lock path, timeout, and 'validate-after-import' behavior as parameters rather than being a straight copy/move. That's a normal refactor, not a rewrite, and it would also fix a real behavioral gap: gb300's version doesn't re-validate after a fresh import (see its own comment noting this), so consolidating would incidentally harden gb300 rather than degrade b300.
Concrete proof of duplication: Compare the core bodies. gb300 (launch_gb300-nv.sh:124-137): acquire flock on ${squash}.lock; unsquashfs -l "" — if valid, skip; else rm -f + enroot import -o "" "docker://". b300 (launch_b300-nv.sh:157-188, new): acquire flock on .locks/$squash_file.lock; unsquashfs -l "" — if valid, skip; else rm -f + enroot import -o "" "docker://" + re-validate + chmod a+r. Strip the cosmetic differences (variable names, invocation mechanism, lock path, timeout) and the control flow, lock discipline, and validity-check semantics are identical — this is the same helper reimplemented with small variations, not two independently-designed pieces of logic.
Suggested fix: Move a parameterized import_squash() into runners/slurm_utils.sh, taking squash_file, image_ref, slurm_account, slurm_partition, an optional lock timeout, and an optional 'revalidate after import' flag, and have both launch_b300-nv.sh and launch_gb300-nv.sh call it. This is a nit — it doesn't block merging this PR — but it's worth doing before a third launcher needs the same logic.
| aggregated_environment: | ||
| PIP_BREAK_SYSTEM_PACKAGES: "1" | ||
| SGLANG_RAGGED_VERIFY_MODE: "static" | ||
| SGLANG_DEFAULT_THINKING: "1" |
There was a problem hiding this comment.
🟡 WARNING: Missing SGLANG_DSV4_REASONING_EFFORT: high in aggregated_environment.
Why it matters: Every existing DSV4 AgentX recipe sets it in each worker env block (GB300 agg-gb300-tp4/tp8-mtp-lowlatency.yaml, all four disagg-gb300-*-mtp-kvoffload.yaml, and agg-h200-tp8-mtp-kvoffload.yaml). Without it the model runs at the default reasoning effort, which changes generation length/profile — so the B300 curve isn't comparable with the GB300/H200 DSV4 AgentX results this PR claims parity with, and the golden acceptance length 3.82 (measured at effort high) no longer matches what eval-only real-verify runs will observe.
Fix:
| SGLANG_DEFAULT_THINKING: "1" | |
| SGLANG_DEFAULT_THINKING: "1" | |
| SGLANG_DSV4_REASONING_EFFORT: high |
(Same fix needed in agg-b300-dep8-mtp-kvoffload.yaml line 60.)
| aggregated_environment: | ||
| PIP_BREAK_SYSTEM_PACKAGES: "1" | ||
| SGLANG_RAGGED_VERIFY_MODE: "static" | ||
| SGLANG_DEFAULT_THINKING: "1" |
There was a problem hiding this comment.
🟡 WARNING: Same as DEP4 — missing SGLANG_DSV4_REASONING_EFFORT: high, which every other DSV4 AgentX recipe (GB300 agg/disagg, H200) sets in each worker env block. Affects comparability and the effort-high-derived golden AL 3.82.
Fix:
| SGLANG_DEFAULT_THINKING: "1" | |
| SGLANG_DEFAULT_THINKING: "1" | |
| SGLANG_DSV4_REASONING_EFFORT: high |
8a4a0a0 to
d9a6672
Compare
Root cause: the B300 PR was stacked on the open GB300 DSpark6 branch, so its non-main base prevented the performance sweep workflow from triggering.\n\nFix: restore every GB300-only recipe, runner, golden curve, master entry, and changelog entry to main while retaining only the B300 DEP4/DEP8 configuration and support changes.\n\nValidation: the resulting diff against main contains only six B300-related files; Bash syntax, YAML parsing, and whitespace checks pass.\n\n根因：B300 PR 堆叠在尚未合并的 GB300 DSpark6 分支上，非 main base 导致性能 sweep 工作流无法触发。\n\n修复：将所有 GB300 专属 recipe、runner、golden curve、master 和 changelog 改动恢复为 main，仅保留 B300 DEP4/DEP8 配置及必要支持。\n\n验证：相对 main 的最终差异仅包含六个 B300 相关文件；Bash 语法、YAML 解析及空白检查均通过。
Root cause: the B300 DSpark6 sweep used acceptance length 3.82, but the DeepSeek-V4-Pro-0813 probabilistic golden curve measures thinking-on block size 6 at 3.77.\n\nFix: set both B300 DEP4 and DEP8 synthetic acceptance lengths to 3.77 and add the complete thinking-on/off block-size curve as the committed source of truth.\n\nValidation: YAML parsing passes and an automated consistency check confirms both master sweep arms equal the thinking-on block-size-6 golden value.\n\n根因：B300 DSpark6 sweep 使用了 3.82，但 DeepSeek-V4-Pro-0813 概率 golden 曲线中 thinking-on、block size 6 的测量值为 3.77。\n\n修复：将 B300 DEP4 和 DEP8 的合成 acceptance length 都改为 3.77，并提交完整的 thinking-on/off block-size 曲线作为依据。\n\n验证：YAML 解析通过，自动一致性检查确认两个 master sweep arm 都等于 thinking-on block-size-6 的 golden 值。
Root cause: the B300 master entry was inserted between two fixed-sequence scenarios of the preceding configuration, so YAML parsed the trailing 8K scenario as a second AgentX scenario with forbidden isl/osl fields.\n\nFix: move the complete B300 entry after the preceding configuration's final scenario at the true top level.\n\nValidation: the repository master-config schema loader accepts the entry and reports only the agentic-coding scenario group; YAML and whitespace checks pass.\n\n根因：B300 master 条目被插入到前一个配置的两个 fixed-sequence scenario 之间，导致 YAML 将后续 8K scenario 解析成带有非法 isl/osl 字段的第二个 AgentX scenario。\n\n修复：将完整 B300 条目移动到前一配置最后一个 scenario 之后，作为真正的顶层配置。\n\n验证：仓库 master-config schema loader 已接受该条目，且只报告 agentic-coding scenario group；YAML 与空白检查通过。
Root cause: DEP8 hicache-ratio 2.0 allocates about 1.545 TB of explicit host KV pools. Together with model loading, JIT, compiler, and page-cache overhead, startup exceeds the B300 x86 node host-memory budget and is OOM-killed before AgentX begins. Fix: cap the DEP8 HiCache ratio at 1.5, reducing the explicit host KV allocation by about 386 GB while retaining host prefix-cache offload. Validation: YAML parsing, whitespace checks, perf changelog validation, and exact-key matrix generation pass for all seven AgentX throughput points. The repository has no configured pre-commit executable in this checkout. 根因：DEP8 的 hicache-ratio 2.0 会分配约 1.545 TB 的显式主机 KV 池；叠加模型加载、JIT、编译器和页缓存开销后，启动阶段超过 B300 x86 节点的主机内存预算，在 AgentX 开始前被 OOM 杀死。 修复：将 DEP8 HiCache ratio 限制为 1.5，使显式主机 KV 分配减少约 386 GB，同时保留主机端 prefix cache offload。 验证：YAML 解析、空白检查、性能变更日志校验以及精确配置键的矩阵生成均通过，覆盖全部七个 AgentX 吞吐点；当前 checkout 未配置可执行的 pre-commit。
Root cause: the HiCache fix changed only the checked-in recipe, while the Run Sweep synchronize trigger is path-filtered to perf-changelog.yaml, so no workflow was created for the repaired commit. Fix: record the DEP8 ratio 1.5 host-DRAM constraint in the PR changelog entry, satisfying the benchmark-change invariant and triggering a fresh sweep. Validation: perf-changelog.yaml parses successfully, remains append-only relative to main, and whitespace checks pass. 根因：HiCache 修复只修改了提交进仓库的 recipe，而 Run Sweep 的 synchronize 触发器只监听 perf-changelog.yaml，因此修复 commit 没有创建 workflow。 修复：在本 PR 的 changelog 条目中记录 DEP8 ratio 1.5 的主机 DRAM 约束，满足 benchmark 变更规则并触发新一轮 sweep。 验证：perf-changelog.yaml 解析成功，相对 main 仍保持仅在末尾追加，且空白检查通过。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31959209232 |
…-agentx # Conflicts: # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31972892435 |
Root cause: build_replay_cmd always emitted --tokenizer $MODEL and the B300 DSpark recipe appended a second --tokenizer /model through INFMAX_AIPERF_TOKENIZER. AIPerf Click parsing rejects duplicate tokenizer options before profiling starts. Fix: select INFMAX_AIPERF_TOKENIZER when provided, otherwise fall back to $MODEL, and emit exactly one tokenizer option. Validation: Bash syntax passes; focused command-construction checks verify one --tokenizer /model for the B300 recipe and one --tokenizer org/model for the fallback path; perf changelog YAML parses successfully. 根因：build_replay_cmd 固定生成 --tokenizer $MODEL，而 B300 DSpark recipe 又通过 INFMAX_AIPERF_TOKENIZER 追加 --tokenizer /model。AIPerf 的 Click 参数解析在 profiling 开始前拒绝重复 tokenizer 选项。 修复：存在 INFMAX_AIPERF_TOKENIZER 时优先使用，否则回退到 $MODEL，并且只生成一个 tokenizer 参数。 验证：Bash 语法通过；聚焦命令构造测试确认 B300 recipe 只生成一个 --tokenizer /model，回退路径只生成一个 --tokenizer org/model；性能变更日志 YAML 解析成功。
Root cause: the c128 diagnostic run exceeded the 3600-second Dynamo session-affinity lease, and multiple frontend processes maintain separate affinity state.\n\nFix: use one Dynamo frontend for both B300 topologies and extend the affinity lease to four hours.\n\nValidation: parsed both changed YAML files and ran git diff --check.
Root cause: chunked-prefill-size is DP-global. Values of 8192 resolved to only 2048 tokens per DEP4 rank and 1024 per DEP8 rank, delaying radix-cache materialization for concurrent AgentX branches. The GB300 reference resolves to 8192 tokens per rank and sustains near-theoretical cache hits.\n\nFix: set the B300 DEP4 and DEP8 global chunk budgets to 32768 and 65536 respectively, preserving an 8192-token local chunk on each DP rank.\n\nValidation: parsed both recipes, verified the configured values, and ran git diff --check.
weireweire
added
full-sweep-enabled
and removed
full-sweep-fail-fast
full-sweep-enabled
labels
…-agentx # Conflicts: # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32003776452 |
Root cause: independent DP schedulers can place some ranks in prefill while peers remain in decode, creating uneven expert-parallel work and idle time.\n\nFix: enable SGLang's prefill delayer with its bounded 30-pass delay on both B300 DEP4 and DEP8 AgentX recipes.\n\nValidation: parsed both YAML recipes, passed git diff checks, and successfully generated both recipes with srt-slurm dry-run on the B300 environment.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32025929150 |
|
crashing, we will resubmit |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32039050094 |
2 participants
Summary / 概要
Adds an independent B300 DeepSeek-V4-Pro DSpark6 AgentX submission using Dynamo-SGLang, following the GB300 tuning in #2623 without depending on that PR.
新增独立的 B300 DeepSeek-V4-Pro DSpark6 AgentX 提交，使用 Dynamo-SGLang，并参考 #2623 的 GB300 调优，但不依赖该 PR。
Adds aggregate DEP4 and DEP8 recipes with Dynamo session-affinity routing and HiCache host offload.
Adds the B300 AgentX sweep: DEP4 concurrency 48/64 and DEP8 concurrency 128/256/384/512/576.
Uses the DSpark6 checkpoint, SGLang image, 8K prefill window, 0.02 SWA cache ratio, and synthetic acceptance length 3.77.
Keeps real draft/verify behavior for eval-only runs.
新增带 Dynamo 会话亲和路由及 HiCache 主机卸载的 DEP4、DEP8 聚合 recipe。
新增 B300 AgentX sweep：DEP4 并发 48/64，DEP8 并发 128/256/384/512/576。
使用 DSpark6 checkpoint、SGLang 镜像、8K prefill、0.02 SWA cache ratio，以及 3.77 合成验收长度。
eval-only 运行仍使用真实 draft/verify。
Validation / 验证
Independence / 独立性
The PR targets main directly. Its final diff contains only the B300 recipes, B300 runner support, master sweep entry, tokenizer mount support, and changelog entry; no GB300 configuration changes are included.
本 PR 直接以 main 为 base。最终差异只包含 B300 recipe、B300 runner 支持、master sweep 配置、tokenizer 挂载支持和 changelog，不包含任何 GB300 配置改动。