From 06daa6a3d5bfa967c56f87428df036cff9d52846 Mon Sep 17 00:00:00 2001 From: chenfeiz0326 <203214996+chenfeiz0326@users.noreply.github.com> Date: Thu, 20 Aug 2026 00:21:37 -0700 Subject: [PATCH] [https://nvbugs/6329155][fix] Raise glm5 tep8 8k1k max_num_tokens to 8192 to fit isl=8192 prefill The perf-sanity aggregated case test_e2e[aggr_upload-glm5_fp4_blackwell-glm5_fp4_tep8_mtp3_8k1k] could never serve a single request. Its server_config declared max_num_tokens: 256 while its only client_config (con2_iter10_8k1k) sends isl=8192, and enable_chunked_prefill is absent (defaults to False), so the whole 8192-token prompt has to fit in one token budget. The OpenAI server rejects any request whose prompt+query length exceeds max_num_tokens, so every request was refused at the API layer before reaching the scheduler: tensorrt_llm.executor.utils.RequestError: The sum of prompt length (8192.0), query length (0) should not exceed max_num_tokens (256) No request was ever served, so the benchmark produced no accounting and the case failed deterministically. Raising the budget to 8192 makes it exactly sufficient for isl=8192 (the check is a strict >). That 8192 is feasible on this node is established two ways: the in-file sibling server_config glm5_fp4_dep8_mtp1_8k1k already runs the same isl=8192/osl=1024 workload with a larger max_num_tokens: 12288, and glm5_fp4_2_nodes_grace_blackwell.yaml declares the same glm5_fp4_tep8_mtp3_8k1k server (same max_batch_size: 4, same client isl/osl) with max_num_tokens: 8192 and is not waived. Verified on B200/computelab (8x B200, node umbriel-b200-093) with the patched config: 20/20 requests successful, 0 failed, total_token_throughput 2226.89 tok/s, all ranks exit 0. Before the change the same case on the same commit produced the RequestError above and no accounting. This change also removes the case's SKIP line from tests/integration/test_lists/waives.txt. The waiver is what stops every pre-merge stage from exercising this config, so leaving it in place would land the fix with no CI coverage; the verified re-run above is what authorizes the unwaive. Only this case's line is removed. nvbugs/6329155, ToT re-check 20260819 (origin/main d0e8baa03b) Signed-off-by: chenfeiz0326 <203214996+chenfeiz0326@users.noreply.github.com> --- tests/integration/test_lists/waives.txt | 1 - tests/scripts/perf-sanity/aggregated/glm5_fp4_blackwell.yaml | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/integration/test_lists/waives.txt b/tests/integration/test_lists/waives.txt index 8cc2375e04db..48ef33a329f2 100644 --- a/tests/integration/test_lists/waives.txt +++ b/tests/integration/test_lists/waives.txt @@ -339,7 +339,6 @@ perf/test_perf_sanity.py::test_e2e[aggr_upload-ctx_only-gb300_glm-5-fp4_8k1k_con perf/test_perf_sanity.py::test_e2e[aggr_upload-deepseek_r1_fp4_v2_grace_blackwell-r1_fp4_v2_dep4_mtp1_1k8k] SKIP (https://nvbugs/6601537) perf/test_perf_sanity.py::test_e2e[aggr_upload-deepseek_r1_fp8_blackwell-r1_fp8_tp8_mtp3_8k1k] SKIP (https://nvbugs/6432948) perf/test_perf_sanity.py::test_e2e[aggr_upload-dynamo_gpt_oss_120b_fp4_blackwell-gpt_oss_fp4_tep4_adp_cutlass_8k1k] SKIP (https://nvbugs/6374910) -perf/test_perf_sanity.py::test_e2e[aggr_upload-glm5_fp4_blackwell-glm5_fp4_tep8_mtp3_8k1k] SKIP (https://nvbugs/6329155) perf/test_perf_sanity.py::test_e2e[aggr_upload-super_ad_blackwell-super_ad_ws1_1k1k] SKIP (https://nvbugs/6153575) perf/test_perf_sanity.py::test_e2e[disagg_upload-e2e-gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL] SKIP (https://nvbugs/6601537) perf/test_perf_sanity.py::test_e2e[disagg_upload-e2e-gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1_ccb-NIXL] SKIP (https://nvbugs/6581075) diff --git a/tests/scripts/perf-sanity/aggregated/glm5_fp4_blackwell.yaml b/tests/scripts/perf-sanity/aggregated/glm5_fp4_blackwell.yaml index 44d859240985..fdaf3a34064b 100644 --- a/tests/scripts/perf-sanity/aggregated/glm5_fp4_blackwell.yaml +++ b/tests/scripts/perf-sanity/aggregated/glm5_fp4_blackwell.yaml @@ -13,7 +13,7 @@ server_configs: moe_expert_parallel_size: 8 pipeline_parallel_size: 1 max_batch_size: 4 - max_num_tokens: 256 + max_num_tokens: 8192 attn_backend: "TRTLLM" enable_attention_dp: false moe_config: