Skip to content

fix(train): raise clear error when computed train_iters is 0 - #2366

Draft
Xuan-1998 wants to merge 1 commit into
THUDM:mainfrom
Xuan-1998:fix/train-iters-zero-error
Draft

fix(train): raise clear error when computed train_iters is 0#2366
Xuan-1998 wants to merge 1 commit into
THUDM:mainfrom
Xuan-1998:fix/train-iters-zero-error

Conversation

@Xuan-1998

Copy link
Copy Markdown

What this fixes

Launching training with a rollout configuration whose total sample count is smaller than the global batch size, for example:

--num-rollout 2 --rollout-batch-size 8 --n-samples-per-prompt 8 --global-batch-size 256

computes train_iters = 2 * 8 * 8 // 256 = 0 in get_optimizer_param_scheduler (slime/backends/megatron_utils/model.py). The zero propagates into Megatron's OptimizerParamScheduler, which fails during actor bring-up with a bare AssertionError at megatron/core/optimizer_param_scheduler.py (assert self.lr_decay_steps > 0, no message). Nothing in the traceback points at the slime config knobs that caused it.

What changes

A ValueError is raised right after train_iters is computed, at the point where slime owns the math. The message names the four inputs with their values, shows the product versus global_batch_size, and says how to fix the config:

ValueError: computed train_iters is 0: num_rollout (2) * rollout_batch_size (8) * n_samples_per_prompt (8) = 128 < global_batch_size (256). The optimizer LR schedule needs at least one training step; increase num_rollout, rollout_batch_size, or n_samples_per_prompt, or decrease global_batch_size.

No behavior change for valid configs.

Prior art

Test

tests/test_train_iters_validation.py is a CPU unit test following the megatron-stubbing pattern of tests/test_model_provider_freeze.py: it loads model.py with stubbed megatron.* modules and calls get_optimizer_param_scheduler directly. One case asserts the ValueError names all four inputs; one case asserts a valid config still computes train_iters and constructs the scheduler with the same step counts as before.

tests/test_train_iters_validation.py::test_zero_train_iters_raises_value_error_naming_inputs PASSED
tests/test_train_iters_validation.py::test_valid_config_builds_scheduler PASSED

When num_rollout * rollout_batch_size * n_samples_per_prompt is smaller
than global_batch_size, the floor division in
get_optimizer_param_scheduler computes train_iters = 0. The zero
propagates into Megatron's OptimizerParamScheduler, which dies with a
bare 'assert self.lr_decay_steps > 0' during actor bring-up, giving no
hint about which config knob is wrong.

Raise a ValueError at the point where slime computes train_iters,
naming the four inputs, their product, and how to fix the config.
No behavior change for valid configs. The num_rollout == 0 eval-only
path (THUDM#2296) returns before this function is called and is unaffected;
this covers the remaining 0 < product < global_batch_size case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant