[Performance] Offload multimodal rollout preprocessing from the asyncio event loop

[Qi2yU]

[Performance] Offload multimodal rollout preprocessing from the asyncio event loop

Prerequisite bug fix (8d81326)

This PR ships one prerequisite fix that is orthogonal to asynchronous preprocessing but
is required to run any multimodal rollout with --apply-chat-template +
--rollout-max-prompt-len.

fix(rollout): avoid re-parsing chat-template prompt in multimodal length filter

When --apply-chat-template is on, Sample.prompt is stored as the rendered
chat-template string. In filter_long_prompt (slime/utils/data.py), the multimodal
branch then passed that string back into process_vision_info(sample.prompt, ...), which
expects the original message list. qwen_vl_utils iterates it as messages, hits
message["content"] on a str, and crashes — so multimodal length filtering was
unusable:

TypeError: string indices must be integers, not 'str'
AttributeError: 'str' object has no attribute 'get'

Fix: don't re-parse the prompt. The images/videos were already extracted once in
Dataset.__init__ and stored on sample.multimodal_inputs; reuse them and only run the
processor for the length check:

# sample.prompt is already a chat-template string; reuse the
# multimodal_inputs computed in Dataset.__init__ instead of re-parsing it.
multimodal_inputs = {
    key: value
    for key, value in (sample.multimodal_inputs or {}).items()
    if value is not None
}
processor_output = processor(text=sample.prompt, **multimodal_inputs)

This is an existing bug independent of asynchronous preprocessing; it is included here
so the benchmark below (and any multimodal rollout with length filtering) can run at all.

Motivation

In multimodal RL rollout, every generation request must first transform its raw images
into the inputs required by SGLang: run the HF processor to build prompt_ids /
multimodal_train_inputs, and encode each image to a PNG/base64 string for the
rollout engine. Today both steps run synchronously on the asyncio event loop.

Problem

generate() is an async coroutine, but the two heaviest preprocessing pieces block
the event loop:

1. the HF processor call in _prepare_prompt_ids (CPU-heavy tensor building);
2. encode_image_for_rollout_engine, run serially per image for every sample.

While one sample's images are being processed and encoded, no other rollout coroutine can
make progress — the event loop is stalled. The more images a sample carries, and the
more concurrent requests in a batch, the worse this head-of-line blocking gets. On
image-rich workloads, rollout preprocessing can become a significant bottleneck.

Design

Move the blocking work off the event loop and parallelize per-image encoding, without
changing what gets sent to SGLang.

• Off-loop processor call. _prepare_prompt_ids → _prepare_prompt_ids_async
  (slime/rollout/sglang_rollout.py): the HF processor runs via
  loop.run_in_executor(...) on a dedicated thread pool, so it no longer blocks the
  event loop. The existing synchronous _prepare_prompt_ids stays in the tree as the
  ground-truth reference.
• Concurrent image encoding. encode_image_for_rollout_engine →
  async_encode_image_for_rollout_engine (slime/utils/processing_utils.py), and
  generate() now gathers all images of a sample concurrently:

  payload["image_data"] = await asyncio.gather(
      *(async_encode_image_for_rollout_engine(image) for image in images)
  )

  asyncio.gather preserves image order regardless of completion order.
• Dedicated executor. A module-level ThreadPoolExecutor(thread_name_prefix= "slime-multimodal") keeps this blocking work off both the event loop and asyncio's
  default executor, so it can't starve other async I/O.

This is purely an execution-mode change (sync → async/threaded). It introduces
no new CLI flags and does not touch the request contents, so baseline and
candidate run with identical arguments; the only difference is the code version.

Correctness

Async preprocessing only changes how multimodal inputs are prepared, not what is
produced. We verified the change locally at both the preprocessing and rollout levels:

• CPU preprocessing parity: the asynchronous and synchronous paths produce identical
  prompt_ids and multimodal_train_inputs on the multimodal, text-only, and
  existing-token branches. Blocking work runs outside the event-loop thread, and
  concurrent samples do not cross-contaminate processor outputs.
• Real HF processor parity: prompt_ids and multimodal_train_inputs are identical
  (torch.equal) to the synchronous reference, including under 64-way concurrency.
• SGLang request parity: the payload produced by generate() is identical to the
  synchronous reference, including input_ids, text, ordered image_data, and
  sampling_params.
• End-to-end rollout validation: with --debug-rollout-only, using the same seed and
  dataset order, the saved rollout dumps match on the two groups async preprocessing
  affects:
  • request (prompt-token prefix stored in each completed sample): identical (8/8)
  • processor (multimodal_train_inputs tensors): identical (8/8)
  • response differs only where SGLang inference is non-deterministic (see note).

Note: --sglang-enable-deterministic-inference can't be used in the multimodal path
on our SGLang build — its batch-invariant GEMM crashes DeepGEMM on the Qwen3-VL vision
encoder's small matrices (CUDA_ERROR_INVALID_VALUE). So bit-identical response is
not guaranteed with images. Preprocessing and request-payload parity, together with
the request/processor parity in the saved rollout dumps, pin down correctness
independently of inference determinism.

Performance

E2E rollout time in --debug-rollout-only (SGLang only), sweeping
(rollout_batch_size, n_samples_per_prompt) from small to large so the number of
generation requests — and thus the image-processing pressure — grows from 4 to 128.

Setup

• Model: Qwen3.5-35B-A3B (MoE, with vision encoder)
• Cluster: 4×8 = 32 GPU; 8 SGLang engines (--rollout-num-gpus-per-engine 4)
• Data: real multi-image dataset — 40 samples, avg 5.1 images/sample, max 10 (204
  images total). Image-rich on purpose: preprocessing is a larger share of rollout here.
• max_response_len=1024, near-greedy (--rollout-temperature 1e-6), --rollout-seed 42, no shuffle
• Each point: 3 rollouts; step 0 = warm-up, mean over the rest
• baseline = synchronous preprocessing (4c193f1); candidate = asynchronous preprocessing (4bb098a)

Rollout time (mean, warm-up excluded)

batch_size	n_samples	gen_reqs	baseline (s)	candidate (s)	speedup	time saved
4	1	4	10.37	6.78	1.53×	34.6%
8	1	8	16.80	10.60	1.59×	36.9%
8	2	16	28.11	16.12	1.74×	42.7%
16	2	32	39.45	24.18	1.63×	38.7%
16	4	64	74.83	28.85	2.59×	61.4%
32	4	128	141.35	62.04	2.28×	56.1%

The speedup grows with image-processing pressure: ~1.5× at small batches, 2.3–2.6× at
large batches. At the biggest point (bs=32, n=4, 128 requests) baseline spends ~141s/step
on synchronous preprocessing vs ~62s/step for asynchronous preprocessing.

Per-step raw perf/rollout_time (warm-up first)

batch_size	n_samples	baseline steps (s)	candidate steps (s)
4	1	15.95 / 9.15 / 11.58	13.20 / 6.08 / 7.47
8	1	17.96 / 17.96 / 15.64	13.65 / 10.41 / 10.78
8	2	25.91 / 30.92 / 25.30	16.26 / 16.20 / 16.04
16	2	53.69 / 42.53 / 36.37	25.93 / 33.01 / 15.36
16	4	100.60 / 79.48 / 70.17	42.88 / 33.18 / 24.52
32	4	153.81 / 142.54 / 140.16	69.05 / 62.64 / 61.45

Checklist

• Format code with pre-commit.
• Validate preprocessing, request-payload, and end-to-end rollout parity locally.
• Provide correctness (parity) and speed benchmark results.
• Follow the slime code style guidance.

Contributors

Engine Architecture Group 5, Engine Infrastructure Department, Xiaohongshu
Yu Qi, Zhaokai Luo, Kaicheng Sun