Corrupted T2I output from LightLLM + LightX2V colocate serving (/generate_image) while the reference examples/t2i/inference.py is correct

[wfs2018]

Corrupted T2I output from the LightLLM + LightX2V serving stack (colocate /generate_image), while the reference examples/t2i/inference.py is correct

Summary

When serving SenseNova-U1-8B-MoT through the LightLLM + LightX2V colocate API using the published docker image lightx2v/lightllm_lightx2v:20260407 and configs/neopp/neopp_dense.json, every generated image is corrupted — rainbow / block-glitch artifacts with only a faint correct structure underneath (see stack_output_corrupt.png).

The same checkpoint with the same sampling parameters produces correct images through the reference script examples/t2i/inference.py (see reference_output_correct.png). This points to a bug in the serving-stack generation path, not the checkpoint.

Environment

• Docker image: lightx2v/lightllm_lightx2v:20260407
• LightLLM: branch neo_plus_clean
• LightX2V: latest main
• Model: sensenova/SenseNova-U1-8B-MoT (official, standard HF layout, 8 shards)
• GPU: single H100 80GB, --tp 1, colocate
• Gen config: configs/neopp/neopp_dense.json ("attn_type": "flash_attn3", "use_triton_qknorm_rope": true)
• torch 2.8 / CUDA 12.8 (as shipped in the image); flash_attn_3 3.0.0b1 present in the image

Reproduction

# 1) image + container
docker pull lightx2v/lightllm_lightx2v:20260407
docker run -d --name repro --gpus '"device=0"' --ipc=host --network host \
  -v /path/to/data:/data lightx2v/lightllm_lightx2v:20260407 sleep infinity

# 2) runtime deps (as per docs/deployment.md)
docker exec repro bash -lc 'cd /workspace &&
  git clone https://github.com/ModelTC/LightX2V.git &&
  git clone https://github.com/ModelTC/LightLLM.git &&
  cd LightLLM && git checkout neo_plus_clean'

# 3) model
docker exec repro bash -lc 'hf download sensenova/SenseNova-U1-8B-MoT --local-dir /data/SenseNova-U1-8B-MoT'

# 4) launch the colocate API server
docker exec -d repro bash -lc 'cd /workspace/LightLLM &&
  PYTHONPATH=/workspace/LightX2V:/workspace/LightLLM python -m lightllm.server.api_server \
    --model_dir /data/SenseNova-U1-8B-MoT \
    --enable_multimodal_x2i --x2i_server_deploy_mode colocate --x2i_server_used_gpus 1 \
    --x2v_gen_model_config /workspace/LightX2V/configs/neopp/neopp_dense.json \
    --host 0.0.0.0 --port 8000 --max_req_total_len 16384 --mem_fraction 0.6 --tp 1'

# 5) once healthy (GET /health == 200), request a t2i image
curl -s http://127.0.0.1:8000/generate_image -H 'Content-Type: application/json' -d '{
  "inputs": "A photograph of a golden retriever sitting on a park bench in warm autumn afternoon light",
  "parameters": {"width":1024,"height":1024,"steps":50,"guidance_scale":4.0,
                 "image_guidance_scale":1.0,"cfg_norm":0,"timestep_shift":3.0,"num_images":1,"seed":42},
  "multimodal_params":{"images":[]}
}' > out.json
# decode out.json -> images[0] (base64) -> PNG  ==> CORRUPTED (see stack_output_corrupt.png)

Expected: a correct photo. Actual: rainbow/block glitch (stack_output_corrupt.png).

Contrast — the reference script is correct (same model, same params)

python examples/t2i/inference.py \
  --model_path /data/SenseNova-U1-8B-MoT \
  --prompt "A photograph of a golden retriever sitting on a park bench in warm autumn afternoon light" \
  --cfg_scale 4.0 --cfg_norm none --timestep_shift 3.0 --num_steps 50 --seed 42
# ==> correct image (reference_output_correct.png)

What we already ruled out

• Not cfg_norm: both cfg_norm=0 (none) and cfg_norm=2 (global) produce corrupt output.
• Not the attention kernel: the image ships flash_attn_3 3.0.0b1; building/installing the neo FA3 branch (github.com/WANDY666/flash-attention, hopper, sm_90) so that flash_attn_interface resolves to it produces a byte-identical corrupt image (same MD5). Swapping the attention implementation changes nothing.
• Not checkpoint loading: no missing/unexpected keys are reported when loading the model.
• The corruption is universal (every t2i and i2i request) and stable across seeds.

Hypothesis

A numerical / pipeline bug in the neopp generation runner on the serving path only (e.g. decode / normalization / RoPE handling in the colocate x2i path), since the reference examples/t2i/inference.py with identical model + params + attn_backend=flash is correct while the LightLLM+LightX2V colocate path is not.

Attachments

• reference_output_correct.png — reference-script output for the prompt above (correct).
• stack_output_corrupt.png — representative serving-stack output showing the glitch signature. Because the corruption is prompt-independent (every request produces the same class of artifacts), this sample illustrates the failure; a corrupt sample for the exact prompt above looks identical.

Reference script output (examples/t2i/inference.py) — correct:

Serving-stack output (/generate_image, same model + params) — corrupted:

[kingder]

please check sample script

The /generate_image API expects the user to provide the raw prompt string, including all required special tokens and the system prompt. Alternatively, you can use the /v1/chat/completions API (as in the sample script above) and provide a list of messages containing both the default system and user messages. The required prompt will then be constructed automatically.