Skip to content

feat: upgrade vllm from 0.27.1 to 0.28.0 for CUDA and ROCm backends - #261

Open
axsapronov wants to merge 1 commit into
gpustack:mainfrom
axsapronov:feature/vllm-0.28
Open

feat: upgrade vllm from 0.27.1 to 0.28.0 for CUDA and ROCm backends#261
axsapronov wants to merge 1 commit into
gpustack:mainfrom
axsapronov:feature/vllm-0.28

Conversation

@axsapronov

Copy link
Copy Markdown
Contributor

Key upstream changes in vllm 0.28.0:

  • CUDA torch: 0.28.0 base images (vllm/vllm-openai:v0.28.0-ubuntu2404, vllm/vllm-openai:v0.28.0-cu129-ubuntu2404, vllm/vllm-openai-rocm:v0.28.0).
  • PyTorch: CUDA stays at 2.13.0 (unchanged); ROCm moves from the 2.13.0 mislabel to 2.12.0, which v0.28.0's Dockerfile.rocm_base actually builds (release/2.12).
  • VLLM_OMNI_COMMIT -> 0d582c2da0 (tip of vllm-project/vllm-omni dev/vllm-align, rebased onto vLLM v0.28.0 final 2cf0a6915ce5; main's Dockerfile.ci still targets v0.27.0, so the align branch is pinned by immutable commit).
  • lmcache 0.5.4 and ray 2.54.0 unchanged.

Patch changes:

  • 001_wrong_dp_ray.patch: rewritten for v0.28.0's refactored get_open_port() (fix #50965), which now routes through _get_reserved_port_range() and _get_open_port(start_port, max_attempts). The sequential-port goal is kept: a module-level _next_port cursor feeds start_port so co-located ray DP workers get distinct ports.
  • 002_shm_broadcast_port_race.patch: removed — v0.28.0 ships the fix upstream (shm_broadcast.py binds XPUB to tcp://:0 and reads the port back via zmq.LAST_ENDPOINT; get_open_port no longer imported there).
  • vllm_omni/001_wrong_patch.patch: unchanged; still applies to 0d582c2da0.

Upgrade vllm from 0.27.1 to 0.28.0 across CUDA 13.0, CUDA 12.9, and ROCm 7.2 backends.

Key upstream changes in vllm 0.28.0:
- CUDA torch: 0.28.0 base images (vllm/vllm-openai:v0.28.0-ubuntu2404,
vllm/vllm-openai:v0.28.0-cu129-ubuntu2404, vllm/vllm-openai-rocm:v0.28.0).
- PyTorch: CUDA stays at 2.13.0 (unchanged); ROCm moves from the 2.13.0 mislabel
to 2.12.0, which v0.28.0's Dockerfile.rocm_base actually builds (release/2.12).
- VLLM_OMNI_COMMIT -> 0d582c2da0 (tip of vllm-project/vllm-omni dev/vllm-align,
rebased onto vLLM v0.28.0 final 2cf0a6915ce5; main's Dockerfile.ci still targets
v0.27.0, so the align branch is pinned by immutable commit).
- lmcache 0.5.4 and ray 2.54.0 unchanged.

Patch changes:
- 001_wrong_dp_ray.patch: rewritten for v0.28.0's refactored get_open_port()
  (fix #50965), which now routes through _get_reserved_port_range() and
  _get_open_port(start_port, max_attempts). The sequential-port goal is kept:
  a module-level _next_port cursor feeds start_port so co-located ray DP
  workers get distinct ports.
- 002_shm_broadcast_port_race.patch: removed — v0.28.0 ships the fix upstream
  (shm_broadcast.py binds XPUB to tcp://<ip>:0 and reads the port back via
  zmq.LAST_ENDPOINT; get_open_port no longer imported there).
- vllm_omni/001_wrong_patch.patch: unchanged; still applies to 0d582c2da0.

The cuda and rocm patch trees stay byte-identical. runner.py.json and test
fixtures are left for the GHA pack workflow's merge-runner job to regenerate.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the vLLM version from 0.27.1 to 0.28.0 across CUDA and ROCm configurations, including updates to Dockerfiles, matrix definitions, and documentation. It also adapts the network utility patches for both platforms. Feedback on these patches highlights a potential issue where incrementing the port number to 65536 can cause an OverflowError or ValueError on subsequent socket bindings, and suggests resetting or wrapping the port value to prevent this.

Comment on lines +21 to +25
+ port = _get_open_port(start_port=_next_port)
+ _next_port = port + 1
+ return port


def get_open_ports_list(count: int = 5) -> list[int]:
if port in reserved_port_range:
port = _get_open_port(start_port=reserved_port_range.stop, max_attempts=1000)
+ _next_port = port + 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If port is 65535, then _next_port will be set to 65536. Since port numbers must be in the range [0, 65535], passing 65536 as start_port to _get_open_port on the subsequent call will raise an OverflowError or ValueError during socket binding. To prevent this, we should reset _next_port to None (or wrap it) if it exceeds 65535.

    port = _get_open_port(start_port=_next_port)
    _next_port = (port + 1) if port < 65535 else None
    if port in reserved_port_range:
        port = _get_open_port(start_port=reserved_port_range.stop, max_attempts=1000)
        _next_port = (port + 1) if port < 65535 else None

Comment on lines +21 to +25
+ port = _get_open_port(start_port=_next_port)
+ _next_port = port + 1
+ return port


def get_open_ports_list(count: int = 5) -> list[int]:
if port in reserved_port_range:
port = _get_open_port(start_port=reserved_port_range.stop, max_attempts=1000)
+ _next_port = port + 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If port is 65535, then _next_port will be set to 65536. Since port numbers must be in the range [0, 65535], passing 65536 as start_port to _get_open_port on the subsequent call will raise an OverflowError or ValueError during socket binding. To prevent this, we should reset _next_port to None (or wrap it) if it exceeds 65535.

    port = _get_open_port(start_port=_next_port)
    _next_port = (port + 1) if port < 65535 else None
    if port in reserved_port_range:
        port = _get_open_port(start_port=reserved_port_range.stop, max_attempts=1000)
        _next_port = (port + 1) if port < 65535 else None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant