feat: upgrade vllm from 0.27.1 to 0.28.0 for CUDA and ROCm backends - #261
feat: upgrade vllm from 0.27.1 to 0.28.0 for CUDA and ROCm backends#261axsapronov wants to merge 1 commit into
Conversation
Upgrade vllm from 0.27.1 to 0.28.0 across CUDA 13.0, CUDA 12.9, and ROCm 7.2 backends. Key upstream changes in vllm 0.28.0: - CUDA torch: 0.28.0 base images (vllm/vllm-openai:v0.28.0-ubuntu2404, vllm/vllm-openai:v0.28.0-cu129-ubuntu2404, vllm/vllm-openai-rocm:v0.28.0). - PyTorch: CUDA stays at 2.13.0 (unchanged); ROCm moves from the 2.13.0 mislabel to 2.12.0, which v0.28.0's Dockerfile.rocm_base actually builds (release/2.12). - VLLM_OMNI_COMMIT -> 0d582c2da0 (tip of vllm-project/vllm-omni dev/vllm-align, rebased onto vLLM v0.28.0 final 2cf0a6915ce5; main's Dockerfile.ci still targets v0.27.0, so the align branch is pinned by immutable commit). - lmcache 0.5.4 and ray 2.54.0 unchanged. Patch changes: - 001_wrong_dp_ray.patch: rewritten for v0.28.0's refactored get_open_port() (fix #50965), which now routes through _get_reserved_port_range() and _get_open_port(start_port, max_attempts). The sequential-port goal is kept: a module-level _next_port cursor feeds start_port so co-located ray DP workers get distinct ports. - 002_shm_broadcast_port_race.patch: removed — v0.28.0 ships the fix upstream (shm_broadcast.py binds XPUB to tcp://<ip>:0 and reads the port back via zmq.LAST_ENDPOINT; get_open_port no longer imported there). - vllm_omni/001_wrong_patch.patch: unchanged; still applies to 0d582c2da0. The cuda and rocm patch trees stay byte-identical. runner.py.json and test fixtures are left for the GHA pack workflow's merge-runner job to regenerate.
There was a problem hiding this comment.
Code Review
This pull request updates the vLLM version from 0.27.1 to 0.28.0 across CUDA and ROCm configurations, including updates to Dockerfiles, matrix definitions, and documentation. It also adapts the network utility patches for both platforms. Feedback on these patches highlights a potential issue where incrementing the port number to 65536 can cause an OverflowError or ValueError on subsequent socket bindings, and suggests resetting or wrapping the port value to prevent this.
| + port = _get_open_port(start_port=_next_port) | ||
| + _next_port = port + 1 | ||
| + return port | ||
|
|
||
|
|
||
| def get_open_ports_list(count: int = 5) -> list[int]: | ||
| if port in reserved_port_range: | ||
| port = _get_open_port(start_port=reserved_port_range.stop, max_attempts=1000) | ||
| + _next_port = port + 1 |
There was a problem hiding this comment.
If port is 65535, then _next_port will be set to 65536. Since port numbers must be in the range [0, 65535], passing 65536 as start_port to _get_open_port on the subsequent call will raise an OverflowError or ValueError during socket binding. To prevent this, we should reset _next_port to None (or wrap it) if it exceeds 65535.
port = _get_open_port(start_port=_next_port)
_next_port = (port + 1) if port < 65535 else None
if port in reserved_port_range:
port = _get_open_port(start_port=reserved_port_range.stop, max_attempts=1000)
_next_port = (port + 1) if port < 65535 else None
| + port = _get_open_port(start_port=_next_port) | ||
| + _next_port = port + 1 | ||
| + return port | ||
|
|
||
|
|
||
| def get_open_ports_list(count: int = 5) -> list[int]: | ||
| if port in reserved_port_range: | ||
| port = _get_open_port(start_port=reserved_port_range.stop, max_attempts=1000) | ||
| + _next_port = port + 1 |
There was a problem hiding this comment.
If port is 65535, then _next_port will be set to 65536. Since port numbers must be in the range [0, 65535], passing 65536 as start_port to _get_open_port on the subsequent call will raise an OverflowError or ValueError during socket binding. To prevent this, we should reset _next_port to None (or wrap it) if it exceeds 65535.
port = _get_open_port(start_port=_next_port)
_next_port = (port + 1) if port < 65535 else None
if port in reserved_port_range:
port = _get_open_port(start_port=reserved_port_range.stop, max_attempts=1000)
_next_port = (port + 1) if port < 65535 else None
Key upstream changes in vllm 0.28.0:
Patch changes: