fix: retry transient vLLM adapter reload failures - #778
Open
yaodong-shen wants to merge 1 commit into
Open
Conversation
Retry connection failures and transient gateway responses while reloading dedicated LoRA adapters. Stop immediately when the supervised child exits and report runtime diagnostics after exhaustion. Fixes OpenPipe#678
Author
|
The Prek, Package Install, and TrainerRank GPU workflows are waiting for maintainer approval and created no jobs for this external-fork PR. The focused retry tests and the repository Prek checks pass locally. Could a maintainer approve the workflow runs when convenient? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #678
Retry safety
The request is retried at most four times with exponential backoff. Client errors and read/write failures are not retried because the server may already have applied the non-idempotent adapter load.
Validation
The complete unit suite requires optional backend and GPU dependencies that are not installable in this macOS environment; repository CI installs those extras in its CUDA runner.