Repository navigation
Cancel the HTTP call when a model stream is closed, drop the openai-java hold - #58
Merged
Merged
Conversation
…ava hold From openai-java 4.66.0 on, closing a stream from another thread closes the OkHttp response under the in-flight read, so a stopped, deleted or timed-out chat turn left the model generating. CancelOnClose makes closing a response body cancel its call first; ChatModels hands it to both OpenAI clients. The four stop tests are green on 4.80.0 and 4.101.0, so openai-java comes from the Spring AI BOM again and its Renovate rule goes.
This was referenced Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the openai-java hold at 4.65.0 with a fix on our side, so #55 is no longer needed.
Cause
From openai-java 4.66.0 on (openai/openai-java#1027),
StreamResponse.close()closes the OkHttp response before the reader. A chat turn is stopped, deleted or timed out by closing its stream from a thread other than the one blocked reading it. That close then runs under the read still in flight: okio throws "Unbalanced enter/exit", the socket stays open, and the model keeps generating an answer nobody reads.Fix
llm/CancelOnClose, an OkHttp interceptor: closing a response body callsCall.cancel()first, which is OkHttp's own thread-safe way to end a call from outside.ChatModelshands it to the sync and the async OpenAI client throughOpenAiSetup's customizer list.strictlyconstraint, theopenaiJavacatalog entry and openai-java's own Renovate rule are gone. openai-java comes from the Spring AI BOM again (4.80.0).docs/decisions/chat.md§ Stop and regenerate has the reasoning,backend/CLAUDE.mda one-line rule, and the CHANGELOG is corrected.Verification
ChatStreamTest,ChatTurnDeadlineTestand twoChatConversationTestcases are red ("the stub saw its connection closed", "the model call was cancelled")../gradlew build: 1304 backend cases, 96 unit and 22 browser test files, all green; the 13 skipped are the corpus tests without.emlsamples.After the merge, #55 points at a catalog entry that no longer exists and can be closed.