fix(streaming): wrap midstream transport errors in APITimeoutError / APIConnectionError (#3811) - #3813
Open
katariyaVivek wants to merge 1 commit into
Open
Conversation
…APIConnectionError (openai#3811)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When consuming a streaming response (e.g.
client.chat.completions.create(..., stream=True)), transport errors that occur mid-stream (such as a read timeout or dropped socket connection) escaped as rawhttpx.ReadTimeout,httpx2.ReadTimeout, orhttpx2.RemoteProtocolError.Because these exceptions are not subclasses of
openai.APIError(they do not inherit fromAPIConnectionErrororAPITimeoutError), standard exception-handling blocks around API calls:fail to catch the most common streaming failures, breaking client resilience and leaving mid-stream errors inconsistent with the initial request-send path.
Root Cause
Stream.__stream__()andAsyncStream.__stream__()insrc/openai/_streaming.pyiterated over SSE events without exception-wrapping blocks. Unlike_base_client.py(which mapstimeout_exceptions()toAPITimeoutErrorand transport exceptions toAPIConnectionError), the streaming iterator let rawhttpx2/ legacyhttpxexceptions bubble out directly to the consumer.Solution
src/openai/_streaming.py, wrapped the iteration loop inStream.__stream__()andAsyncStream.__stream__():except timeout_exceptions() as err:->raise APITimeoutError(request=request) from errexcept _transport_exceptions() as err:->raise APIConnectionError(request=request) from err_transport_exceptions()covershttpx2.TransportErrorandhttpx.TransportError(when legacy httpx is loaded), matching thetimeout_exceptions()pattern.response.close()/await response.aclose()is safely retained in thefinally:block.src/openai/lib/streaming/_assistants.py, handledAPITimeoutErrorandAPIConnectionErrorinAssistantEventHandlerandAsyncAssistantEventHandler, unwrappingexc.__cause__so that existingon_timeout()andon_exception()callbacks receive the underlying transport exception family, preserving the contract tested bytest_assistant_stream_timeout_callbacks_preserve_httpx2_family.Testing
Added 4 new test cases to
tests/test_httpx2.py:test_chat_stream_midstream_timeout_wrapped(syncReadTimeout->APITimeoutError)test_chat_stream_midstream_connection_error_wrapped(asyncRemoteProtocolError->APIConnectionError)test_chat_stream_midstream_connection_error_wrapped_sync(syncRemoteProtocolError->APIConnectionError)test_chat_stream_midstream_timeout_wrapped_async(asyncReadTimeout->APITimeoutError)Verified:
httpx2.ReadTimeoutandhttpx2.RemoteProtocolErrorescape raw).__cause__correctly chained.tests/test_httpx2.pyandtests/test_streaming.pypass cleanly (41 passed, 1 skipped).Verification
tests/test_httpx2.pytests/test_httpx2.pyandtests/test_streaming.py_constants.py,_httpx2.py, and_exceptions.pyruff checkpassed with 0 errorsruff format --checkpassed (3 files checked)Impact
Consumers iterating over streams can now reliably catch
openai.APIError,openai.APIConnectionError, andopenai.APITimeoutErrorfor all midstream transport failures, consistent with the rest of the SDK.Fixes #3811