fix(llm): classify Azure payload-size and image-count 400s as context overflow - #40167
Open
dotCipher wants to merge 1 commit into
Open
fix(llm): classify Azure payload-size and image-count 400s as context overflow#40167dotCipher wants to merge 1 commit into
dotCipher wants to merge 1 commit into
Conversation
… overflow Azure OpenAI enforces two per-request caps that are returned as 400s with prose messages rather than 413s: Request content length exceeded 32 MB limit. Exceeded maximum number of images (50) allowed in the request. Neither matched isContextOverflow, so media-strip compaction never ran and the oversized history was resent on every turn, leaving the session permanently unrecoverable (/compact fails identically). The existing 413 handling and /request entity too large/ pattern already cover this semantic failure for the 413 spelling; these two patterns extend the same treatment to Azure's 400 phrasing. Note the near-miss against /context[_ ]length[_ ]exceeded/ - Azure says "content length", not "context length". Patterns are placed after the exclusions list so throttling and rate-limit messages are still correctly excluded.
github-actions
Bot
added
the
needs:compliance
github-actions
Bot
removed
the
needs:compliance
Contributor
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #39677
Type of change
What does this PR do?
isContextOverflow()is what decides whether a failed request gets media-strip compaction and a retry. Its pattern list only covers token/context-window phrasing, so two Azure caps that really mean "your payload is too big" fall through as generic non-retryable errors:Because the full history is resent on every turn, that leaves the session unrecoverable — the next turn fails identically, and
/compactfails too since it sends the same oversized payload.413 is already handled via
/request entity too large/iplus thestatusCode === 413check added in #14562. Azure just spells the same condition as a 400 with a prose message, so these two patterns extend that existing handling to Azure's phrasing. Worth noting it says "content length", not "context length", which is why/context[_ ]length[_ ]exceeded/idoesn't already catch it.Both patterns are added after the
exclusionslist, so the throttling/rate-limit carve-out from #39623 still takes precedence.How did you verify your code works?
bun test test/provider-error.test.tsinpackages/llm— 3 pass. Added a case asserting both messages classify as overflow, covering the bare message, a status-code-prefixed variant, and the OpenRouter shape where the upstream body arrives escaped insidemetadata.raw.bun run typecheckandbunx prettier --checkon both changed files.readresults, ~70 MB of base64, every turn failing on the 32 MB cap with no way to recover in-product.Screenshots / recordings
N/A — not a UI change.
Checklist