fix(api): pass abortSignal to streaming API calls for all providers (#404)#1
Open
easonLiangWorldedtech wants to merge 2 commits into
Open
fix(api): pass abortSignal to streaming API calls for all providers (#404)#1easonLiangWorldedtech wants to merge 2 commits into
easonLiangWorldedtech wants to merge 2 commits into
Conversation
easonLiangWorldedtech
force-pushed
the
fix/abort-signal-openai-compatible
branch
4 times, most recently
from
277e296 to
c674eec
Compare
easonLiangWorldedtech
force-pushed
the
fix/abort-signal-openai-compatible
branch
5 times, most recently
from
79dd3fb to
c5a6f53
Compare
easonLiangWorldedtech
force-pushed
the
main
branch
from
13ba72d to
1b0e73f
Compare
easonLiangWorldedtech
force-pushed
the
fix/abort-signal-openai-compatible
branch
3 times, most recently
from
fab4533 to
b1c1c4f
Compare
easonLiangWorldedtech
force-pushed
the
main
branch
from
1b0e73f to
6bcd398
Compare
easonLiangWorldedtech
force-pushed
the
fix/abort-signal-openai-compatible
branch
5 times, most recently
from
19d668d to
b1a2469
Compare
…ode-Org#434) - Add AbortController support to openai-compatible provider - Pass abort signal through Task execution chain - Add comprehensive tests for abort signal behavior across all providers
easonLiangWorldedtech
force-pushed
the
fix/abort-signal-openai-compatible
branch
2 times, most recently
from
cdb9a74 to
cdfbc4b
Compare
Previously, these three providers used their own AbortControllers but did not listen to the external abort signal from the Task execution chain. This meant clicking stop would not immediately cancel requests to these providers. Now all providers consistently respect the user's stop action. Also add optional metadata parameter with abortSignal to completePrompt methods across 22 provider implementations: - Anthropic, Anthropic Vertex, Base OpenAI Compatible - Gemini/Vertex, LiteLLM, LM Studio, MiniMax, Mistral - Native Ollama, OpenAI, OpenAI Compatible, OpenRouter - Qwen Code, Requesty, Unbound, Vercel AI Gateway - Opencode Go, xAI, Zoo Gateway, VSCode LM, Poe, Bedrock Add abortSignal tests for all modified providers. - Add comment explaining Mistral completePrompt does not support non-streaming abort (Mistral SDK limitation)
easonLiangWorldedtech
force-pushed
the
fix/abort-signal-openai-compatible
branch
from
cdfbc4b to
32de395
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes Zoo-Code-Org#404 — When the user clicks Stop during streaming, the HTTP request to the provider continues running because the abort signal wasn't being passed through. This wastes API tokens/compute on the provider side.
Changes
Added
{ signal: metadata?.abortSignal }to streaming.chat.completions.create()calls in 13 providers:Also added
abortSignal?: AbortSignaltoApiHandlerCreateMessageMetadatainterface and wired it through Task.ts's AbortController.Test Coverage
p.s All code generated by AI, this is the testing video(I only tested LM Studio)
2026-06-01.230625.mp4