Skip to content

refactor(api): move hand-written abort bridges to the shared helper #2486

Open

Description

@edelauna

Seven providers on main bridge the caller's abortSignal with their own AbortController and abort listener. bridgeExternalAbortSignal in src/api/providers/utils/abort-signal.ts does the same work. anthropic.ts, anthropic-vertex.ts, minimax.ts and xai.ts already use it. Each hand-written copy adds one more place for a listener leak or a missed pre-aborted check.

Sites at origin/main 66b07e0e65:

  • bedrock.ts (~593): its 10-minute timer also aborts the request controller.
  • lm-studio.ts (~137) and qwen-code.ts (~293): both need a signal even with no caller signal, and abort the controller in finally.
  • openai-native.ts (~471, attachExternalAbort): the controller is exposed to the stop-button path.
  • openai-codex.ts (~524): both transports abort through the controller.
  • native-ollama.ts (~481 and ~736): the abort callback also calls client.abort().
  • vscode-lm.ts (~1198 and ~1684): the callback cancels a token source and records abortedByCaller.

fetchers/modelCache.ts has two abort listeners. They are not request bridges, so leave them as they are.

The helper returns only a signal, and returns undefined when there is no caller signal. Most sites above need a controller or an extra callback. Extend the helper first, for example with an optional controller and an onAbort hook. Then migrate each site and keep its provider tests green.

Also in scope:

  • main has two createTimeoutError helpers: one in utils/abort-signal.ts and one in utils/request-timeout.ts with a phase argument. Keep one. Move the callers.
  • After the migration, add a lint selector in packages/config-eslint/api-provider-hygiene.js that flags addEventListener("abort", …) outside the helper. Add one that flags a hand-built TimeoutError. Both need zero violations when they land.

Context: review of #1535 and the open abort PRs #1545 and #1311.

Activity

  1. edelauna commented on Nov 18, 2026

    @edelauna
    ContributorAuthor

    @easonLiangWorldedtech - could you look at redoing some of the previously merged prs.

  2. easonLiangWorldedtech commented on Nov 18, 2026

    @easonLiangWorldedtech
    Contributor

    Ok, I will check it.

  3. easonLiangWorldedtech commented on Nov 18, 2026

    @easonLiangWorldedtech
    Contributor

    Split plan (posted before any split PR opens)

    Shared-symbol owner. src/api/providers/utils/abort-signal.ts and src/api/providers/utils/request-timeout.ts belong to the chain owner (@easonLiangWorldedtech). Split PRs may touch them only to resolve a merge conflict with main; every helper API change lands in U0a/U0b first and the provider units rebase onto them.

    Per-unit budget, measured against each unit's own base: <= 400 a+d (hard ceiling 1000), <= 500 changed executable lines, <= 400 valid mutants. Every new branch gets a test plus a negative control. src/eslint-suppressions.json counts never increase.

    Remaining hand-written bridges at main 75a93a9d9acc (git grep -c 'addEventListener("abort")' src/api/providers — production files only): bedrock.ts 1, lm-studio.ts 1, native-ollama.ts 2, openai-codex.ts 1, openai-native.ts 1, qwen-code.ts 1, vscode-lm.ts 2 — seven provider files, matching this issue's scope. fetchers/modelCache.ts (2) is the shared fetch layer and stays out of these units.

    Unit Scope Files Lands
    U0a Give bridgeExternalAbortSignal an optional controller/onAbort hook so the abandoned-generator teardown keeps a controller without hand-writing addEventListener("abort"). Additive: default behaviour unchanged. Driver: the openrouter.ts teardown controller pinned by openrouter.spec.ts:1463 (raised on #1545). utils/abort-signal.ts, utils/__tests__/abort-signal.spec.ts 1st
    U0b Collapse the duplicate createTimeoutError definitions (abort-signal.ts:304 single-arg vs request-timeout.ts:23 (providerName, phase)) into the phase-aware one, and add the optional cause the timeout call sites need (raised on #1311 and #1535). Call-site switches in lm-studio.ts / qwen-code.ts are import-line + one-argument edits only, so U2 can rebase cleanly. utils/abort-signal.ts, utils/request-timeout.ts, both specs, friendli.ts (+ import/call lines in lm-studio.ts, qwen-code.ts) 2nd
    U1 bedrock.ts: replace the hand-written bridge with bridgeExternalAbortSignal + dispose in finally. bedrock.ts, __tests__/bedrock.spec.ts after U0b
    U2 lm-studio.ts + qwen-code.ts: same migration, keep their timeout semantics. those 2 providers + lm-studio*.spec.ts, qwen-code*.spec.ts after U0b
    U3 openai-native.ts: same migration (3 controllers today, 1 bridge + teardown). openai-native.ts, openai-native.spec.ts after U0b
    U4 openai-codex.ts: same migration. openai-codex.ts, openai-codex.spec.ts, openai-codex-native-tool-calls.spec.ts after U0b
    U5 native-ollama.ts: same migration (2 bridges). native-ollama.ts, native-ollama.spec.ts after U0b
    U6 vscode-lm.ts: same migration (2 bridges) + github-copilot inherits it. vscode-lm.ts, vscode-lm.spec.ts after U0b
    U7 Add the two hygiene selectors that fail a hand-written bridge at lint time, so new providers cannot reintroduce one. Suppression counts may only fall. packages/config-eslint/api-provider-hygiene.js + its test, src/eslint-suppressions.json last, once U1-U6 are merged

    Merge order: U0a -> U0b -> U1 -> U2 -> U3 -> U4 -> U5 -> U6 -> U7. U1-U6 are independent of each other; they only share the helper base.

    Two notes for the series, not units here:

  4. easonLiangWorldedtech commented on Nov 18, 2026

    @easonLiangWorldedtech
    Contributor

    U6 (Kenari, #2500) realized shape

    #2500 landed the RouterProvider/OpenAI-SDK streaming contract in a form later units should copy. Beyond the promise-shaped completePrompt wiring, review rounds required three stream-side pieces that U1-U6 should treat as part of the convention, not optional polish:

    1. Bridge the caller signal for streams too. The pinned openai SDK does signal.addEventListener('abort', () => controller.abort()) in client.mjs and never removes it, so handing the SDK a live caller signal leaves one listener and one request controller behind per request. createMessage now bridges and disposes on every exit path: the generator finally after the request starts, and the model lookup's rejection handler before it.
    2. Race create(...) with the signal. await rejectOnAbort(pendingCompletion, bridgedCallerSignal, providerName) so a request that never returns settles as the contract AbortError on cancel; keep the one-argument call and plain await when no signal exists.
    3. Guard the emit loop, not just the tail. if (signal?.aborted) break at the top of the for await body (a tail check then turns it into the AbortError) plus a check before emitting items still buffered inside the chunk - the pair openai-codex.ts:567-586 already ships. A tail-only check cannot stop a second tool_call_partial from the same SDK chunk.

    Budget note: those three pieces pushed #2500 to 483 a+d (kenari.ts 109/-37, kenari.spec.ts 335/-2), 83 past the 400-line unit budget, because each arrived with the regression test its finding demanded (five tests, ~90 of the lines are spec). Units that start from this shape should budget for them up front instead of discovering them in review, and U7 should make the emit-loop guard and the bridge enforceable so the next provider does not have to learn them one review round at a time.

  5. easonLiangWorldedtech commented on Nov 19, 2026

    @easonLiangWorldedtech
    Contributor

    Three helper-side candidates came out of the verification work on the abort-signal units. Recorded here so the helper unit has a defined scope when it is cut; none of them is being fixed inside a provider unit.

    1. Give bridgeExternalAbortSignal a cancel handle. Today dispose is only externalSignal.removeEventListener(...) (src/api/providers/utils/abort-signal.ts:103-106), so a handler that exits its generator early can drain the stream but has no request-local signal to cancel with. The Poe unit had to reach for consumeStream() - the only call site in the repo - because there is nothing else to abort: bridgeExternalAbortSignal returns { signal, dispose } and the provider never owns the controller. Returning the controller (or an explicit cancel(reason) alongside dispose) removes that trap for every future provider.
    2. Add createTimeoutError(provider, phase, cause?). The providers currently spell the timeout failure in three shapes - createAbortError reused for a deadline, a hand-built TimeoutError, and throw signal.reason (src/api/providers/utils/abort-signal.ts plus base-provider.ts:184) - and the mutants on those lines survive because the assertion differs per provider. One constructor with a phase argument makes the message a single tested surface.
    3. Decide whether a stalled response body counts as a missed abort. unbound.ts enforces its timeout only through body consumption, so a server that sends headers and then stops mid-body is not covered by the request-level deadline. A helper that arms the deadline against the first byte and then re-arms it per chunk would close it; that is a helper change, not a per-provider stall timer, and it should not be patched provider by provider.

    The two files are src/api/providers/utils/abort-signal.ts and src/api/providers/utils/request-timeout.ts. The provider units stay off both files, so this stays one unit.

  6. easonLiangWorldedtech commented on Nov 19, 2026

    @easonLiangWorldedtech
    Contributor

    Round note (2026-11-19, 20:00Z) - state after the latest fix round

    Seven PRs, none merged. Heads, sizes, what the changed-code mutation instrument actually reported at that head, open review threads, and whose review is actually blocking:

    unit head size mutation-diff at this head threads blocking review
    #1311 openai family 2f0d1d87 29 files +4374/-239 preflight 420 against the 400 limit, run reports Advisory incomplete - 457 changed executable lines, 0 valid mutants, no scoring 48, 0 open maintainer @edelauna CHANGES_REQUESTED 11-18; CodeRabbit APPROVED this head at 19:01Z
    #1535 poe (U1) d6e0c134 3 files +993/-85 blocking 0 0 open @edelauna 11-18; CodeRabbit APPROVED 18:00Z
    #1538 requesty (U5) 2ae76dc5 7 files +1178/-72 blocking 0 0 open @edelauna 11-14 plus a CodeRabbit CHANGES_REQUESTED that belongs to a superseded head - this head has never been reviewed
    #1545 openrouter (U4) 834ffb4b 5 files +3965/-450 322 changed executable lines, 385 valid, 362 killed, blocking 23, all dispositioned row by row in one comment 0 open @edelauna 11-18 plus a superseded CodeRabbit CHANGES_REQUESTED
    #1652 opencode-go (U2) 371445df 2 files +2933/-291 198 changed executable lines, 198 valid, 176 killed, blocking 22, all dispositioned 40, 0 open CodeRabbit CHANGES_REQUESTED 19:41Z - the rows are spec-side only
    #1653 gateway siblings (U3) f395acce 9 files +6036/-386, of which 3224 is the byte-identical #1652 overlap that drops out on merge and rebase preflight 402 against the 400 limit, Advisory incomplete 58, 0 open CodeRabbit CHANGES_REQUESTED 19:40Z - spec-side rows plus two pre-merge checklist rows
    #2500 kenari patch 69c10fc6 2 files +452/-39 provider patch, single provider 0 open REVIEW_REQUIRED - CodeRabbit approved 11-18, no human review yet

    What actually binds size

    No unit is inside 400 additions+deletions, and none needs to be: the operative caps are the instrument's - changed executable lines (198 on #1652, 322 on #1545, 362-378 on #1653, 457 on #1311, all under 500) and valid mutants (limit 400). Two units sit over the mutant cap in preflight, #1311 at 420 and #1653 at 402, and in both cases the run stops before scoring and publishes Advisory incomplete rather than a survivor list. #1311 was already at 413 before this round, so this is a property of the planned unit, not of the fixes added today. Both bodies state the reading and ask the maintainer to choose between a further split and a reasoned narrow exclusion; #1311 names the two surfaces an exclusion would cover first and why.

    Surviving mutants: dispositioned, not suppressed

    No Stryker directive, @ts-ignore or eslint-disable line exists anywhere in the series - verified by diffing each unit against its merge base. The 23 rows on #1545 and the 22 on #1652 are answered one row per line in a single comment each: file, line, status, mutator and replacement, and why the mutated program is not observably different or which test pins it. On #1652 the argument is measured, not asserted: the 17 literal rows are unkillable by assertions because every guard throw sits inside a try whose catch re-throws the contract error with its own unmutated literal and isRequestAborted selects on name === "AbortError", verified by blanking each cited literal (227/227 green, 17 of 17) while blanking the two catch literals fails 8 and 7 tests; the 5 optional-chain rows are equivalent at the call site because the helper hands the streaming paths bridgedSignal ?? new AbortController().signal (removing all five chains at once is still 227/227 green, including the two runs that pass no caller signal).

    Issues opened this round so every unit satisfies the template

    #2508 poe, #2509 openrouter, #2510 requesty, #2511 opencode-go, #2512 unbound/vercel/zoo-gateway - one per provider group so each unit's body carries a real Closes: # line. Three follow-ups came out of verification rather than out of the plan: #2513 (model-cache arms AbortSignal.timeout per fetch, so shuffled runs can surface timing coupling in fetchers/__tests__/modelCache.spec.ts), #2514 (scripts/stryker-diff.mjs spawns the extensionless node_modules/.bin/stryker shim without shell: true, so local Stryker preflight cannot run on Windows - CI is the only authority today), #2515 (kimi-code.spec.ts inherits queued mockResolvedValueOnce fetch responses across tests, so a shuffled order changes the result; CI's declaration order hides it).

    Remaining gate

    CodeRabbit has approved #1311 and #1535 at their current heads, and #2500 earlier. What still stands is a human review - @edelauna's CHANGES_REQUESTED on #1311, #1535, #1538 and #1545 - plus, on #1538 and #1545, a CodeRabbit verdict that belongs to a superseded head and only a fresh review of the current head can replace. Every CI-required context (check-translations, platform-unit-test on both runners, compile, knip, e2e-mock) is green at all seven heads, and mutation-diff is green everywhere it ran, including the two Advisory incomplete runs.

    Merge order and the unit map are unchanged: see comment 6984183124 on this issue.

  7. easonLiangWorldedtech commented on Nov 19, 2026

    @easonLiangWorldedtech
    Contributor

    Final state of this round (2026-11-19, 21:15Z) - supersedes the head values in the 20:00Z note

    Pushes finished and stopped. Per unit: head, size, mutation reading at that head, open threads, and the review verdict actually governing it.

    unit head size mutation reading at this head threads governing verdict
    #1311 openai family e3170ad6 29 files +4459/-240 427 preflight mutants against 400 - advisory only, run not scored 53, 0 open CodeRabbit CHANGES_REQUESTED 19:52Z belongs to head 2f0d1d87; its five rows are fixed in e3170ad6 and each thread carries the answer plus its negative control. The human gate is @edelauna's CHANGES_REQUESTED of 11-18.
    #1535 poe (U1) d6e0c134 3 files +993/-85 blocking 0 0 open CodeRabbit APPROVED 18:00Z; gate is @edelauna 11-18.
    #1538 requesty (U5) 2ae76dc5 7 files +1178/-72 blocking 0 0 open CodeRabbit dismissed its stale CHANGES_REQUESTED at 20:00Z; gate is @edelauna 11-14.
    #1545 openrouter (U4) 834ffb4b 5 files +3965/-450 322 changed executable lines, 385 valid, 362 killed, blocking 23 - all dispositioned row by row 0 open CodeRabbit APPROVED 19:59Z; gate is @edelauna 11-18.
    #1652 opencode-go (U2) 7cbce9dc 2 files +3137/-282 217 generated / 210 valid / 186 killed / 22 surviving, Advisory findings; the 22 are 18 provider-name StringLiteral and 4 OptionalChaining, dispositioned 48, 0 open CodeRabbit APPROVED 21:03Z at this head; label awaiting-maintainer.
    #1653 gateway siblings (U3) a128b0ee 9 files +6240/-377 (3419 of it is the byte-identical #1652 carry) 414 preflight mutants against 400 - advisory only, run not scored, blocking 0 63, 0 open CodeRabbit CHANGES_REQUESTED 21:01Z whose rows are answered in-thread; label awaiting-maintainer.
    #2500 kenari patch 69c10fc6 2 files +452/-39 provider patch 0 open CodeRabbit APPROVED 11-18 20:52Z; no human review yet, so REVIEW_REQUIRED.

    All seven: the six required contexts (check-translations, platform-unit-test on ubuntu and windows, compile, knip, e2e-mock) and mutation-diff are success at the current heads, mergeable: true, mergeStateStatus: BLOCKED. Nothing is merged.

    Two things landed late and matter more than the review rows

    1. Terminal cost over-reported output tokens. opencode-go.ts:805 accumulated each message_delta usage onto the message_start seed, while anthropic.ts:314-316 and minimax.ts:178-180 adopt the reported running total with the comment that the API reports a cumulative count. The Anthropic-format handler here now matches them, and the test that feeds 4 then 6 asserts the cumulative reading (title changed to say so). Control: reverting the line fails exactly that test. CodeRabbit separately argues the yield should be incremental so Task.ts:4319 does not double count; the answer recorded on both units is that the three same-format handlers agree today, and reconciling the convention across providers plus the Task.ts consumer is a series-level decision, not a feat(api): abort signal support for opencode-go #1652 change.
    2. Two guards make cancellation observable in the Responses stream. :645-647 checks the signal immediately before forwarding a processed chunk, because the pre-pull check can pass while the SDK returns a buffered response.completed; :663-665 runs the normal-exit check after the iterator cleanup, because a cancel that lands during streamIterator.return() otherwise resolves done: true and is reported as a normal end. Controls, on the committed tree: disabling the first fails 1 of 232, disabling the second fails 4, and merely moving the second back before the cleanup - same text - fails 1, which is the proof that the position is what makes cancellation-during-cleanup observable. Both branches carry opencode-go.ts at blob da0c605549 and the spec at 9931ea5ea8, byte-identical, verified from the remote refs.

    Mutation posture of the whole series

    Not one Stryker directive, @ts-ignore or eslint-disable line is added by any unit - verified against merge base 75a93a9d9acc by counting matching added lines on every branch. The two unscored runs (#1311 at 427, #1653 at 414 against the 400 preflight limit) are properties of the planned units, not of this round's fixes: #1311 measured 413 before the round opened. The 22 survivors on #1652 include a measured case that no assertion can kill: the provider-name argument passed to createAbortError at :646 and :664 is replaced upstream by the outer normalization catch, so labelling those literals with distinct tags changed 60 test outputs and never printed either tag, while the same experiment on the eight outer-catch literals printed all eight. The mutant on the pricing line at :805 is killed. Suppression counts are unchanged on every branch.

    History and identity

    #1652 and #1653 are each a single commit on 75a93a9d9acc with author and committer Eason Liang <136036952+easonLiangWorldedtech@users.noreply.github.com>; the unit commit messages are unchanged. This needed one force-push per branch, spent together with the pricing fix rather than separately.

    What still needs a maintainer, in order of what unblocks most

    1. The standing human CHANGES_REQUESTED from @edelauna on feat(api): abort signal support for openai, openai-compatible base, zai, kimi-code (round 2) #1311, feat(api): abort signal support for poe (completePrompt + createMessage) #1535, feat(api): abort signal support for requesty (createMessage + kill tests) #1538 and feat(api): abort signal support for openrouter (completePrompt + createMessage) #1545, and a first review on feat(api): cancel Kenari requests through the shared abort helpers #2500. CodeRabbit's verdicts on these heads are approval or already-dismissed, and the open-thread counts are zero, so these four units are blocked on a person, not on work - the same shape as Nine campaign PRs blocked only by a human CHANGES_REQUESTED at a superseded commit #2415.
    2. A decision on the two unscored mutation runs: a further split, or a reasoned narrow exclusion. feat(api): abort signal support for openai, openai-compatible base, zai, kimi-code (round 2) #1311 names the surfaces an exclusion would cover first, and feat(api): abort signal support for unbound, vercel-ai-gateway, zoo-gateway #1653's overage is 14 mutants over the cap.
    3. Whether the cumulative-versus-incremental token convention should be reconciled across the Anthropic-format handlers and the Task.ts consumer.

    Follow-up issues filed this round and left open for their own PRs: #2508 poe, #2509 openrouter, #2510 requesty, #2511 opencode-go, #2512 unbound/vercel/zoo-gateway, #2513 model-cache armed deadline versus shuffled runs, #2514 scripts/stryker-diff.mjs cannot spawn the extensionless Windows shim, #2515 kimi-code.spec.ts inherits queued one-shot responses so a shuffled order changes results.

    Every actionable item recorded against #1311, #1535, #1545 and #1653 today is answered and resolved, so what remains on them is a review by a person rather than work by the author. The awaiting-author labels on those four are the maintainer-side ones to move; the automation token cannot edit labels here (easonLiangWorldedtech does not have the correct permissions to execute AddLabelsToLabelable), so this flags it instead of changing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions