Skip to content

fix(api): Kenari handler drops caller cancellation and the configured request timeout #2499

Open

Description

@easonLiangWorldedtech

What is missing

The Kenari handler never sees caller cancellation or the configured request timeout. Evidence at main 75a93a9d9acc:

$ git grep -c -F abortSignal upstream/main -- src/api/providers/kenari.ts
(no hits)
$ git grep -n -F signal upstream/main -- src/api/providers/kenari.ts
src/api/providers/kenari.ts:61: async fetchModels(options: GetModelsOptions, signal?: AbortSignal): Promise<ModelRecord> {
src/api/providers/kenari.ts:62:  const fetchOpts: [] | [{ signal: AbortSignal }] = signal ? [{ signal }] : []
  • createMessage (kenari.ts:70-134) awaits this.fetchModel() and await this.client.chat.completions.create(body) (:96) with no request options, no race and no try/catch, so metadata.abortSignal is dropped on the floor.
  • completePrompt(prompt, _options?) (kenari.ts:143) discards the whole CompletePromptOptions (src/api/index.ts:61-63: abortSignal + timeoutMs); the request at :159 is sent without options, and every failure - including a cancellation - is rewrapped as Kenari completion error: <message> (:163), so a cancel surfaces as a plain Error instead of name === "AbortError".
  • fetchModels does accept an optional signal (:61-64), but nothing in the handler ever passes one, and fetchers/kenari.ts has no abortSignal handling either.

Why it matters

Planned fix

One PR, mirroring the merged contract in requesty.ts:265-346 and zoo-gateway.ts:309-397 - no helper API change and no new abstraction:

  1. createMessage: resolve the model through resolveModelWithAbort, pass the caller signal to the SDK as request options, and normalize an SDK abort into createAbortError("Kenari"); ordinary failures keep propagating unchanged.
  2. completePrompt: bridgeExternalAbortSignal(options?.abortSignal) so the SDK listener never lands on the caller's long-lived signal, mergeAbortSignalAndTimeout to fold the timeout into that same per-request signal, resolveModelWithAbort for the lookup, rejectOnAbort around the completion (the SDK does not observe the signal while it sleeps between retries), drop a response that lands after the cancel, forward hasUsableTimeout/toSdkTimeoutMs as the SDK timeout, keep the Kenari completion error: wrapping for non-abort failures, and dispose() the bridge in finally.
  3. Keep the one-argument create(...) call shape when neither cancellation nor a usable timeout is configured, so the existing request-shape assertions in kenari.spec.ts stay valid.

Budget (per the split discipline posted in #2486): a single unit at <= 400 a+d against its own base - the patch measures 398 a+d (provider 152+/37-, spec 246+), well inside the 500 changed-executable-line and 400-valid-mutant ceilings. The new cancellation branches are covered by 19 tests, each pinned by a negative control.

Out of scope: threading a signal through RouterProvider.fetchModel() (#2202), and the helper / createTimeoutError consolidation tracked as U0a/U0b in #2486.

Related: #616 (abort pass-through), #2202 (signal-aware model discovery), #2486 (abort-signal refactor umbrella).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions