Skip to content

[Fix] Reasoning with tool use fails on OpenAI Compatible endpoints that require the Responses API - #2498

Open
zoomote2[bot] wants to merge 7 commits into
mainfrom
fix/openai-compatible-responses-api-1ybm193i5siuq
Open

zoomote2[bot] wants to merge 7 commits into
mainfrom
fix/openai-compatible-responses-api-1ybm193i5siuq

Conversation

@zoomote2

@zoomote2 zoomote2 Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

​Opened on behalf of @taltas. Follow up by mentioning @roomote, in the web UI, or in Discord.

Related GitHub Issue

Fixes #2040
Closes #1579

#2040 reports the same Azure/OpenAI-compatible error when reasoning and function tools require /responses. #1579 requests Responses API support through a per-profile opt-in setting.

Description

Some OpenAI-compatible endpoints require /responses to combine reasoning with function tools. This adds an opt-in Use Responses API setting for both task messages and single completions such as prompt enhancement and condensation. Chat Completions remains the default. Custom deployment names, headers, reasoning effort, and function tools are retained.

A pure buildResponsesRequest() prepares the request from explicit inputs. It honors streaming, temperature capability, output limits, strict schemas, and Extra Body precedence. Zero and omitted values follow the existing Chat Completions policy. Function allowlists retain their restriction mode and use the flattened Responses shape; malformed function references are rejected before listener registration.

Failed, incomplete, cancelled, and error responses now raise an error instead of appearing successful. Premature stream EOF without a completion event is also rejected. Output-token truncation retains OutputTokenLimitError. Cancellation signals and valid timeout overrides reach the SDK, and external bridge listeners are released on completion, failure, and early generator exit.

Completion and its optional output snapshot now use one observer callback. Item events remain the fallback when a gateway omits the snapshot; an explicit empty snapshot clears captured items. The endpoint fingerprint is computed once per history read, and a redundant Azure guard was removed.

Encrypted reasoning items are captured only for successful task responses. The completed output snapshot sets their order. The existing history hooks persist them with a provider tag and a fingerprint of the effective SDK endpoint; URL credentials and query secrets are not stored in that metadata. Responses replay requires matching ownership after serialization or reload. Raw Responses items are excluded from Chat Completions, while other providers' reasoning formats retain their existing handling. Older untagged items are omitted on the OpenAI-compatible path because their source cannot be verified. A new request clears prior reasoning; failure and early close cannot publish partial reasoning. Single completions do not overwrite task reasoning state.

Dedicated Kimi Code, Moonshot, MiMo, and DeepSeek handlers explicitly ignore a stale Responses opt-in from another profile.

For Azure, explicit /openai/v1 URLs are preserved and use the standard OpenAI client, including modern services.ai.azure.com/openai/v1 endpoints. An older Azure OpenAI root defaults to /openai/v1 when Responses is enabled and no dated version was selected. Explicit dated versions remain available for legacy Azure endpoints. The older Azure AI Inference route rejects this combination with an instruction to use /openai/v1.

The SDD artifacts follow docs/architecture/tla/AGENTS.md. Three models separate request routing/cleanup, request-option rules, and protocol/reasoning history. Their headers record the review ratchet, input domains, findings, costs, and manual gate results. Models check abstractions; they do not prove the implementation correct.

Test Procedure

  • 999 focused tests passed across 13 suites, with one existing skip: OpenAI, DeepSeek, Kimi Code, Moonshot, MiMo, xAI, OpenCode Go, the pure request builder, Responses input/stream/contracts, Chat Completions conversion, and history persistence.
  • Tests cover both completion entry points, o1/o3/o4 routing precedence, SDK cancellation and timeout options, terminal failures, missing/null/empty completion snapshots, token-limit classification, zero/default/omitted options, strict schemas, filtered tools, function allowlists, and malformed restrictions.
  • Reasoning tests cover multiple encrypted items, serialization/reload, tool-call/result pairing, completed-output ordering, ownership of standalone items, provider/endpoint changes, Responses-to-Chat switches, copy isolation, and clearing state after failure, early close, or construction failure.
  • The pre-fix code failed 24 focused regression cases. The current implementation passes the expanded suites.
  • Local patch line coverage across the changed runtime files: 154/155 executable lines (99.4%). Both Codecov patch checks passed at a60cdbcec4; the fresh result for this refactor is pending CI.
  • Extension types, ESLint, and formatting pass. Six existing any suppressions were removed; no suppression count increased.
  • Manual TLC gates pass: routing/cleanup 6,912 states, request parameters 19,440 states, protocol/reasoning 10,368 states. All 29 sabotage configurations and the older unfixed configuration fail as expected.
  • Separately, 30 corresponding code mutations were made by hand and rejected by focused real-code tests. docs/architecture/tla/ResponsesReviewReplays.md maps each sabotage configuration to its test; ownership has separate Responses and Chat Completions mutations. These are manual checks, not an automated model-to-code correspondence proof or a CI gate.
  • To repeat the unit gate, run the affected Vitest suites from src/. To repeat TLC, run each .cfg against its matching *MC.tla module from docs/architecture/tla, using -metadir outside the repository. Fixed configs must pass; sabotage configs must exit with invariant failure 12.
  • Manual endpoint check: enable Responses for a supported gateway, run a reasoning/tool task through multiple turns, then test prompt enhancement and condensation. Repeat task requests with streaming disabled.

All 17 non-English locales include the setting text. Five committed visual baselines came from pinned CI container artifact 14438633679; their webview comparisons passed in run 44544690027. This review round does not change the UI layout.

Pre-Submission Checklist

Visual Snapshots

Committed baselines cover the Azure guidance story and the Extra Body story in dark, light, high-contrast, and high-contrast-light themes.

Videos (interaction / animation only)

Not applicable.

Documentation Updates

  • User documentation: Zoo-Code-Docs #72.
  • SDD model, MC module, and pass/fail configs under docs/architecture/tla/.

Additional Notes

  • Azure v1 guidance: Microsoft Responses API documentation.
  • Reasoning effort support is model-specific. max is not rejected globally; none does not request encrypted reasoning. See OpenAI reasoning guidance.
  • DeepSeek now documents Responses support, but its dedicated provider retains its existing Chat Completions transport; an unrelated stale OpenAI option must not select it. See DeepSeek Responses guidance.
  • The function-tool mapper retains strict: null and parameters: null when unspecified, with explicit regression coverage. See OpenAI function calling.
  • Consolidating the tool-choice mapping with xAI remains a follow-up. The current path preserves function restrictions; it no longer silently drops allowed_tools.

Get in Touch

Discord: taltas1

…ible provider

Reasoning models such as GPT-6.1 Terra on Azure OpenAI reject
reasoning_effort combined with function tools on /chat/completions and
require /v1/responses. The OpenAI Compatible provider only spoke Chat
Completions, so reasoning could not be combined with tool use on those
endpoints.

Add an openAiUseResponsesApi setting. When enabled, createMessage posts
to {baseUrl}/responses with the provider's custom base URL, custom
model/deployment ID, and custom headers. The resolved reasoning_effort
is translated to the Responses reasoning.effort shape, tools use the
flattened Responses function shape, and the shared Responses input and
stream transforms (also used by the xAI provider) handle conversion.
Chat Completions stays the default, so existing setups are unchanged.
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one): 1

  • coderabbit-review-active

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
Pre-merge checks: Passed 3 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Regression Evidence Warning Summary: Most changed behavior has focused tests at the unit layer. One stated negative behavior does not. The PR states that single completions "do not overwrite task reasoning state." No test ch… Add a focused unit test in src/api/providers/__tests__/openai.spec.ts, in the OpenAiHandler Responses API block: 1. Configure openAiUseResponsesApi: true. 2. Run createMessage with a mocked response.completed whose output includ…
Passed Passed checks: 3 passed

Check name Status Explanation
Security Boundaries Passed Summary: No changed path in PR #2498 leaks secrets or PII, trusts unvalidated input in a way that executes it, or bypasses approval or allowlist controls. The PR's own commits (05e80b1..`9ac392d…
Persistence Integrity Passed Result: PASS. I found no changed persistence path in this PR that can lose or corrupt state through an un-awaited operation, a non-atomic write, missing rollback, or an unpropagated persisted defa…
Lifecycle Resource Cleanup Passed Check: Lifecycle resource cleanup. Result: no changed lifecycle path found that leaks a resource or duplicates work. Scope reviewed: the PR's own commits, 05e80b1^..9ac392d (the branch also contains u…
Full details: Security Boundaries

Explanation

Summary: No changed path in PR #2498 leaks secrets or PII, trusts unvalidated input in a way that executes it, or bypasses approval or allowlist controls. The PR's own commits (05e80b1..9ac392d, base 75a93a9) were reviewed directly. Evidence checked: - Function allowlists: buildResponsesRequest in src/api/providers/utils/openai-compatible-responses.ts (lines 120-143) keeps allowed_tools restriction mode and flattens only function references. A missing or non-string name throws. The mapping runs before bridgeExternalAbortSignal registers any listener. The Chat Completions path forwards tool_choice unchanged, so the Responses path does not weaken the restriction. - Extra Body precedence: The Responses path returns { ...extraBody, ...requestBody }. The Chat path uses { ...this.extraBody, ...requestOptions }. Both keep Zoo-managed fields in control. - Endpoint and credentials: responsesReasoningScope (src/api/transform/responses-api-reasoning.ts lines 35-37) stores only a SHA-256 digest of the base URL. No plaintext URL, key, or header is persisted. The PR adds no logging of keys or request bodies in src or packages. - Stored reasoning replay: readResponsesReasoningItem checks item type, id, encrypted content, and summary shape. readReplayableResponsesReasoning requires matching format, provider, and endpoint. Items are replayed only to the same endpoint. - Azure URL handling: Headers and API key reuse the existing client options. The Azure v1 URL path only rewrites the path of the user's configured base URL. It adds no new host. - Error handling: assertResponsesApiSuccess turns failed, incomplete, and cancelled responses into errors. This avoids reporting partial output as success. Minor observation (not a failure): The endpoint digest hashes the full base URL, including any userinfo or query. The digest is one-way. It is not a plaintext leak, so it does not meet the check's failure condition. The check was judged on code evidence from the local checkout. No changed path matches a failure condition.

Full details: Persistence Integrity

Explanation

Result: PASS. I found no changed persistence path in this PR that can lose or corrupt state through an un-awaited operation, a non-atomic write, missing rollback, or an unpropagated persisted default. Scope. The local origin/main is behind, so a plain three-dot diff pulls in 11 unrelated upstream commits. I scoped the review to the PR's seven commits, 75a93a9..9ac392d (05e80b1 through 9ac392d). Those touch the OpenAI Responses path, the Chat Completions format filter, the provider settings schema, and the settings UI. The shared task core (src/core/task/*) is unchanged. Changed persistence paths checked: 1. Encrypted reasoning capture (src/api/providers/openai.ts). this.responsesReasoning is reset at the start of each request. It is assigned only after the stream loop or the non-stream response completes successfully. An early generator close, a failed or incomplete response, or a premature EOF leaves it empty, so partial reasoning is never published. 2. Write ordering. getReasoningDetails() is read inside addToApiConversationHistory, which Task.ts awaits at line 5135 after the stream loop ends. The save uses the existing history hooks, and their write path is unchanged. 3. Replay and ownership (responses-api-reasoning.ts, responses-api-input.ts). Tagged items replay only when format, provider, and the endpoint hash match. Untagged items are omitted on this path, as the description states. This is a deliberate scope choice, not a silent loss on an existing path. 4. Chat Completions filter (openai-format.ts). The filter drops type: "reasoning" entries from reasoning_details. I checked the other providers' writers. OpenRouter stores reasoning.text, reasoning.summary, and reasoning.encrypted, and no other writer stores type: "reasoning" there, so their data is not dropped. 5. Persisted setting openAiUseResponsesApi. It is declared in the same openAiSchema as openAiStrictToolSchemas, and a profile round-trip test covers it. Consumers read it by truthiness. The webview uses ?? false, and the dedicated Kimi Code, Moonshot, MiMo, and DeepSeek handlers set it to false. The default is therefore propagated consistently. Residual notes (not failing criteria): - An explicit empty output in response.completed clears items captured from earlier events. This follows the completed snapshot as the authoritative output and is documented in the PR. - The reasoning text block is skipped when reasoning_details is present. This affects display or summary text, not encrypted state, which is stored in the details. Evidence is from direct code reads and diffs; no build or test execution was used.

Full details: Lifecycle Resource Cleanup

Explanation

Check: Lifecycle resource cleanup. Result: no changed lifecycle path found that leaks a resource or duplicates work. Scope reviewed: the PR's own commits, 05e80b1^..9ac392d (the branch also contains unrelated commits from main). I grepped every added non-test source line for listener, timer, watcher, dispose, abort, and close primitives. Only two lifecycle sites were added: the bridge and its release in src/api/providers/openai.ts. Evidence: 1. Listener bridge. handleResponsesApiMessage (openai.ts) builds the request first. It calls bridgeExternalAbortSignal(metadata?.abortSignal) only after construction, directly before the try. It releases the listener in finally. The comment on the bridge states this ordering is intentional. 2. Bridge semantics. src/api/providers/utils/abort-signal.ts registers one anonymous-retained listener with a named callback. dispose() calls removeEventListener. An already-aborted signal does not register a listener. 3. Early generator exit. Consumer return() runs the finally, which removes the listener. For the streaming path, yield* forwards return into processResponsesApiStream's for-await. The OpenAI SDK 5.23.2 iterator (src/node_modules/openai/core/streaming.js) calls controller.abort() in its finally when not done. The HTTP connection therefore closes on early break. 4. Cancellation. A mid-stream external abort ends the stream without a completed event. The code then throws, and isRequestAborted(error, abortSignal) normalizes it to an AbortError. No retry or re-issue path exists in the PR, so no duplicate work follows. 5. Non-stream and completePrompt paths. These pass the signal directly to the SDK and register no listener. The timeout is passed only when usable. 6. Reasoning state. this.responsesReasoning is reset at the start of each task request and set only after success, so a failed or closed request does not publish partial state. This is state handling, not a resource. 7. Azure client. The constructor assigns one new OpenAI client. The Azure AI Inference guard throws before any client is created, so nothing is left allocated. Limitations: I did not execute the test suites, per the sandbox rules. The verdict relies on source reading and the SDK source in the sandbox.

Full details: Regression Evidence

Explanation

Summary: Most changed behavior has focused tests at the unit layer. One stated negative behavior does not. The PR states that single completions "do not overwrite task reasoning state." No test checks this. Scope checked: The PR-specific commits are 05e80b1 through 9ac392d. I diffed from parent 75a93a9 because origin/main has moved past the PR base. Covered (adequate evidence): - Terminal stream failures (response.failed, incomplete, cancelled, error, response.error, and completed with a failed status) are rejected. Tests are in responses-api-stream.spec.ts (≈L458–475) and responses-api-contract.spec.ts (≈L38–42). - Token-limit classification (OutputTokenLimitError) is tested for both message and completion entry points (openai.spec.ts ≈L2491). - Premature EOF without a completion event is rejected and does not publish partial reasoning (openai.spec.ts ≈L2775). - Abort and timeout forwarding, and listener release on completion, failure, and early exit, are tested (≈L2305–2369). - The Kimi Code, Moonshot, MiMo, and DeepSeek stale-flag isolation is tested (≈L2505). - Azure v1 routing, dated versions, and the AI Inference rejection are tested (≈L2711–2761). - Reasoning round-trip, ownership, endpoint change, and clearing on failure, close, or construction error are tested (≈L2521–2678). - Malformed function references are rejected (openai-compatible-responses.spec.ts ≈L146). - The settings UI has Vitest coverage (OpenAICompatible.spec.tsx, SettingsView.spec.tsx). The visible toggle also has committed Playwright visual baselines for the Azure and Extra Body stories (OpenAICompatible.visual.tsx; five PNGs changed). Gap (FAIL basis): - Changed behavior: completePrompt now has a Responses path (openai.ts ≈L501–545). Prompt enhancement and condensation call it. The PR says these calls "do not overwrite task reasoning state." - The only writes to this.responsesReasoning are in handleResponsesApiMessage (lines 360, 401, 425). completePrompt does not write it. That is correct today. - No test in openai.spec.ts checks the invariant. Reasoning assertions (getReasoningDetails) appear only in the task and clearing paths (≈L2543–2789). The single-completion routing test (≈L2433–2520) does not assert reasoning state. - Plausible regression: a single completion in the middle of a task wipes or replaces captured encrypted reasoning. The next tool-call turn then loses its replay chain. This is a real failure mode, so the omission is not a theoretical branch. Note on the check: The PR's manual mutation claims do not name this invariant. The TLA+ models do not test the Responses completePrompt path against the real handler.

Resolution

Add a focused unit test in src/api/providers/__tests__/openai.spec.ts, in the OpenAiHandler Responses API block: 1. Configure openAiUseResponsesApi: true. 2. Run createMessage with a mocked response.completed whose output includes a reasoning item. Assert getReasoningDetails() returns that item. 3. Call completePrompt("prompt") with a mocked Responses result. The result may include a different reasoning item, or none. 4. Assert getReasoningDetails() still returns the item from step 2, unchanged. 5. Optionally add a case where the completePrompt call fails. Assert the task reasoning is still unchanged. Run the test with the existing Vitest command for openai.spec.ts. Confirm that a mutation making completePrompt assign this.responsesReasoning fails the new test.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@zoomote2

zoomote2 Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor Author

No new code issues found. See session

  • src/api/providers/openai.ts:344 — Responses API ignores the disabled-streaming setting.
  • src/api/providers/openai.ts:405 — Responses API drops configured Extra Body fields.
  • src/api/providers/openai.ts:342 — Responses API does not sanitize lone surrogates in prompts and history.
  • webview-ui/src/i18n/locales/en/settings.json:602-603 — New setting strings are missing from all non-English locales; check-translations fails.
  • webview-ui/src/components/settings/providers/OpenAICompatible.tsx:203-212 — Visual baselines were not regenerated; webview-visual reports five failures.
  • src/api/providers/openai.ts:492 — DeepSeek inherits the Responses single-completion route without clearing a stale opt-in flag.
  • src/api/providers/openai.ts:429-432 — Raw Responses reasoning items leak into Chat Completions history after the setting is disabled.

Reviewed 9ac392d

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Review status

This PR was opened by an automated account. A human maintainer must verify the change intent, provenance, and validation before merging.

Current step: Awaiting fresh human maintainer or CODEOWNER approval.

Review-state labels are managed by this workflow; do not edit them manually. community-approved is managed the same way — do not add or remove it manually. It signals a fresh community code approval for the current head as an advisory priority only; maintainer review is still required.

Comment thread src/api/providers/openai.ts Resolved
Comment thread src/api/providers/openai.ts Resolved
Comment thread src/api/providers/openai.ts Resolved
@codecov

codecov Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.93023% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/api/providers/openai.ts 91.42% 1 Missing and 5 partials ⚠️
src/api/transform/responses-api-stream.ts 90.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Comment thread src/api/providers/openai.ts Resolved
Comment thread src/api/providers/openai.ts Resolved
@edelauna
edelauna marked this pull request as ready for review November 19, 2026 17:01
@github-actions github-actions Bot added the awaiting-maintainer CodeRabbit approved; waiting for a human maintainer label Nov 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Changed-code mutation testing

Report for PR head 9ac392da247d.

Surviving / uncovered mutants Advisories
0 1
Details: mutants, guidance, and reproduction

Changed-code mutation testing

PackageChanged executable linesValidKilledTimeoutSurvivedNo coverageResult
types100000No mutants generated
extension30100000Advisory incomplete
webview1055000Passed

Focused tests

  • types (cwd packages/types): No mutants were generated; selected test file names were not recorded.
  • extension (cwd src): The mutation run was incomplete; selected test file names were not recorded. See Advisory findings for the failure.
  • webview (cwd .): webview-ui/src/components/settings/providers/__tests__/OpenAICompatible.spec.tsx

Reproduce locally

From a full checkout containing both commits:

pnpm install --frozen-lockfile
node scripts/stryker-diff.mjs ci --base a462937b671a5f88d617c28fdd80cf2c28bbd0af --head f2bd2733fdfae35c11409fb9e222dc5668972d7c

Mutation reports

  • types: reports/mutation/types/mutation.html
  • extension: reports/mutation/extension/mutation.html
  • webview: reports/mutation/webview/mutation.html

The workflow uploads generated reports in the changed-code-mutation-report artifact. A direct artifact link appears below after upload.

Advisory findings

  • extension generated 484 mutants in preflight (limit 400). Split the PR or obtain a maintainer-reviewed narrow exclusion.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting-maintainer CodeRabbit approved; waiting for a human maintainer

Projects

None yet

2 participants