Repository navigation
[Fix] Reasoning with tool use fails on OpenAI Compatible endpoints that require the Responses API - #2498
[Fix] Reasoning with tool use fails on OpenAI Compatible endpoints that require the Responses API#2498zoomote2[bot] wants to merge 7 commits into
Conversation
…ible provider
Reasoning models such as GPT-6.1 Terra on Azure OpenAI reject
reasoning_effort combined with function tools on /chat/completions and
require /v1/responses. The OpenAI Compatible provider only spoke Chat
Completions, so reasoning could not be combined with tool use on those
endpoints.
Add an openAiUseResponsesApi setting. When enabled, createMessage posts
to {baseUrl}/responses with the provider's custom base URL, custom
model/deployment ID, and custom headers. The resolved reasoning_effort
is translated to the Responses reasoning.effort shape, tools use the
flattened Responses function shape, and the shared Responses input and
stream transforms (also used by the xAI provider) handle conversion.
Chat Completions stays the default, so existing setups are unchanged.
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one): 1
Please check the settings in the CodeRabbit UI or the
|
| Check name | Status | Explanation |
|---|---|---|
| Security Boundaries | Summary: No changed path in PR #2498 leaks secrets or PII, trusts unvalidated input in a way that executes it, or bypasses approval or allowlist controls. The PR's own commits (05e80b1..`9ac392d… |
|
| Persistence Integrity | Result: PASS. I found no changed persistence path in this PR that can lose or corrupt state through an un-awaited operation, a non-atomic write, missing rollback, or an unpropagated persisted defa… | |
| Lifecycle Resource Cleanup | Check: Lifecycle resource cleanup. Result: no changed lifecycle path found that leaks a resource or duplicates work. Scope reviewed: the PR's own commits, 05e80b1^..9ac392d (the branch also contains u… |
Full details: Security Boundaries
-
Explanation
Summary: No changed path in PR
#2498leaks secrets or PII, trusts unvalidated input in a way that executes it, or bypasses approval or allowlist controls. The PR's own commits (05e80b1..9ac392d, base75a93a9) were reviewed directly. Evidence checked: - Function allowlists:buildResponsesRequestinsrc/api/providers/utils/openai-compatible-responses.ts(lines 120-143) keepsallowed_toolsrestriction mode and flattens only function references. A missing or non-string name throws. The mapping runs beforebridgeExternalAbortSignalregisters any listener. The Chat Completions path forwardstool_choiceunchanged, so the Responses path does not weaken the restriction. - Extra Body precedence: The Responses path returns{ ...extraBody, ...requestBody }. The Chat path uses{ ...this.extraBody, ...requestOptions }. Both keep Zoo-managed fields in control. - Endpoint and credentials:responsesReasoningScope(src/api/transform/responses-api-reasoning.tslines 35-37) stores only a SHA-256 digest of the base URL. No plaintext URL, key, or header is persisted. The PR adds no logging of keys or request bodies insrcorpackages. - Stored reasoning replay:readResponsesReasoningItemchecks item type, id, encrypted content, and summary shape.readReplayableResponsesReasoningrequires matching format, provider, and endpoint. Items are replayed only to the same endpoint. - Azure URL handling: Headers and API key reuse the existing client options. The Azure v1 URL path only rewrites the path of the user's configured base URL. It adds no new host. - Error handling:assertResponsesApiSuccessturns failed, incomplete, and cancelled responses into errors. This avoids reporting partial output as success. Minor observation (not a failure): The endpoint digest hashes the full base URL, including any userinfo or query. The digest is one-way. It is not a plaintext leak, so it does not meet the check's failure condition. The check was judged on code evidence from the local checkout. No changed path matches a failure condition.
Full details: Persistence Integrity
-
Explanation
Result: PASS. I found no changed persistence path in this PR that can lose or corrupt state through an un-awaited operation, a non-atomic write, missing rollback, or an unpropagated persisted default. Scope. The local
origin/mainis behind, so a plain three-dot diff pulls in 11 unrelated upstream commits. I scoped the review to the PR's seven commits,75a93a9..9ac392d(05e80b1 through 9ac392d). Those touch the OpenAI Responses path, the Chat Completions format filter, the provider settings schema, and the settings UI. The shared task core (src/core/task/*) is unchanged. Changed persistence paths checked: 1. Encrypted reasoning capture (src/api/providers/openai.ts).this.responsesReasoningis reset at the start of each request. It is assigned only after the stream loop or the non-stream response completes successfully. An early generator close, a failed or incomplete response, or a premature EOF leaves it empty, so partial reasoning is never published. 2. Write ordering.getReasoningDetails()is read insideaddToApiConversationHistory, which Task.ts awaits at line 5135 after the stream loop ends. The save uses the existing history hooks, and their write path is unchanged. 3. Replay and ownership (responses-api-reasoning.ts,responses-api-input.ts). Tagged items replay only whenformat,provider, and the endpoint hash match. Untagged items are omitted on this path, as the description states. This is a deliberate scope choice, not a silent loss on an existing path. 4. Chat Completions filter (openai-format.ts). The filter dropstype: "reasoning"entries fromreasoning_details. I checked the other providers' writers. OpenRouter storesreasoning.text,reasoning.summary, andreasoning.encrypted, and no other writer storestype: "reasoning"there, so their data is not dropped. 5. Persisted settingopenAiUseResponsesApi. It is declared in the sameopenAiSchemaasopenAiStrictToolSchemas, and a profile round-trip test covers it. Consumers read it by truthiness. The webview uses?? false, and the dedicated Kimi Code, Moonshot, MiMo, and DeepSeek handlers set it tofalse. The default is therefore propagated consistently. Residual notes (not failing criteria): - An explicit emptyoutputinresponse.completedclears items captured from earlier events. This follows the completed snapshot as the authoritative output and is documented in the PR. - Thereasoningtext block is skipped whenreasoning_detailsis present. This affects display or summary text, not encrypted state, which is stored in the details. Evidence is from direct code reads and diffs; no build or test execution was used.
Full details: Lifecycle Resource Cleanup
-
Explanation
Check: Lifecycle resource cleanup. Result: no changed lifecycle path found that leaks a resource or duplicates work. Scope reviewed: the PR's own commits, 05e80b1^..9ac392d (the branch also contains unrelated commits from main). I grepped every added non-test source line for listener, timer, watcher, dispose, abort, and close primitives. Only two lifecycle sites were added: the bridge and its release in src/api/providers/openai.ts. Evidence: 1. Listener bridge. handleResponsesApiMessage (openai.ts) builds the request first. It calls bridgeExternalAbortSignal(metadata?.abortSignal) only after construction, directly before the try. It releases the listener in finally. The comment on the bridge states this ordering is intentional. 2. Bridge semantics. src/api/providers/utils/abort-signal.ts registers one anonymous-retained listener with a named callback. dispose() calls removeEventListener. An already-aborted signal does not register a listener. 3. Early generator exit. Consumer return() runs the finally, which removes the listener. For the streaming path, yield* forwards return into processResponsesApiStream's for-await. The OpenAI SDK 5.23.2 iterator (src/node_modules/openai/core/streaming.js) calls controller.abort() in its finally when not done. The HTTP connection therefore closes on early break. 4. Cancellation. A mid-stream external abort ends the stream without a completed event. The code then throws, and isRequestAborted(error, abortSignal) normalizes it to an AbortError. No retry or re-issue path exists in the PR, so no duplicate work follows. 5. Non-stream and completePrompt paths. These pass the signal directly to the SDK and register no listener. The timeout is passed only when usable. 6. Reasoning state. this.responsesReasoning is reset at the start of each task request and set only after success, so a failed or closed request does not publish partial state. This is state handling, not a resource. 7. Azure client. The constructor assigns one new OpenAI client. The Azure AI Inference guard throws before any client is created, so nothing is left allocated. Limitations: I did not execute the test suites, per the sandbox rules. The verdict relies on source reading and the SDK source in the sandbox.
Full details: Regression Evidence
-
Explanation
Summary: Most changed behavior has focused tests at the unit layer. One stated negative behavior does not. The PR states that single completions "do not overwrite task reasoning state." No test checks this. Scope checked: The PR-specific commits are
05e80b1through9ac392d. I diffed from parent75a93a9becauseorigin/mainhas moved past the PR base. Covered (adequate evidence): - Terminal stream failures (response.failed,incomplete,cancelled,error,response.error, andcompletedwith a failed status) are rejected. Tests are inresponses-api-stream.spec.ts(≈L458–475) andresponses-api-contract.spec.ts(≈L38–42). - Token-limit classification (OutputTokenLimitError) is tested for both message and completion entry points (openai.spec.ts≈L2491). - Premature EOF without a completion event is rejected and does not publish partial reasoning (openai.spec.ts≈L2775). - Abort and timeout forwarding, and listener release on completion, failure, and early exit, are tested (≈L2305–2369). - The Kimi Code, Moonshot, MiMo, and DeepSeek stale-flag isolation is tested (≈L2505). - Azure v1 routing, dated versions, and the AI Inference rejection are tested (≈L2711–2761). - Reasoning round-trip, ownership, endpoint change, and clearing on failure, close, or construction error are tested (≈L2521–2678). - Malformed function references are rejected (openai-compatible-responses.spec.ts≈L146). - The settings UI has Vitest coverage (OpenAICompatible.spec.tsx,SettingsView.spec.tsx). The visible toggle also has committed Playwright visual baselines for the Azure and Extra Body stories (OpenAICompatible.visual.tsx; five PNGs changed). Gap (FAIL basis): - Changed behavior:completePromptnow has a Responses path (openai.ts≈L501–545). Prompt enhancement and condensation call it. The PR says these calls "do not overwrite task reasoning state." - The only writes tothis.responsesReasoningare inhandleResponsesApiMessage(lines 360, 401, 425).completePromptdoes not write it. That is correct today. - No test inopenai.spec.tschecks the invariant. Reasoning assertions (getReasoningDetails) appear only in the task and clearing paths (≈L2543–2789). The single-completion routing test (≈L2433–2520) does not assert reasoning state. - Plausible regression: a single completion in the middle of a task wipes or replaces captured encrypted reasoning. The next tool-call turn then loses its replay chain. This is a real failure mode, so the omission is not a theoretical branch. Note on the check: The PR's manual mutation claims do not name this invariant. The TLA+ models do not test the ResponsescompletePromptpath against the real handler.Resolution
Add a focused unit test in
src/api/providers/__tests__/openai.spec.ts, in theOpenAiHandler Responses APIblock: 1. ConfigureopenAiUseResponsesApi: true. 2. RuncreateMessagewith a mockedresponse.completedwhoseoutputincludes a reasoning item. AssertgetReasoningDetails()returns that item. 3. CallcompletePrompt("prompt")with a mocked Responses result. The result may include a different reasoning item, or none. 4. AssertgetReasoningDetails()still returns the item from step 2, unchanged. 5. Optionally add a case where thecompletePromptcall fails. Assert the task reasoning is still unchanged. Run the test with the existing Vitest command foropenai.spec.ts. Confirm that a mutation makingcompletePromptassignthis.responsesReasoningfails the new test.
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
|
No new code issues found. See session
Reviewed 9ac392d |
Review statusThis PR was opened by an automated account. A human maintainer must verify the change intent, provenance, and validation before merging. Current step: Awaiting fresh human maintainer or CODEOWNER approval. Review-state labels are managed by this workflow; do not edit them manually. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Changed-code mutation testingReport for PR head
Details: mutants, guidance, and reproductionChanged-code mutation testing
Focused tests
Reproduce locallyFrom a full checkout containing both commits: Mutation reports
The workflow uploads generated reports in the Advisory findings
|
Related GitHub Issue
Fixes #2040
Closes #1579
#2040 reports the same Azure/OpenAI-compatible error when reasoning and function tools require
/responses. #1579 requests Responses API support through a per-profile opt-in setting.Description
Some OpenAI-compatible endpoints require
/responsesto combine reasoning with function tools. This adds an opt-in Use Responses API setting for both task messages and single completions such as prompt enhancement and condensation. Chat Completions remains the default. Custom deployment names, headers, reasoning effort, and function tools are retained.A pure
buildResponsesRequest()prepares the request from explicit inputs. It honors streaming, temperature capability, output limits, strict schemas, and Extra Body precedence. Zero and omitted values follow the existing Chat Completions policy. Function allowlists retain their restriction mode and use the flattened Responses shape; malformed function references are rejected before listener registration.Failed, incomplete, cancelled, and error responses now raise an error instead of appearing successful. Premature stream EOF without a completion event is also rejected. Output-token truncation retains
OutputTokenLimitError. Cancellation signals and valid timeout overrides reach the SDK, and external bridge listeners are released on completion, failure, and early generator exit.Completion and its optional output snapshot now use one observer callback. Item events remain the fallback when a gateway omits the snapshot; an explicit empty snapshot clears captured items. The endpoint fingerprint is computed once per history read, and a redundant Azure guard was removed.
Encrypted reasoning items are captured only for successful task responses. The completed output snapshot sets their order. The existing history hooks persist them with a provider tag and a fingerprint of the effective SDK endpoint; URL credentials and query secrets are not stored in that metadata. Responses replay requires matching ownership after serialization or reload. Raw Responses items are excluded from Chat Completions, while other providers' reasoning formats retain their existing handling. Older untagged items are omitted on the OpenAI-compatible path because their source cannot be verified. A new request clears prior reasoning; failure and early close cannot publish partial reasoning. Single completions do not overwrite task reasoning state.
Dedicated Kimi Code, Moonshot, MiMo, and DeepSeek handlers explicitly ignore a stale Responses opt-in from another profile.
For Azure, explicit
/openai/v1URLs are preserved and use the standard OpenAI client, including modernservices.ai.azure.com/openai/v1endpoints. An older Azure OpenAI root defaults to/openai/v1when Responses is enabled and no dated version was selected. Explicit dated versions remain available for legacy Azure endpoints. The older Azure AI Inference route rejects this combination with an instruction to use/openai/v1.The SDD artifacts follow
docs/architecture/tla/AGENTS.md. Three models separate request routing/cleanup, request-option rules, and protocol/reasoning history. Their headers record the review ratchet, input domains, findings, costs, and manual gate results. Models check abstractions; they do not prove the implementation correct.Test Procedure
o1/o3/o4routing precedence, SDK cancellation and timeout options, terminal failures, missing/null/empty completion snapshots, token-limit classification, zero/default/omitted options, strict schemas, filtered tools, function allowlists, and malformed restrictions.a60cdbcec4; the fresh result for this refactor is pending CI.anysuppressions were removed; no suppression count increased.docs/architecture/tla/ResponsesReviewReplays.mdmaps each sabotage configuration to its test; ownership has separate Responses and Chat Completions mutations. These are manual checks, not an automated model-to-code correspondence proof or a CI gate.src/. To repeat TLC, run each.cfgagainst its matching*MC.tlamodule fromdocs/architecture/tla, using-metadiroutside the repository. Fixed configs must pass; sabotage configs must exit with invariant failure 12.All 17 non-English locales include the setting text. Five committed visual baselines came from pinned CI container artifact
14438633679; their webview comparisons passed in run 44544690027. This review round does not change the UI layout.Pre-Submission Checklist
Visual Snapshots
Committed baselines cover the Azure guidance story and the Extra Body story in dark, light, high-contrast, and high-contrast-light themes.
Videos (interaction / animation only)
Not applicable.
Documentation Updates
docs/architecture/tla/.Additional Notes
maxis not rejected globally;nonedoes not request encrypted reasoning. See OpenAI reasoning guidance.strict: nullandparameters: nullwhen unspecified, with explicit regression coverage. See OpenAI function calling.allowed_tools.Get in Touch
Discord: taltas1