fix(gateway): surface in-stream errors from Zoo and Vercel AI gateways

[JamesRobert20]

Related GitHub Issue

Closes: #587

Description

When a gateway response starts streaming, the HTTP status is already committed as 200. Errors raised by the upstream model provider mid-stream are therefore delivered as an in-band error chunk on the SSE stream rather than as a thrown HTTP error.

Both the Zoo Gateway and the Vercel AI Gateway handlers iterated chunks and only read chunk.choices[0]?.delta, silently ignoring any chunk shaped like { error: ... }. The stream then ended with no content, so the extension surfaced a generic "the model did not provide a response" message even though the real provider error was visible in the gateway logs. This affects any mid-stream provider error (server errors, content filtering, upstream timeouts, rate limits, etc.) — a 429 Too Many Requests is just the example that surfaced it. openrouter.ts already guards against this with an if ("error" in chunk) check — these two handlers did not.

How this is fixed:

• zoo-gateway.ts — detect the in-stream error chunk and rebuild it via a new toGatewayStreamError helper into an Error carrying status/code. Because it is thrown from inside the existing try block, the current surfaceGatewayApiError / classifyGatewayApiError path handles it unchanged (sign-in on 401, add-credits on 402, budget prompt on budget 429, contact-support on 403), and the upstream message reaches the user for everything else.
• vercel-ai-gateway.ts — detect the in-stream error chunk and throw the upstream message, or a default fallback when the chunk carries no message (this handler has no Zoo-specific auth/credit UX).

Note: plain rate-limit 429s deliberately do not raise a toast (classifyGatewayApiError returns none); the real message now surfaces inline in the chat, which matches the existing tested behavior for non-budget 429s.

Test Procedure

Unit tests, scoped:

pnpm exec vitest run api/providers/__tests__/zoo-gateway.spec.ts api/providers/__tests__/vercel-ai-gateway.spec.ts

Added coverage:

• Zoo Gateway throws the upstream reason when an in-stream error chunk is received.
• Zoo Gateway still surfaces the add-credits prompt when the in-stream error carries a budget code.
• toGatewayStreamError maps message/status/code and falls back to a default message.
• Vercel AI Gateway throws the upstream reason from an in-stream error chunk, and a default message when the chunk has no message.

All specs pass; check-types and lint are green across the workspace.

Pre-Submission Checklist

• Issue Linked: This PR is linked to an approved GitHub Issue.
• Scope: Changes are focused on the gateway in-stream error bug.
• Self-Review: Performed a thorough self-review.
• Testing: New unit tests added for both handlers.
• Documentation Impact: No user-facing docs needed.
• Contribution Guidelines: Read and followed.

Documentation Updates

• No documentation updates are required.

Additional Notes

This PR pairs with a server-side change in the website repo that emits the in-band error part on the SSE stream (the AI SDK reports upstream failures as an error part on fullStream instead of throwing). This extension change makes the client honor those chunks.

[coderabbitai]

📝 Walkthrough

Walkthrough

Zoo and Vercel AI gateway handlers now detect in-band error chunks mid-stream and surface the upstream error message. Zoo adds toGatewayStreamError to convert raw stream error payloads into Errors with optional status/code; Vercel detects chunk.error and throws with a fallback message.

Changes

Stream Error Propagation for AI Gateway Providers

Layer / File(s)	Summary
Zoo Gateway error reconstruction utility and streaming detection src/api/providers/zoo-gateway.ts, src/api/providers/__tests__/zoo-gateway.spec.ts	Introduces exported toGatewayStreamError(raw) that reconstructs Error instances from in-band error payloads (preserving message, status, code or using defaults). The Zoo streaming loop now detects chunk.error and throws the reconstructed error to reuse existing error-classification/UI flows. Tests validate utility behavior, upstream message propagation, and budget-exceeded UX routing.
Vercel AI Gateway in-stream error detection src/api/providers/vercel-ai-gateway.ts, src/api/providers/__tests__/vercel-ai-gateway.spec.ts	Vercel streaming loop treats chunk.error as fatal, reads chunk.error.message or falls back to "Vercel AI Gateway stream error", and throws to stop streaming. Tests assert stream iteration rejects with the in-band message or the fallback.
Release notes .changeset/gateway-stream-error-propagation.md	Adds a changeset documenting patch release and the streaming error propagation behavior for Zoo and Vercel gateways, noting existing Zoo UX prompts remain in place.

Sequence Diagram

sequenceDiagram
  participant Client
  participant GatewayHandler
  participant StreamLoop
  participant toGatewayStreamError
  participant UIErrorHandler

  Client->>GatewayHandler: createMessage()
  GatewayHandler->>StreamLoop: iterate stream chunks
  StreamLoop->>StreamLoop: receive chunk
  alt chunk.error present
    StreamLoop->>toGatewayStreamError: convert raw error
    toGatewayStreamError->>StreamLoop: Error(message,status,code)
    StreamLoop->>UIErrorHandler: throw Error
    UIErrorHandler->>Client: surface upstream message / mapped UX
  else normal chunk
    StreamLoop->>GatewayHandler: emit content chunk
    GatewayHandler->>Client: stream content
  end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• Zoo-Code-Org/Zoo-Code#344: Modifies the Zoo Gateway streaming createMessage logic and is related to prior wiring of the same stream loop.

Suggested reviewers

• navedmerchant
• hannesrudolph
• edelauna
• taltas

Poem

A rabbit watched the streaming flow,
Where hidden errors used to go,
Now chunks can speak, no message lost—
Upstream words return across the host. 🐇✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: detecting and surfacing in-stream errors from two gateway handlers.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	PR description comprehensively covers all required template sections: issue linked (#587), detailed implementation approach with trade-offs, specific test procedure with command, and pre-submission checklist fully completed.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/gateway-stream-error-propagation

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[navedmerchant]

Please link an issue number and remove changeset, otherwise looks good to merge

[navedmerchant]

We do not need a changeset file

[JamesRobert20]

Both points addressed:

• Removed the changeset file.
• Opened and linked [BUG] Provider errors raised mid-stream are not propagated to the extension #587 (filed against the bug template) — Closes: [BUG] Provider errors raised mid-stream are not propagated to the extension #587 is now in the description.