fix(server-utils): Stop reporting caller-handled errors from the OpenAI, Anthropic & Google GenAI instrumentations - #23024
Merged
nicohrubec merged 7 commits intoAug 26, 2026
Conversation
lux-in-tenebris-lucet
marked this pull request as ready for review
August 4, 2026 14:35
lux-in-tenebris-lucet
requested review from
isaacs and
mydea
and removed request for
a team
August 4, 2026 14:35
Contributor
|
👋 @isaacs, @mydea, @nicohrubec, @andreiborza, @getsentry/team-javascript-sdks — Please review this PR when you get a chance! |
lux-in-tenebris-lucet
force-pushed
the
fix/ai-instrumentation-error-capture
branch
from
August 8, 2026 13:01
c7facfc to
4c52da8
Compare
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
bricefriha
reviewed
Aug 9, 2026
Contributor
|
👋 @isaacs, @mydea, @nicohrubec, @andreiborza, @getsentry/team-javascript-sdks — Please review this PR when you get a chance! |
… instrumentation level The exported instrumentOpenAiClient, instrumentAnthropicAiClient and instrumentGoogleGenAIClient wrappers captured provider errors with mechanism.handled = false and then rethrew, so the SDK classified the error as an unhandled crash before the application's retry or fallback logic ran. A call that succeeded on retry still produced an unhandled event, and each retry produced another one. Applies the convention established for the channel-based OpenAI integration in getsentry#21877 to the manual client instrumentation, which is the only available path on the edge and serverless runtimes. Error span status and the original error identity are unchanged. The anthropic suite asserted this divergence directly, expecting the model-error event only while orchestrion was disabled. Both paths agree now, so that branch and its expectation are gone. Captures are kept where a provider reports an error as data on an otherwise successful call, since the caller never sees those as a thrown error. The AI integration suites no longer mask these events with .ignore('event'), so they fail if the capture returns. Dropping the capture also left the .catch() in createWithResponseWrapper rethrowing into a promise nothing observes, which is not a handler at all: awaiting the two promises in sequence orphans the second whenever the first rejects. They are awaited together now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Removing `captureException` put the import at 111 chars, so oxfmt collapsed it to a single line. Pin the previous layout with `oxfmt-ignore` per review feedback. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lux-in-tenebris-lucet
force-pushed
the
fix/ai-instrumentation-error-capture
branch
from
August 13, 2026 13:11
31e179c to
28d6e59
Compare
Contributor
|
👋 @isaacs, @mydea, @nicohrubec, @andreiborza, @getsentry/team-javascript-sdks — Please review this PR when you get a chance! |
2 similar comments
Contributor
|
👋 @isaacs, @mydea, @nicohrubec, @andreiborza, @getsentry/team-javascript-sdks — Please review this PR when you get a chance! |
Contributor
|
👋 @isaacs, @mydea, @nicohrubec, @andreiborza, @getsentry/team-javascript-sdks — Please review this PR when you get a chance! |
# Conflicts: # dev-packages/node-integration-tests/suites/tracing/anthropic/test.ts
nicohrubec
force-pushed
the
fix/ai-instrumentation-error-capture
branch
from
August 25, 2026 13:31
0af60ab to
d08b788
Compare
Anthropic stream errors — connection drops and `error`-type stream events — are
surfaced to the caller (the async iterator rejects, or their `error` listener
fires), so recording them as `handled: false` at the instrumentation level makes
the same premature "unhandled crash" decision that was fixed for the client
wrappers. Mark the span failed and leave the error for the caller.
Also assert the genuine error-as-data captures instead of masking them with
`.ignore('event')`: Google GenAI blocked content and Anthropic error-shaped
responses are returned as data on a resolved value and never thrown, so the
instrumentation intentionally captures them. Those cases now have explicit event
assertions, and the synthetic captureException-mock unit test is dropped in
favour of this integration coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nicohrubec
changed the title
nicohrubec
approved these changes
Aug 25, 2026
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 41f07bc. Configure here.
This was referenced Aug 25, 2026
andreiborza
approved these changes
Aug 26, 2026
andreiborza
left a comment
andreiborza
left a comment
Member
There was a problem hiding this comment.
LGTM, thanks for raising and fixing this!
nicohrubec
added a commit
that referenced
this pull request
Aug 26, 2026
LangChain's callback error handlers (handleLLMError, handleChainError, handleToolError) and the embeddings wrapper called captureException with handled: false. These errors reject invoke()/embedQuery() to the caller, so the instrumentation recorded an "unhandled crash" for an error the application handles, the same issue fixed for the OpenAI, Anthropic and Google GenAI client wrappers in #23024. The span is still marked failed and the error still propagates; only the event goes away. The integration tests masked these with .ignore('event'); those masks are removed so the tests no longer hide instrumentation-level captures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nicohrubec
added a commit
that referenced
this pull request
Aug 26, 2026
LangGraph's compiled-graph invoke and tool-node wrappers called captureException with handled: false and then rethrew. The error rejects invoke() to the caller, so the instrumentation recorded an "unhandled crash" for an error the application handles, the same issue fixed for the client wrappers (#23024) and LangChain. The span is still marked failed and the error still propagates; only the event goes away. The integration tests masked these with .ignore('event'); those masks are removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…g them
Attaching the MessageStream `error` listener stops the error being raised as an
unhandled rejection, so not capturing it swallows the error for callers that
don't await or iterate the stream. Restore the capture in the stream.on('error')
handler.
The type: 'error' stream-event path (isErrorEvent) stays uncaptured: the SDK
throws that to the caller and never delivers it as a stream event, so it is a
caller-handled error rather than error-as-data.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nicohrubec
added a commit
that referenced
this pull request
Aug 26, 2026
LangGraph's compiled-graph invoke and tool-node wrappers called captureException with handled: false and then rethrew. The error rejects invoke() to the caller, so the instrumentation recorded an "unhandled crash" for an error the application handles, the same issue fixed for the client wrappers (#23024) and LangChain. The span is still marked failed and the error still propagates; only the event goes away. The integration tests masked these with .ignore('event'); those masks are removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nicohrubec
added a commit
that referenced
this pull request
Aug 26, 2026
) LangChain's callback error handlers (`handleLLMError`, `handleChainError`, `handleToolError`) and the embeddings wrapper called `captureException(error, { handled: false })`. These errors reject `invoke()`/`embedQuery()` to the caller, so the instrumentation recorded an "unhandled crash" for an error the application handles. This is the same problem fixed for the OpenAI, Anthropic, and Google GenAI client wrappers in #23024. Related to #23023 --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nicohrubec
added a commit
that referenced
this pull request
Aug 26, 2026
) LangGraph's compiled-graph invoke wrapper and tool-node wrapper called `captureException(error, { handled: false })` and then rethrew. The error rejects `invoke()` to the caller, so the instrumentation recorded an "unhandled crash" for an error the application handles. This is the same problem fixed for the OpenAI, Anthropic, and Google GenAI client wrappers in #23024, and for LangChain in #23593. Related to #23023 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Aug 26, 2026
nicohrubec
added a commit
that referenced
this pull request
Aug 26, 2026
…Anthropic & Google GenAI instrumentations (#23638) Backport of: #23024 ## Differences to the original PR - On v10 the AI instrumentation lives in `packages/core/src/tracing/` (it moved to `@sentry/server-utils` on v11), so the same change is applied there with v10's internal imports and `GEN_AI_*_ATTRIBUTE` names. - Anthropic test only: v10 also captured the handled `error-model` rejection on the non-orchestrion path, so its tests asserted that event. The fix stops that capture, so those `EXPECTED_MODEL_ERROR` expectations are removed. v10's existing truncation test is kept next to the new error-shaped-response test. Co-authored-by: Abhilaksh <abhilaksh@nox.today>
nicohrubec
added a commit
that referenced
this pull request
Aug 26, 2026
… fixes (#23641) Adds an Important Changes changelog entry explaining that the AI integrations no longer capture provider errors thrown back to the caller, and credits @lux-in-tenebris-lucet for the original contribution (#23024).
zkasuran
added a commit
to zkasuran/sentry-javascript
that referenced
this pull request
Aug 28, 2026
Two conflicts, both in the google-genai instrumentation that develop moved under us. packages/server-utils/src/ai/google-genai/index.ts: develop added low-cardinality span names when span streaming is on (getsentry#23573) and dropped the captureException calls from the instrumentation (getsentry#23024), touching the same lines where this branch threads the chat create-time params through. Kept both: the merged apply() computes attributeParams via mergeChatCreateParams, feeds those to extractRequestAttributes plus addPrivateRequestAttributes, then derives model with develop's || 'unknown' and its spanName sentinel logic. packages/server-utils/test/ai/lib/tracing/google-genai.test.ts: add/add, develop landed a span-names suite at the same path this branch used for the config suite. Took develop's file whole and re-added the config suite beside it, hoisting the shared setupClient to module scope and switching the config assertions from spanToJSON to spanToStaticSpanJSON, since spanToJSON now returns the streamed shape (attributes, no data). Also updated the Cloudflare integration test. Its chat span assertion matches attributes exhaustively with toEqual, while that scenario passes a config to chats.create, so propagating it adds temperature, top_p and max_tokens to the chat span. Verified the assertion fails without those three keys and passes with them. The node suites assert per key, so they were unaffected. Verified: @sentry/server-utils 432 tests passing (44 files), node google-genai and google-genai-v2 integration 24 passing, cloudflare google-genai integration 1 passing, oxfmt --check, oxlint --type-aware, oxlint src --type-aware --type-check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The exported AI client instrumentations (
instrumentOpenAiClient,instrumentAnthropicAiClient,instrumentGoogleGenAIClient) calledcaptureException(error, { handled: false })and rethrew. The wrapper runs inside the provider call, before any caller retry or fallback, so it reports an "unhandled crash" for an error the app may handle. A 429 the app retried and recovered from got reported as a crash, once per attempt.This drops the capture everywhere the error reaches the caller, keeping it only where the provider returns the error as data on a resolved value (Anthropic error-shaped 200 responses, Google GenAI blocked content). The span is still marked
internal_errorand the error still propagates, so only the event goes away, leaving thehandleddecision to the application boundary.This also covers the Anthropic streaming paths (
stream.on('error')anderror-type events), which surfaced to the caller but were still captured. LangChain and LangGraph have the same problem in their callback handlers, which will be handled in a follow-up.The suites previously masked these events with
.ignore('event'); those are replaced with explicit expectations (no event on caller-handled paths, event still captured on error-as-data paths), and the syntheticcaptureException-mock unit test is removed in favour of that coverage.Fixes #23023