feat(eap): Implement gen_ai attribute transformations by constantinius · Pull Request #6201 · getsentry/relay

constantinius · 2026-07-09T14:41:52Z

Summary

Implements the two gen_ai attribute transformations introduced in sentry-conventions PR #465:

gen_ai_request_messages_to_input_messages — transforms gen_ai.request.messages (with content field per message) into gen_ai.input.messages (with parts array)
gen_ai_response_to_output_messages — combines gen_ai.response.text and gen_ai.response.tool_calls into a single gen_ai.output.messages assistant message with typed parts

These go beyond simple key renaming — they reshape attribute values to match the new schema.

How it works

Attributes with _status: "transform" in sentry-conventions produce WriteBehavior::CurrentName, so normalize_attribute_names leaves them alone. The dedicated transformation code in the new gen_ai_transform module handles the full move-and-reshape, called from normalize_ai as part of AI span normalization (processing mode only).

The gen_ai.request.messages, gen_ai.response.text, and gen_ai.response.tool_calls legacy aliases are removed from SpanV1 SpanData so these deprecated keys survive into SpanV2 attributes where the transformation can process them.

`gen_ai.request.messages` → `gen_ai.input.messages`

For each message object, the content field is converted to a parts array:

String content "hello" → [{"type": "text", "content": "hello"}]
Array content (already parts-like) → kept, with text copied to content on each part if missing
Other content types (e.g., tool message objects) → left unchanged
Non-JSON values → moved as-is (key rename only)

`gen_ai.response.text` + `gen_ai.response.tool_calls` → `gen_ai.output.messages`

gen_ai.response.text handles multiple formats:

Plain string, JSON string, JSON array of strings, JSON object with content, JSON array of objects with content

Tool calls get "type": "tool_call" added. Both are combined into one assistant message: [{"role": "assistant", "parts": [...]}].

Changes

relay-conventions/build/attributes.rs — add Transform variant to DeprecationStatus, producing WriteBehavior::CurrentName
relay-event-normalization/src/eap/gen_ai_transform.rs — new module with transformation logic
relay-event-normalization/src/eap/ai.rs — normalize_ai calls transform_gen_ai as its first step
relay-event-schema/src/protocol/span.rs — remove legacy aliases for the three deprecated attributes
relay-conventions/sentry-conventions — update submodule to PR fix(server): Allow multipart requests without trailing newline #465 branch

Tests

21 Rust unit tests covering all input formats, edge cases, and idempotency
4 Python integration tests covering both SpanV1 (transaction) and SpanV2 (direct ingestion) paths
Updated existing test_ai_spans_example_transaction expectations

Contributes to TET-2587

Implement the two attribute transformations introduced in sentry-conventions PR #465: `gen_ai_request_messages_to_input_messages` and `gen_ai_response_to_output_messages`. These go beyond simple key renaming — they reshape attribute values to match the new schema. `gen_ai.request.messages` with a `content` field on each message is converted to `gen_ai.input.messages` with a `parts` array. `gen_ai.response.text` (plain string, JSON string array, or objects with content) and `gen_ai.response.tool_calls` are combined into a single `gen_ai.output.messages` assistant message with typed parts. The transformations run inside `normalize_ai` as part of AI span normalization (processing mode only). Attributes with the new `"transform"` deprecation status produce `WriteBehavior::CurrentName` so `normalize_attribute_names` leaves them alone — the dedicated transformation code handles the full move-and-reshape. The `gen_ai.request.messages`, `gen_ai.response.text`, and `gen_ai.response.tool_calls` legacy aliases are removed from SpanV1 SpanData so these deprecated keys survive into SpanV2 attributes where the transformation can process them. Key changes: - Add `Transform` variant to `DeprecationStatus` in the build script, producing `WriteBehavior::CurrentName` instead of `NewName` - New `gen_ai_transform` module with the two transformation functions - `normalize_ai` calls `transform_gen_ai` as its first step - Remove legacy aliases for the three deprecated attributes from SpanData - Update sentry-conventions submodule to PR #465 branch - 21 unit tests, 4 new integration tests covering SpanV1 and SpanV2

linear-code · 2026-07-09T14:41:56Z

TET-2587

Clippy lint fix for the Transform arm in format_write_behavior.

The legacy standalone spans and transaction extraction paths convert SpanV1 to SpanV2 but skip normalize_ai. With the legacy aliases removed, deprecated keys like gen_ai.response.text survive untransformed. Call transform_gen_ai directly after span_v1_to_span_v2 on both paths so the deprecated attributes get reshaped regardless of pipeline. Also fix clippy collapsible_if lints in gen_ai_transform.

When gen_ai.response.tool_calls is not valid JSON (e.g., "some_tool_calls"), the transformation cannot parse it, so the deprecated key stays as-is. Update the test expectation to match this behavior.

Dav1dde · 2026-07-10T10:04:52Z

+    let mut span_v2 = span.map_value(|span| relay_spans::span_v1_to_span_v2(span, true));
+    if let Some(span_v2) = span_v2.value_mut()
+        && let Some(attributes) = span_v2.attributes.value_mut()
+    {
+        eap::transform_gen_ai(attributes);
+    }


Gonna have to talk to the team about this, I understand the desire of adding it in here. As it stands right now, the right approach is to implement the normalization independent of the span format and then add it to the proper normalization paths.

Solutions which come to mind:

Add an abstraction (maybe something like Getter just attribute based) which can be backed by a V1 and V2 span

Introduce a proper "post transaction" normalization, and figure out what the implications of this are

@constantinius will get back to you, this shouldn't be something you have to figure out and do, this is more a general platform problem.

- Always remove deprecated keys even when canonical key already exists or when no parts could be extracted from the input values - Remove transform_gen_ai calls from legacy_spans/store.rs and transactions/spans.rs — transformation only runs in the EAP pipeline via normalize_ai; other paths are a platform concern - Use module import instead of super:: reference in ai.rs - Narrow visibility of transform_gen_ai to pub(crate)

cursor · 2026-07-10T10:40:24Z

+    attributes.remove(GEN_AI__RESPONSE__TOOL_CALLS);
+
+    if parts.is_empty() {
+        return;


Response attrs dropped when empty

High Severity

When gen_ai.response.text and/or gen_ai.response.tool_calls are present but parsing yields no output parts (e.g. non-JSON tool calls like some_tool_calls, or JSON text with no usable content), the code still removes both deprecated keys and writes nothing to gen_ai.output.messages, so response data is lost instead of being moved or preserved like invalid gen_ai.request.messages.

^{Reviewed by Cursor Bugbot for commit aa94a95. Configure here.}

…ansformations

sentry · 2026-07-13T12:06:50Z

    /// The output messages from the model call.
    #[metastructure(
        field = "gen_ai.output.messages",
-        legacy_alias = "gen_ai.response.tool_calls",
        legacy_alias = "ai.response.toolCalls",
        legacy_alias = "ai.tool_calls",
-        legacy_alias = "gen_ai.response.text",
        legacy_alias = "ai.response.text",
        legacy_alias = "ai.responses"
    )]


Bug: Removing legacy gen_ai aliases from SpanData causes silent data loss in non-EAP ingestion paths (legacy spans, transactions) that don't use the new normalize_ai transformation.
_{Severity: HIGH}

Suggested Fix

Reinstate the legacy aliases in SpanData to ensure backward compatibility for ingestion paths that do not use the EAP normalization pipeline. Alternatively, modify the legacy span and transaction extraction paths to also call the normalize_ai function to handle the attribute transformation.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: relay-event-schema/src/protocol/span.rs#L569-L576 Potential issue: The removal of legacy aliases for `gen_ai.request.messages`, `gen_ai.response.text`, and `gen_ai.response.tool_calls` from the `SpanData` protocol definition will cause data loss. Span ingestion paths for legacy standalone spans and transaction extraction rely on these aliases to convert SpanV1 data to SpanV2 attributes. These paths do not use the new `normalize_ai` transformation logic. Without the aliases, when `span_v1_to_span_v2` is called, the deprecated attributes are not mapped to their canonical names and are silently dropped.

Also affects:

relay-server/src/processing/legacy_spans/store.rs:28

relay-server/src/processing/transactions/spans.rs:118

sentry · 2026-07-13T12:06:50Z

+    if let Some(ref raw) = tool_calls_raw {
+        tool_calls_to_parts(raw, &mut parts);
+    }
+
+    // Always clean up deprecated keys, even if no parts could be extracted.
+    attributes.remove(GEN_AI__RESPONSE__TEXT);
+    attributes.remove(GEN_AI__RESPONSE__TOOL_CALLS);
+


Bug: Non-JSON data in gen_ai.response.tool_calls is silently dropped because the attribute is removed after a failed parse, and no canonical key is written.
_{Severity: MEDIUM}

Suggested Fix

Modify the handling of gen_ai.response.tool_calls to match gen_ai.request.messages. If serde_json::from_str fails, preserve the data by renaming the attribute to a canonical key instead of removing it. This prevents silent data loss for non-JSON values.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: relay-event-normalization/src/eap/gen_ai_transform.rs#L131-L138 Potential issue: In the `transform_gen_ai` function, if the `gen_ai.response.tool_calls` attribute contains a string that is not valid JSON, the `tool_calls_to_parts` function will fail to parse it and return early. The logic then unconditionally removes the `gen_ai.response.tool_calls` attribute. Because parsing failed, no new canonical attribute (`gen_ai.output.messages`) is written, resulting in silent data loss. This behavior is inconsistent with the handling for `gen_ai.request.messages`, which preserves non-JSON data by renaming the key.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f1a76fe. Configure here.}

cursor · 2026-07-13T12:08:40Z

        return;
    }

+    gen_ai_transform::transform_gen_ai(attributes);


Transaction spans skip gen_ai transform

High Severity

transform_gen_ai runs only from span V2 processing (normalize_ai in the spans pipeline). Transaction-extracted spans are converted with span_v1_to_span_v2 and stored without that step. Removing SpanData legacy aliases for gen_ai.request.messages, gen_ai.response.text, and gen_ai.response.tool_calls stops mapping those keys to canonical fields, so transaction AI spans can keep deprecated names and miss reshaped gen_ai.input.messages / gen_ai.output.messages.

Additional Locations (1)

relay-event-schema/src/protocol/span.rs#L545-L576

^{Reviewed by Cursor Bugbot for commit f1a76fe. Configure here.}

Dav1dde · 2026-07-13T15:22:12Z

@constantinius talked with the team, we'll look into dealing with the transaction normalization case in #6216 - Since you already removed the code for it, is this normalization required to run on transaction spans?

So to get you unblocked I would propose we merge this change for the span streaming pipeline and we as a team take over the transaction change, if it is necessary.

Also please remove the redundant information from the PR description, changed files and tests I can tell by the diff, even the functional changes I can see in code, the important information in the description and later commit is the motivation and the effect not the functional changes, which are already present in code.

constantinius added 3 commits July 9, 2026 16:52

fix(conventions): remove unneeded return statement

6e7299f

Clippy lint fix for the Transform arm in format_write_behavior.

fix(test): update test expectation for non-JSON tool_calls

62ec370

When gen_ai.response.tool_calls is not valid JSON (e.g., "some_tool_calls"), the transformation cannot parse it, so the deprecated key stays as-is. Update the test expectation to match this behavior.

constantinius requested a review from a team July 10, 2026 09:51

constantinius marked this pull request as ready for review July 10, 2026 09:51

constantinius requested a review from a team as a code owner July 10, 2026 09:51

cursor Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

sentry Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

constantinius changed the title ~~feat(eap): implement gen_ai attribute transformations~~ feat(eap): Implement gen_ai attribute transformations Jul 10, 2026

docs: add changelog entry for gen_ai attribute transformations

16f88a2

Dav1dde requested changes Jul 10, 2026

View reviewed changes

cursor Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/ai.rs Outdated

sentry Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

cursor Bot reviewed Jul 10, 2026

View reviewed changes

sentry Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/ai.rs

Merge branch 'master' into constantinius/feat/eap/gen-ai-attribute-tr…

f1a76fe

…ansformations

sentry Bot reviewed Jul 13, 2026

View reviewed changes

cursor Bot reviewed Jul 13, 2026

View reviewed changes

constantinius requested a review from Dav1dde July 13, 2026 12:10

Dav1dde mentioned this pull request Jul 13, 2026

Basic abstraction over Spans to support shared normalizations #6216

Open

Uh oh!

Conversation

constantinius commented Jul 9, 2026

Summary

How it works

gen_ai.request.messages → gen_ai.input.messages

gen_ai.response.text + gen_ai.response.tool_calls → gen_ai.output.messages

Changes

Tests

Uh oh!

linear-code Bot commented Jul 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Dav1dde Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dav1dde Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot Jul 10, 2026

Choose a reason for hiding this comment

Response attrs dropped when empty

Uh oh!

Uh oh!

sentry Bot Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

sentry Bot Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jul 13, 2026

Choose a reason for hiding this comment

Transaction spans skip gen_ai transform

Uh oh!

Dav1dde commented Jul 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

`gen_ai.request.messages` → `gen_ai.input.messages`

`gen_ai.response.text` + `gen_ai.response.tool_calls` → `gen_ai.output.messages`

Dav1dde Jul 10, 2026 •

edited

Loading