Skip to content

fix(telemetry): export thought parts as reasoning, not text - #8096

Open
sunhouy wants to merge 1 commit into
google:mainfrom
sunhouy:telemetry/thinking-parts
Open

sunhouy wants to merge 1 commit into
google:mainfrom
sunhouy:telemetry/thinking-parts

Conversation

@sunhouy

@sunhouy sunhouy commented Nov 11, 2026

Copy link
Copy Markdown

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

Problem:
Part(text=..., thought=True) is exported to gen_ai.output.messages as a text part, so the trace shows the chain of thought as part of the answer. The thought flag survives on the event but not on the wire, and everything downstream only sees the part type.

Solution:
_to_part checks part.thought and returns a Reasoning part (type: "reasoning") for it. That is the ReasoningPart of the v1.39.0 output-message schema the module already references — same content field, different type. Input messages share _to_part, so a thought replayed in the history is typed correctly too. Parts without the flag, and every other part kind, are untouched.

I picked the part-type mapping over dropping thought parts entirely because the capture rules (ContentCapturingMode) already decide whether reasoning is captured at all; once it is captured, keeping it in its own part is the only way a backend can collapse or hide it.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Added to tests/unittests/telemetry/test_experimental_semconv.py:

  • test_response_attributes_report_thought_parts_as_reasoning — a thought part before and after the answer, asserting the whole gen_ai.output.messages value.
  • test_input_attributes_report_thought_parts_as_reasoning — the same for a replayed thought in gen_ai.input.messages.

Both fail on main (2 failed, 25 passed) and pass with the change (2 passed, 25 passed), so they do pin the behaviour rather than just documenting it.

$ pytest tests/unittests/telemetry -q
2 failed, 897 passed, 3 xfailed

The two failures are test_google_cloud.py::test_get_gcp_otlp_metric_exporter_sends_agent_engine_user_agent and test_spans.py::test_detect_error_discovery_engine_search_tool; both fail identically on main without this change, they want a live GCP / Discovery Engine setup.

Manual End-to-End (E2E) Tests:

Scripted BaseLlm yielding [Part(text='Synthetic thought', thought=True), Part(text='Synthetic answer')], wrapped in ContentCapturingMode.SPAN_ONLY, run through InMemoryRunner and read back off the finished span — no provider call, no credentials:

$ python repro.py
Model event thought flags: [True, False]
Exported part types: ['text', 'text']      # before
Exported part types: ['reasoning', 'text'] # after

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

The ReasoningPart shape (type: "reasoning", content: string, both required) is taken from gen-ai-output-messages.json v1.39.0, which _experimental_semconv already cites in its module docstring.

This does change the wire format for anyone currently reading thinking content out of a text part. I think that is the point of the issue, but happy to gate it if you would rather keep the old shape behind a flag.

`_to_part` in the experimental semconv builder only looked at `part.text`,
so `Part(text=..., thought=True)` ended up as a `TextPart` on
`gen_ai.output.messages`. The `thought` flag is all a consumer has to tell
the model's reasoning from its answer, and once the part is on the wire as
`text` that information is gone -- a trace backend renders the chain of
thought as part of the reply.

The v1.39.0 semconv the module already points at has a separate
`ReasoningPart` (`type: "reasoning"`, same `content` field), so map thought
parts onto it. Input messages go through the same `_to_part`, which keeps
a replayed thought in the history typed correctly as well.

Verified with a scripted `BaseLlm` (no provider call): before the change
the exported part types were `['text', 'text']`, after they are
`['reasoning', 'text']`. `tests/unittests/telemetry` passes apart from two
cases that need a live GCP/Discovery Engine setup and fail the same way
without this change.
@google-cla

google-cla Bot commented Nov 11, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@Binary-Refinery

Copy link
Copy Markdown

I can run the candidate patch with a local span exporter and validate the serialized mixed reasoning/text payload against the referenced schema. Would that add useful coverage beyond the attribute assertions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GenAI telemetry exports thought parts as text instead of reasoning

3 participants