Repository navigation
Conversation
`_to_part` in the experimental semconv builder only looked at `part.text`, so `Part(text=..., thought=True)` ended up as a `TextPart` on `gen_ai.output.messages`. The `thought` flag is all a consumer has to tell the model's reasoning from its answer, and once the part is on the wire as `text` that information is gone -- a trace backend renders the chain of thought as part of the reply. The v1.39.0 semconv the module already points at has a separate `ReasoningPart` (`type: "reasoning"`, same `content` field), so map thought parts onto it. Input messages go through the same `_to_part`, which keeps a replayed thought in the history typed correctly as well. Verified with a scripted `BaseLlm` (no provider call): before the change the exported part types were `['text', 'text']`, after they are `['reasoning', 'text']`. `tests/unittests/telemetry` passes apart from two cases that need a live GCP/Discovery Engine setup and fail the same way without this change.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
I can run the candidate patch with a local span exporter and validate the serialized mixed reasoning/text payload against the referenced schema. Would that add useful coverage beyond the attribute assertions? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
2. Or, if no issue exists, describe the change:
Problem:
Part(text=..., thought=True)is exported togen_ai.output.messagesas atextpart, so the trace shows the chain of thought as part of the answer. Thethoughtflag survives on the event but not on the wire, and everything downstream only sees the part type.Solution:
_to_partcheckspart.thoughtand returns aReasoningpart (type: "reasoning") for it. That is theReasoningPartof the v1.39.0 output-message schema the module already references — samecontentfield, differenttype. Input messages share_to_part, so a thought replayed in the history is typed correctly too. Parts without the flag, and every other part kind, are untouched.I picked the part-type mapping over dropping thought parts entirely because the capture rules (
ContentCapturingMode) already decide whether reasoning is captured at all; once it is captured, keeping it in its own part is the only way a backend can collapse or hide it.Testing Plan
Unit Tests:
Added to
tests/unittests/telemetry/test_experimental_semconv.py:test_response_attributes_report_thought_parts_as_reasoning— a thought part before and after the answer, asserting the wholegen_ai.output.messagesvalue.test_input_attributes_report_thought_parts_as_reasoning— the same for a replayed thought ingen_ai.input.messages.Both fail on
main(2 failed, 25 passed) and pass with the change (2 passed, 25 passed), so they do pin the behaviour rather than just documenting it.The two failures are
test_google_cloud.py::test_get_gcp_otlp_metric_exporter_sends_agent_engine_user_agentandtest_spans.py::test_detect_error_discovery_engine_search_tool; both fail identically onmainwithout this change, they want a live GCP / Discovery Engine setup.Manual End-to-End (E2E) Tests:
Scripted
BaseLlmyielding[Part(text='Synthetic thought', thought=True), Part(text='Synthetic answer')], wrapped inContentCapturingMode.SPAN_ONLY, run throughInMemoryRunnerand read back off the finished span — no provider call, no credentials:Checklist
Additional context
The
ReasoningPartshape (type: "reasoning",content: string, both required) is taken fromgen-ai-output-messages.jsonv1.39.0, which_experimental_semconvalready cites in its module docstring.This does change the wire format for anyone currently reading thinking content out of a
textpart. I think that is the point of the issue, but happy to gate it if you would rather keep the old shape behind a flag.