[fix](ci) Migrate code review tracing to OTLP

[shuke987]

What problem does this PR solve?

Issue Number: None

Related PR: #67416

Problem Summary:

The current Litefuse deployment rejects the legacy trace-create, span-create, and generation-create events sent to /api/public/ingestion. The Code Review workflow itself remained green because trace recording is non-blocking, but the Litefuse step returned HTTP 400 and new review traces stopped appearing after August 28.

This PR is the minimal functional fix. It converts the existing in-memory review event model to OTLP/HTTP JSON spans and sends them to /api/public/otel/v1/traces with x-langfuse-ingestion-version: 4. It preserves the trace/span hierarchy, root and child observation input/output, trace metadata and tags, model usage, encoded-payload chunking, HTTP 413 handling, transport-error splitting, and OTLP partial-success detection.

This PR does not change review decisions, token handling, workflow triggers, or Doris product code. Transient HTTP retry/backoff, v2 observation pagination, complete read-back validation, duplicate detection, and large-trace chunking optimization are intentionally separated into #67416.

Release note

None

Check List (For Author)

• Test: Unit Test / Manual test
  • python3 .github/scripts/test_emit_litefuse_otel_io.py (8 tests passed)
  • Local tests cover OTLP conversion and hierarchy, v4 headers, encoded-payload chunking, oversized-span truncation, HTTP 413 splitting, transport-error splitting, and partial-success rejection
  • Live Litefuse OTLP canary write returned HTTP 200
  • Read-back verification found all 3 expected observations with valid hierarchy and input/output
• Behavior changed: Yes. Litefuse trace export switches from the legacy ingestion API to OTLP/HTTP JSON. Code review behavior is unchanged.
• Does this need documentation: No

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[shuke987]

/review

[github-actions]

Requesting changes for three reproducible OTLP migration defects.

Critical checkpoint conclusions:

• Goal and correctness: the endpoint, authentication/version header, OTLP IDs, timestamps, hierarchy, attributes, and partial-success handling are generally sound, but subagent root I/O, full-cap sizing, and singleton-413 recovery are not.
• Scope: the change is focused on the exporter and its tests.
• Concurrency and lifecycle: there is no shared-memory concurrency; the relevant network export lifecycle and main/subagent parentage were reviewed. At-least-once resend after an unacknowledged disconnect is OTLP's explicit delivery tradeoff, so it is not raised as a correctness defect here.
• Configuration and compatibility: no new Doris configuration, FE/BE protocol, storage format, persistence, transaction, or rolling-upgrade path is involved; the v4 endpoint and header are correct.
• Parallel and conditional paths: main/subagent, singleton/multi-span, 413, transport-error, HTTP-error, partial-success, dry-run, and readback paths were traced. The three inline comments cover the distinct surviving failures.
• Tests and results: PYTHONDONTWRITEBYTECODE=1 python3 .github/scripts/test_emit_litefuse_otel_io.py -v passes all 8 tests, but the suite misses the three producer/boundary cases reported inline. Minimum-only readback and complete ID/subagent verification are explicitly deferred to #67416.
• Observability and performance: acknowledged-request counters are coherent, but premature prechunking silently discards valid telemetry and singleton 413 recovery can fail without trying a request that would fit.
• Security and data writes: not applicable; this change does not alter a Doris product data path or security boundary.
• User focus: no additional user-provided focus was supplied, so the full PR was reviewed.

Review status: complete after 3 rounds; every candidate was accepted, deduplicated, or dismissed with evidence.

[github-actions]

[P1] Preserve the subagent task on the v4 root observation

This always sources OTLP input from the span body, but build_subagent_session_payload stores the actual first user message only in trace-create.body.input; its parentless codex.subagent.review span contains just session_file and thread_id. Since the trace event is discarded, the v4 root/evaluation surface loses the overall task even though the text survives on a child user-message observation and the placeholder root input lets read-back look non-empty. Langfuse v4 requires overall request/response on the root observation. Please put the task input on the subagent root while retaining the file/thread identifiers as metadata, and cover the producer-to-OTLP path in a test.

[github-actions]

[P1] Do not shrink a span against the half-size prechunk limit

Passing max_payload_bytes // 2 into chunk_payload makes it truncate a single legacy event before add_chunk measures the real OTLP request against the configured limit. I reproduced a 2,101,028-byte OTLP span under the workflow's 4 MB cap being reduced to about 3 KB with nearly all I/O replaced by truncated_json. This is reachable when an agent-message input accumulates many context events. Please use the pre-pass only to partition multi-event batches; measure each individual OTLP span against the full active limit before shrinking it, and add a half-limit/full-limit boundary test.

[github-actions]

[P1] Do not make half the rejected size a hard OTLP retry ceiling

For a singleton 413, request_size // 2 becomes the maximum for the complete re-encoded OTLP envelope, including fixed resource/scope and repeated trace attributes that shrink_event_for_payload cannot reduce. I reproduced a 4,112-byte request rejected by a 3,912-byte server cap: a moderately reduced 3,912-byte request would fit, but this path instead required the whole envelope to fit 2,056 bytes and raised before making a second request. Please shrink the observation content stepwise while measuring the full envelope, retry only requests that are strictly smaller than the rejected one, and fail explicitly at the fixed-envelope floor instead of imposing an arbitrary half-size ceiling; add a singleton threshold regression.

[shuke987]

/review

[github-actions]

Requesting changes for two additional, independently reproducible defects in the OTLP migration.

Critical checkpoint conclusions:

• Correctness and schema: OTLP IDs, timestamps, hierarchy, AnyValue attributes, root input placement, status/usage mapping, endpoint, authentication/version headers, and partial-success handling were checked against the v4 contract. The two findings below are the remaining correctness failures.
• Scope: the PR is focused on the exporter migration and its Python tests; no unrelated Doris product paths are changed.
• Concurrency and lifecycle: there is no shared-memory concurrency. Main/subagent posting, singleton/multi-span splitting, 413 handling, transport-error handling, dry-run, and readback fallback paths were traced.
• Configuration and compatibility: the v4 endpoint/header migration is appropriate, but v2 readback compatibility is incomplete because its cursor is not consumed.
• Parallel and conditional paths: existing inline threads were used as duplicate fences. The initial OTLP singleton truncation path and the v2 newest-first pagination path are distinct from those threads.
• Tests and results: PYTHONDONTWRITEBYTECODE=1 python3 .github/scripts/test_emit_litefuse_otel_io.py -v passes all 11 tests; py_compile and git diff --check also pass. A just-over-limit singleton and a >100-observation v2 readback case are not covered.
• Observability and performance: MF-001 silently discards valid input/output near the configured cap; MF-002 can report verification failure after successful ingestion. Transport replay, complete subagent verification, and test-harness wiring were reviewed but are respectively an already-documented at-least-once tradeoff, prior deferred scope, and test debt rather than new blocking findings.
• Security and data writes: not applicable; this script does not alter a Doris data/security boundary.
• User focus: review_focus.txt contains no additional focus, so the full PR was reviewed.

Additional findings:

1. MF-001: the initial singleton truncation loop starts at max_payload_bytes // 2, so an OTLP request only a few bytes over the cap is re-encoded far below the limit and loses most of its output. The inline comment is on emit_litefuse_otel_io.py diff position 208.
2. MF-002: fetch_observations_v2 requests only the first 100 newest observations and does not follow meta.cursor; on traces with more than 100 observations, the oldest codex.review root can be omitted and --verify fails to recover root input/output even though the v4 ingest succeeded. The inline comment is anchored to the changed v4 endpoint line and points to the reader at source line 1507.

The three existing inline threads (r3910285346, r3910285352, and r3910285357) were re-read and treated as hard duplicate fences; no substantially similar comments are submitted here. All candidates from both review rounds are now accepted, deduplicated, or dismissed with evidence.

Review status: complete after 2 rounds.

[github-actions]

Requesting changes for two additional, independently reproducible defects in the OTLP migration.

Critical checkpoint conclusions:

• Correctness and schema: OTLP IDs, timestamps, hierarchy, AnyValue attributes, root input placement, status/usage mapping, endpoint, authentication/version headers, and partial-success handling were checked against the v4 contract. The two findings below are the remaining correctness failures.
• Scope: the PR is focused on the exporter migration and its Python tests; no unrelated Doris product paths are changed.
• Concurrency and lifecycle: there is no shared-memory concurrency. Main/subagent posting, singleton/multi-span splitting, 413 handling, transport-error handling, dry-run, and readback fallback paths were traced.
• Configuration and compatibility: the v4 endpoint/header migration is appropriate, but v2 readback compatibility is incomplete because its cursor is not consumed.
• Parallel and conditional paths: existing inline threads were used as duplicate fences. The initial OTLP singleton truncation path and the v2 newest-first pagination path are distinct from those threads.
• Tests and results: PYTHONDONTWRITEBYTECODE=1 python3 .github/scripts/test_emit_litefuse_otel_io.py -v passes all 11 tests; py_compile and git diff --check also pass. A just-over-limit singleton and a >100-observation v2 readback case are not covered.
• Observability and performance: MF-001 silently discards valid input/output near the configured cap; MF-002 can report verification failure after successful ingestion. Transport replay, complete subagent verification, and test-harness wiring were reviewed but are respectively an already-documented at-least-once tradeoff, prior deferred scope, and test debt rather than new blocking findings.
• Security and data writes: not applicable; this script does not alter a Doris data/security boundary.
• User focus: review_focus.txt contains no additional focus, so the full PR was reviewed.

Additional findings:

1. MF-001: the initial singleton truncation loop starts at max_payload_bytes // 2, so an OTLP request only a few bytes over the cap is re-encoded far below the limit and loses most of its output. See the inline comment at the divisor-2 target.
2. MF-002: fetch_observations_v2 requests only the first 100 newest observations and does not follow meta.cursor; on traces with more than 100 observations, the oldest codex.review root can be omitted and --verify fails to recover root input/output even though the v4 ingest succeeded. See the inline comment at the changed v4 endpoint, which points to the reader at source line 1507.

The three existing inline threads (r3910285346, r3910285352, and r3910285357) were re-read and treated as hard duplicate fences; no substantially similar comments are submitted here. All candidates from both review rounds are now accepted, deduplicated, or dismissed with evidence.

Review status: complete after 2 rounds.

[github-actions]

[P1] Preserve near-limit singleton payloads

When an OTLP singleton is only slightly over max_payload_bytes, the first fallback target is max_payload_bytes // 2, so a 6,022-byte request under a 6,000-byte cap gets re-encoded around 3,075 bytes and loses most of its output. Please choose a target close to the active limit (for example, search/binary-search the minimum truncation needed), and add a just-over-limit regression; the current 50,000-byte oversized test will not catch this data loss.

[github-actions]

[P1] Paginate v2 readback before declaring verification failure

The v2 endpoint is cursor-paginated and returns newest observations first. A trace with more than 100 observations therefore omits the oldest codex.review root from this single-page request; when the legacy endpoint is unavailable, verify_trace then reports failure even though ingestion succeeded because it cannot recover root input/output. Follow meta.cursor (or query the root explicitly) and add a >100-observation regression; the reader is fetch_observations_v2 at source line 1507.