Skip to content

fix(telemetry): close four AiRequest attribution gaps (UM-1599) - #1703

Merged
jaywedgeworth22 merged 1 commit into
mainfrom
claude/UM-1599-telemetry-attribution
Oct 18, 2026
Merged

jaywedgeworth22 merged 1 commit into
mainfrom
claude/UM-1599-telemetry-attribution

Conversation

@jaywedgeworth22

@jaywedgeworth22 jaywedgeworth22 commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

Fixes four attribution gaps in the per-request telemetry from #1697.  Fixes UM-1599.  Refs UM-1590.

Summary

Closes four gaps where OTLP telemetry from Claude Code and Codex could not be fully attributed to a seat, project, or model in AiRequest records. Each fix is backed by new unit tests.

1. Producer token can write Codex harness rows

Tokens minted under USAGE_INGEST_PRODUCER_TOKENS were only allowed to write rows whose source app exactly matched the producer id. The Codex quota/usage collector writes sourceApp openai-codex, while the logs mapper tags rows with harness codex, so a scoped Codex token was rejected for its own log rows.

  • Added an explicit allow-list, PRODUCER_SOURCE_APP_ALIASES, mapping openai-codex → codex.
  • The alias widens a token to that one name only; it does not leak to other producers, wildcards, or other harnesses (claude-code remains denied).
  • Behaviour is unchanged whether or not scoped tokens are required: the legacy shared token is still refused when USAGE_INGEST_REQUIRE_SCOPED_TOKENS is set.

2. Seat attribution

  • Codex rows that carry no seat resource attribute now default to the platform default seat CODEX.
  • A seat supplied via OTEL_RESOURCE_ATTRIBUTES is still honoured and normalized.
  • Claude Code rows are deliberately left without a default, so absent seat data stays absent rather than being misattributed.

3. Project attribution

  • Claude Code metrics: when no project / project.name attribute is present, the mapper falls back to vcs.repository.name (the repository identity Claude Code emits under OTEL_METRICS_INCLUDE_REPOSITORY=1). An explicit, operator-supplied project still wins, since it reflects the operator's own naming.
  • Claude Code logs / spans: the workspace path now takes precedence over vcs.repository.name, preserving the repository's original casing instead of the lowercased VCS name. The VCS name remains the fallback when no path is exported.
  • The vcs.repository.name, vcs.repository.url.full, vcs.owner.name, and vcs.provider.name attributes are added to the metadata allow-list so repository identity is retained on events.

4. Served vs. requested model

Neither harness exports a standalone response-model attribute, and both emit their telemetry after the response completes, so the reported model is the one that actually ran.

  • Claude request logs and llm_request spans now populate servedModel by falling back to the request model (gen_ai.response.model → gen_ai.request.model → model); an explicit response model still takes precedence when present.
  • Codex completions set servedModel from the completion event's model.
  • Codex api_request / websocket_request events represent pending requests, so they continue to store requestedModel only and never a served model.

Tests

  • ingest-auth-producer-scope.test.ts: alias resolution, alias non-leakage, and behaviour under required scoped tokens.
  • logs-route.test.ts: end-to-end acceptance of the scoped Codex token for Codex rows, rejection for Claude rows, and rejection of the unscoped token.
  • ai-request-mapper.test.ts: Codex seat defaulting, seat/project pass-through, no seat default for Claude, project precedence, and served-model resolution for both harnesses.
  • claude-code-mapper.test.ts: vcs.repository.name as project fallback, explicit project precedence, and vcs.* metadata retention.

Review notes

The review flagged that the new tests mutate OTLP fixtures through as any casts (claude-code-mapper.test.ts, ai-request-mapper.test.ts) rather than the declared OtlpExportMetricsServiceRequest / OtlpExportLogsServiceRequest types; replacing these with typed helpers would improve type safety without changing behaviour.

- ingest-auth: explicit producer alias so the scoped openai-codex token may
  write harness codex (logs route 403 otherwise); no wildcard.
- ai-request-mapper: Codex seat defaults to CODEX when the resource has none;
  servedModel is filled from the only model attribute each source exports
  (Claude api_request and llm_request, Codex response.completed); workspace
  path wins over lowercased vcs.repository.name for project.
- claude-code-mapper, mapping-utils: use vcs.repository.name (emitted under
  OTEL_METRICS_INCLUDE_REPOSITORY=1) as the metric project and keep vcs.* in
  metadata.

Attribute names verified by capturing real claude and codex-cli 0.163.0
exports on localhost.  Codex honours OTEL_RESOURCE_ATTRIBUTES but has no
config.toml key for resource attributes and sends no cwd.

Refs UM-1590

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

UM-1599

UM-1590

@kody-ai

kody-ai Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Code Review Completed! 🔥

The code review was successfully completed based on your current configurations.

Kody Guide: Usage and Configuration
Interacting with Kody
  • Request a Review: Ask Kody to review your PR manually by adding a comment with the @kody start-review command at the root of your PR.

  • Validate Business Logic: Ask Kody to validate your code against business rules by adding a comment with the @kody -v business-logic command.

  • Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Current Kody Configuration
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ✅

Access your configuration settings here.

​

@jaywedgeworth22
jaywedgeworth22 enabled auto-merge (squash) October 18, 2026 03:57
@jaywedgeworth22
jaywedgeworth22 merged commit 68c419d into main Oct 18, 2026
10 of 12 checks passed
@jaywedgeworth22
jaywedgeworth22 deleted the claude/UM-1599-telemetry-attribution branch October 18, 2026 04:00
@kody-ai

kody-ai Bot commented Oct 18, 2026

Copy link
Copy Markdown

kody code-review Kody Rules medium

as any casts in four new tests at src/lib/otlp/tests/ai-request-mapper.test.ts (lines 74, 84, 93, 101) strip typed OTLP fixture payloads to any, violating Rule 10's prohibition on new as any in added lines. Mutate the fixture through the existing OtlpExportLogsServiceRequest type or a typed builder helper to retain schema validation.

Kody rule violation: Every exported schema or utility ships with tests and must typecheck

@kody-ai

kody-ai Bot commented Oct 18, 2026

Copy link
Copy Markdown

kody code-review Business Logic medium

🤔 Insufficient Task Context

I found a task linked to this PR, but it only contains minimal information (title only, no description or acceptance criteria). To perform a meaningful business rules validation, I need more details.

🔍 What I need to validate:

  • Business requirements and acceptance criteria
  • Expected behavior and business rules
  • Edge cases and constraints to consider

💡 How to improve the task context:

  • Add a description to the linked ticket
  • Include acceptance criteria or business rules
  • Describe the expected behavior after the change

⚠️ Important:

A task title alone is not sufficient to determine whether the implementation is correct or complete.


💡 This validation runs automatically only on the first review of a pull request. To run it again, comment @kody -v business-logic.


function withPointAttrs(extra: unknown[]) {
const request = sampleRequest();
for (const metric of (request.resourceMetrics as any)[0].scopeMetrics[0].metrics) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Kody Rules medium

Disallowed as any cast in the withPointAttrs test helper at src/lib/otlp/tests/claude-code-mapper.test.ts bypasses TypeScript checking in violation of Rule [90]. OtlpExportMetricsServiceRequest.resourceMetrics is a properly typed array, so request.resourceMetrics[0].scopeMetrics[0].metrics typechecks without the cast.

Also found in:

  • src/lib/otlp/__tests__/claude-code-mapper.test.ts:341-341
  • src/lib/otlp/__tests__/ai-request-mapper.test.ts:74-74
  • src/lib/otlp/__tests__/ai-request-mapper.test.ts:101-101
  • src/lib/otlp/__tests__/ai-request-mapper.test.ts:84-84
  • src/lib/otlp/__tests__/ai-request-mapper.test.ts:93-93

Kody rule violation: Keep web TypeScript type-checked: no silent type escapes

Prompt for LLM

File src/lib/otlp/__tests__/claude-code-mapper.test.ts:

Line 319:

Disallowed `as any` cast in the `withPointAttrs` test helper at src/lib/otlp/__tests__/claude-code-mapper.test.ts bypasses TypeScript checking in violation of Rule [90]. `OtlpExportMetricsServiceRequest.resourceMetrics` is a properly typed array, so `request.resourceMetrics[0].scopeMetrics[0].metrics` typechecks without the cast.

**Also found in:**
- `src/lib/otlp/__tests__/claude-code-mapper.test.ts:341-341`
- `src/lib/otlp/__tests__/ai-request-mapper.test.ts:74-74`
- `src/lib/otlp/__tests__/ai-request-mapper.test.ts:101-101`
- `src/lib/otlp/__tests__/ai-request-mapper.test.ts:84-84`
- `src/lib/otlp/__tests__/ai-request-mapper.test.ts:93-93`

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant