Skip to content

feat(ingest): sessions, models, tokens and cost for Codex and Grok CLI agents - #167

Merged
Fl0p merged 2 commits into
mainfrom
feat/ingest-codex-grok
Nov 15, 2026
Merged

Fl0p merged 2 commits into
mainfrom
feat/ingest-codex-grok

Conversation

@Fl0p

@Fl0p Fl0p commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Why

A ping to the Paperclip agents on 2026-11-15 (FLO-1172..1174) showed how non-Claude harnesses reach cotel:

Agent Harness Spans Attributed Session / model / tokens / cost
Aldric Codex (codex-app-server 0.160) yes, ~3.6k per short run yes, by paperclip.agent.id on the span none, $0
Vesper Grok CLI (grok-cli 1.0.77) yes no, Anonymous none, $0
Soren Cursor Agent CLI none - -

Neither harness uses Claude Code's attribute names, so every span was stored with an empty session_id. That is why Aldric is missing from Sessions. Both also emit several spans per logical event, so the fields cannot simply be renamed without double counting.

What

  • internal/ingest/producers.go: one adapter per harness, chosen by service.name. Each runs after the generic extraction and before cost is derived. Claude Code spans are untouched (covered by a test).
    • Codex
      • Session comes from conversation.id, or from thread.id when it is a UUID. On most spans thread.id is the tracing layer's OS thread number, an integer.
      • Usage comes from the turn span's codex.turn.token_usage.*: non_cached_input_tokens as input and cached_input_tokens as cache read. The per-request gen_ai.usage.* on handle_responses is not read, because it sums to the turn total.
      • A tool call is a span with both tool_name and call_id. code_mode.broker.invoke_tool repeats the nested call without a call_id.
    • Grok
      • Session comes from session_id, model from model_id.
      • Tokens are read only from session.process_conversation_turn, because sampling.stream_decode repeats output_tokens per sample. cache_read_tokens is taken out of the inclusive input_tokens.
      • Only tool.execution counts as a tool call, not tool.decision.
  • Prices: gpt-5.6-sol at $4 input, $0.40 cached, $20 output; grok-4.7 and grok-4.6 at $2, $0.50, $6. These are base rates; the long-prompt tiers are not modelled, as with Haiku 5.5. They come from third-party rate cards citing the providers, because openai.com/api/pricing and docs.x.ai were not reachable. Worth a check before merge.
  • Session counts on actors, overview and history use COUNT(DISTINCT NULLIF(session_id, '')). A span without a session was being counted as one session: Aldric showed 1 session on Actors and none on Sessions.

Not in scope

  • Stored spans are not rewritten. session_id is ART-indexed, and bulk updates to an indexed column are the pattern behind the September corruption. Only new Codex and Grok runs get sessions.
  • Grok stays Anonymous. It carries no paperclip.agent.id, either because it ignores OTEL_RESOURCE_ATTRIBUTES or because the adapter does not pass it. To be looked at separately.
  • Cursor exports no telemetry at all.
  • Codex volume. About 3.6k spans per short run, mostly fs.get_metadata, handle_responses and receiving. Filtering them at ingest is a separate decision.

Testing

  • New internal/ingest/producers_test.go covers both harnesses, built on the real span shapes with synthetic ids: sessions, token split, cost, non-session thread.id, no double counting, and tool selection. TestActorSessionsIgnoreEmptySessionID fails on main (1 and 2 sessions instead of 0 and 1) and passes here.

  • Replaying today's production Codex and Grok spans (export, run locally) through the new ingest gives:

    • one Codex session: gpt-5.6-sol, 44k input, 130k cached, 934 output, ≈$0.25;
    • two Grok sessions on grok-4.7, ≈$0.01 and ≈$0.16.

    No tokens remain on sessionless spans.

  • go test ./... is green with TZ=UTC. In a non-UTC zone, TestHistory_TenMinuteGranularity and TestRollupAndPurge_BoundaryDayNotSplit fail. They fail identically on main.

🤖 Generated with Claude Code

…I agents

Paperclip's Codex (codex-app-server) and Grok (grok-cli) agents export
spans that ingest stored with no session, model or tokens: neither
harness uses Claude Code's attribute names, and both emit several spans
per event. They never reached the Sessions page and cost $0.

- internal/ingest/producers.go: one adapter per harness, chosen by
  service.name, run after the generic extraction and before cost is
  derived. Claude Code spans are untouched.
  - Codex: session from conversation.id, else thread.id when it is a
    UUID (elsewhere it is the OS thread number); usage from the turn
    span's codex.turn.token_usage.* with cached input split out; the
    per-request gen_ai.usage.* copies are not read (they sum to the
    turn); a tool call is a span with tool_name and call_id.
  - Grok: session from session_id, model from model_id; tokens only on
    session.process_conversation_turn, with cache_read_tokens taken out
    of its inclusive input_tokens; only tool.execution is a tool call.
- Prices for gpt-5.6-sol ($4/$0.40/$20) and grok-4.7, grok-4.6
  ($2/$0.50/$6), base rates, from third-party rate cards citing the
  providers.
- Session counts (actors, overview, history) ignore an empty session_id,
  which was counted as one session.

Already-stored spans are not rewritten: session_id is an indexed column.
Grok still lands on Anonymous (it carries no paperclip.agent.id); Cursor
exports nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 967d3650-b942-4f63-9c29-e182a8ba2006
Commits

Reviewing files that changed from the base of the PR and between 8c88d83 and 1797917.

Files selected for processing: 9

  • CHANGELOG.md
  • internal/api/handler.go
  • internal/ingest/handler.go
  • internal/ingest/handler_test.go
  • internal/ingest/producers.go
  • internal/ingest/producers_test.go
  • internal/pricing/prices.go
  • internal/storage/actors.go
  • internal/storage/actors_page_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

The dashboard lowercases the Paperclip agent id it stores, but the ingest
lookup compared the id as the sender sent it, so an uppercase UUID left
that agent's spans on Anonymous. Raised by CodeRabbit on #166.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fl0p
Fl0p merged commit e3849ca into main Nov 15, 2026
10 checks passed
@Fl0p
Fl0p deleted the feat/ingest-codex-grok branch November 15, 2026 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant