Skip to content

feat(observability): agent sessions read path — queries, API, scratchpad UI - #513

Open
JeremyFunk wants to merge 11 commits into
ai2/03-vendors-rollupfrom
ai2/04-agent-sessions-read
Open

feat(observability): agent sessions read path — queries, API, scratchpad UI#513
JeremyFunk wants to merge 11 commits into
ai2/03-vendors-rollupfrom
ai2/04-agent-sessions-read

Conversation

@JeremyFunk

@JeremyFunk JeremyFunk commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Stack 4. Base: ai2/02-ingest-write-path's columns via ai2/03-vendors-rollup. The first read path over the AI classification columns: an Agent Sessions page fed by typed warehouse queries through one shared read interface.

What this does

One filter payload feeds two tabs over traces:

  • Sessions — rows whose session key resolved at session granularity (AiSessionKeyState = 6), grouped by AiSessionKeyHash: time window, trace count, keyed-span/error counts, vendors, services.
  • AI traces — every AI-classified row (AiVendor != '') at any key state, grouped by TraceId, with bestSessionKeyState explaining why a trace is or isn't in a session.

Classification is strictly per span and one trace routinely mixes vendors (a CrewAI orchestration span parenting OpenInference-instrumented OpenAI calls; an eve turn root over AI SDK spans), so vendors is an array on every row and vendor/service filters mean containment — they live in HAVING over the same aggregates the row reports, never in WHERE, which would silently drop the non-matching spans from the row's own counts. The facet queries arrayJoin one grouped subquery, so facet counts and list rows share a single grouping by construction.

Layering, bottom to top:

  • packages/query-engine/src/ch/tables.ts — the traces DSL mirror gains the five Ai* columns.
  • ch/queries/agent-sessions.tsagentSessionsListQuery / agentTracesListQuery / agentSessionsFacetsQuery, with SQL-catalog baselines so the compiled SQL is reviewed text.
  • observability/agent-sessions.tslistAgentSessions / listAgentTraces / agentSessionsFacets over WarehouseExecutor: the read interface. The coming MCP tools and CLI call these functions, not the routes.
  • packages/domain/src/http/agent-sessions.ts + apps/api/src/routes/internal/agent-sessions.http.ts — the internal HTTP contract and routes.
  • apps/web/src/routes/agent-sessions/ — a deliberately throwaway single-file dashboard page (owner-declared scratchpad) to exercise the path end to end; the real frontend is a later stack. Its one behavioral fix: Reload bypasses the cache-grid window snap, so freshly ingested rows show up immediately instead of after the 5-minute snap window.

Session-key hashes are the only session identity here — the plaintext lives in SpanAttributes under vendor-specific keys, and resolving it for display is the vendor-integration layer's job on a later detail read. UInt64 hashes ride the JSON wire as strings.

Also ships bun run tinybird:truncate (dev convenience used while seeding this page): truncates the local Tinybird workspace's datasources, refusing non-localhost hosts without --force.

Testing

  • Query tests pin the compiled SQL shapes (containment-in-HAVING, state/vendor predicates, facet grouping); the SQL-catalog e2e sweep runs them against a real ClickHouse (analyzer gate green, 162 tests across catalog + rollup).
  • Verified live against tinybird-local with seeded classified spans: both tabs, containment filters, and per-tab facet counts correct in the browser.
  • @maple/query-engine 1234, @maple/domain 514, apps/api 2082, repo typecheck 40/40.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

JeremyFunk and others added 11 commits August 17, 2026 13:36
…s DSL table

The five Ai* columns landed in the warehouse schema (migration 0016) but the
query DSL's traces mirror never learned them, so no read query could reference
them. The mirror test only catches unknown or mistyped columns, not missing
ones — this is the read path's first prerequisite, not a test failure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…acets queries

The first read queries over the AI classification columns. Sessions are
state-6 rows grouped by AiSessionKeyHash; the traces tab is every classified
span grouped by TraceId. Vendor/service filters are containment predicates in
HAVING — per-span classification means one trace or session routinely mixes
vendors, and a WHERE would drop the non-matching spans from the row's own
aggregates. Facet branches arrayJoin one grouped subquery so their counts
share the list's grouping by construction.

Swept through the real ClickHouse analyzer (sql-catalog e2e, 150 checks).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…internal routes

listAgentSessions / listAgentTraces / agentSessionsFacets live in
query-engine/observability over WarehouseExecutor: one read interface, so the
internal HTTP routes here are thin adapters and the future MCP tools call the
same functions. The domain contract is internal-tier on purpose — the feature
is a product scratchpad and these shapes follow the UI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner-flagged throwaway to try out the product shape — one route file, no
abstractions, expect a full rebuild once it settles. Two tabs (sessions /
raw AI traces) over one filter payload, vendor/service/error quickfilters
from the facets endpoint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 24h preset snaps its window end to a 5-minute grid for stable atom keys,
so rows ingested moments ago sit outside the window — and the route ignored
refreshVersion, so Reload re-fetched the same stale window. Wire the
PageRefreshProvider above the page and drop the snap for the reload mount,
mirroring useEffectiveTimeRange.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lists the workspace's datasources and truncates them (optionally filtered by
name); refuses non-localhost hosts without --force since .env.local sometimes
carries cloud tokens.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
normalizeAiSpan derives facts (role, model, tokens, cost, session display
key, conversational content) from a classified span's attributes. A base
normalizer speaks standard gen_ai.* semconv; vendor integrations override
only what their wire format spells differently, verified against captured
data: mastra, eve, vercel_ai_sdk (both dialects), claude_agent_sdk.
Vendors without an integration and the unknown:* buckets run the base.

Dispatch is strictly per span on the stamped AiVendor slug — a single
trace routinely mixes vendors (CrewAI orchestration parenting
openinference-openai LLM spans), so per-trace dispatch would misread the
proven-normal case. Wrapper spans that repeat their children's aggregated
usage (invoke_agent, the legacy AI SDK umbrellas) stay out of the llm
role so llm-tier token totals never double-count.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two-phase detail: the session key hash resolves to TraceIds (only
AiSessionKeyState = 6 rows carry the hash), then every AI span of those
traces is fetched — the session's substance usually lives on spans that
do not carry the key — and run through the vendor integration layer.
getAgentSessionDetail in observability/ is the one durable interface:
the internal HTTP route adapts it and the future MCP tool calls it too.

Phase 2 takes buffered phase-1 time bounds as a partition hint (TraceId
never seeks on the traces sort key) and ships the full SpanAttributes
map — the integration layer owns which keys matter per vendor, so a
projected key list here would couple the query to every integration.
Fetch caps (200 traces / 2000 spans) surface as a truncated flag;
extracted conversational text is capped server-side per span. Totals sum
the llm role tier only: agent-tier wrappers repeat their children's
usage in both AI SDK dialects and would double-count.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
POST /internal/agent-sessions/detail — a thin adapter over
getAgentSessionDetail. The response ships normalized AI spans only: the
raw attribute maps stay behind the read interface, conversational
content travels as the integration layer's extracted (and truncated)
inputText/outputText, and null session means the hash matched nothing
in the window.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant