feat(observability): agent sessions read path — queries, API, scratchpad UI - #513
Open
JeremyFunk wants to merge 11 commits into
Open
feat(observability): agent sessions read path — queries, API, scratchpad UI#513JeremyFunk wants to merge 11 commits into
JeremyFunk wants to merge 11 commits into
Conversation
…s DSL table The five Ai* columns landed in the warehouse schema (migration 0016) but the query DSL's traces mirror never learned them, so no read query could reference them. The mirror test only catches unknown or mistyped columns, not missing ones — this is the read path's first prerequisite, not a test failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…acets queries The first read queries over the AI classification columns. Sessions are state-6 rows grouped by AiSessionKeyHash; the traces tab is every classified span grouped by TraceId. Vendor/service filters are containment predicates in HAVING — per-span classification means one trace or session routinely mixes vendors, and a WHERE would drop the non-matching spans from the row's own aggregates. Facet branches arrayJoin one grouped subquery so their counts share the list's grouping by construction. Swept through the real ClickHouse analyzer (sql-catalog e2e, 150 checks). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…internal routes listAgentSessions / listAgentTraces / agentSessionsFacets live in query-engine/observability over WarehouseExecutor: one read interface, so the internal HTTP routes here are thin adapters and the future MCP tools call the same functions. The domain contract is internal-tier on purpose — the feature is a product scratchpad and these shapes follow the UI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner-flagged throwaway to try out the product shape — one route file, no abstractions, expect a full rebuild once it settles. Two tabs (sessions / raw AI traces) over one filter payload, vendor/service/error quickfilters from the facets endpoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 24h preset snaps its window end to a 5-minute grid for stable atom keys, so rows ingested moments ago sit outside the window — and the route ignored refreshVersion, so Reload re-fetched the same stale window. Wire the PageRefreshProvider above the page and drop the snap for the reload mount, mirroring useEffectiveTimeRange. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lists the workspace's datasources and truncates them (optionally filtered by name); refuses non-localhost hosts without --force since .env.local sometimes carries cloud tokens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
normalizeAiSpan derives facts (role, model, tokens, cost, session display key, conversational content) from a classified span's attributes. A base normalizer speaks standard gen_ai.* semconv; vendor integrations override only what their wire format spells differently, verified against captured data: mastra, eve, vercel_ai_sdk (both dialects), claude_agent_sdk. Vendors without an integration and the unknown:* buckets run the base. Dispatch is strictly per span on the stamped AiVendor slug — a single trace routinely mixes vendors (CrewAI orchestration parenting openinference-openai LLM spans), so per-trace dispatch would misread the proven-normal case. Wrapper spans that repeat their children's aggregated usage (invoke_agent, the legacy AI SDK umbrellas) stay out of the llm role so llm-tier token totals never double-count. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two-phase detail: the session key hash resolves to TraceIds (only AiSessionKeyState = 6 rows carry the hash), then every AI span of those traces is fetched — the session's substance usually lives on spans that do not carry the key — and run through the vendor integration layer. getAgentSessionDetail in observability/ is the one durable interface: the internal HTTP route adapts it and the future MCP tool calls it too. Phase 2 takes buffered phase-1 time bounds as a partition hint (TraceId never seeks on the traces sort key) and ships the full SpanAttributes map — the integration layer owns which keys matter per vendor, so a projected key list here would couple the query to every integration. Fetch caps (200 traces / 2000 spans) surface as a truncated flag; extracted conversational text is capped server-side per span. Totals sum the llm role tier only: agent-tier wrappers repeat their children's usage in both AI SDK dialects and would double-count. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
POST /internal/agent-sessions/detail — a thin adapter over getAgentSessionDetail. The response ships normalized AI spans only: the raw attribute maps stay behind the read interface, conversational content travels as the integration layer's extracted (and truncated) inputText/outputText, and null session means the hash matched nothing in the window. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack 4. Base:
ai2/02-ingest-write-path's columns viaai2/03-vendors-rollup. The first read path over the AI classification columns: an Agent Sessions page fed by typed warehouse queries through one shared read interface.What this does
One filter payload feeds two tabs over
traces:AiSessionKeyState = 6), grouped byAiSessionKeyHash: time window, trace count, keyed-span/error counts, vendors, services.AiVendor != '') at any key state, grouped byTraceId, withbestSessionKeyStateexplaining why a trace is or isn't in a session.Classification is strictly per span and one trace routinely mixes vendors (a CrewAI orchestration span parenting OpenInference-instrumented OpenAI calls; an eve turn root over AI SDK spans), so
vendorsis an array on every row and vendor/service filters mean containment — they live inHAVINGover the same aggregates the row reports, never inWHERE, which would silently drop the non-matching spans from the row's own counts. The facet queriesarrayJoinone grouped subquery, so facet counts and list rows share a single grouping by construction.Layering, bottom to top:
packages/query-engine/src/ch/tables.ts— the traces DSL mirror gains the fiveAi*columns.ch/queries/agent-sessions.ts—agentSessionsListQuery/agentTracesListQuery/agentSessionsFacetsQuery, with SQL-catalog baselines so the compiled SQL is reviewed text.observability/agent-sessions.ts—listAgentSessions/listAgentTraces/agentSessionsFacetsoverWarehouseExecutor: the read interface. The coming MCP tools and CLI call these functions, not the routes.packages/domain/src/http/agent-sessions.ts+apps/api/src/routes/internal/agent-sessions.http.ts— the internal HTTP contract and routes.apps/web/src/routes/agent-sessions/— a deliberately throwaway single-file dashboard page (owner-declared scratchpad) to exercise the path end to end; the real frontend is a later stack. Its one behavioral fix: Reload bypasses the cache-grid window snap, so freshly ingested rows show up immediately instead of after the 5-minute snap window.Session-key hashes are the only session identity here — the plaintext lives in
SpanAttributesunder vendor-specific keys, and resolving it for display is the vendor-integration layer's job on a later detail read.UInt64hashes ride the JSON wire as strings.Also ships
bun run tinybird:truncate(dev convenience used while seeding this page): truncates the local Tinybird workspace's datasources, refusing non-localhost hosts without--force.Testing
@maple/query-engine1234,@maple/domain514,apps/api2082, repo typecheck 40/40.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.