feat(usage): Capture orchestrator usage through a source ledger - #103
Merged
Merged
Conversation
Count cached tokens inside Codex input once and reject inconsistent token totals. Co-Authored-By: Codex <noreply@anthropic.com>
Store source high-water marks, row attributions, and native session links so usage can be reconciled after restart. Co-Authored-By: Codex <noreply@openai.com>
Keep worker metrics separate from orchestrator usage while exposing a combined cost. Co-Authored-By: Codex <noreply@anthropic.com>
Expose orchestrator and worker totals so usage views include open sessions. Co-Authored-By: Codex <noreply@anthropic.com>
Give each harness a stable source key for usage attribution. Co-Authored-By: Codex <noreply@openai.com>
Track source marks and per-session deltas so cumulative reports can move across rows without double counting. Co-Authored-By: Codex <noreply@openai.com>
Keep every native transcript id linked to its open session and expose unreconciled links for recovery. Co-Authored-By: Codex <noreply@openai.com>
Extract cumulative costs and token totals from Claude transcript files. Co-Authored-By: Codex <noreply@openai.com>
Expose the associated run id to spawned workers for usage attribution. Co-Authored-By: Codex <noreply@openai.com>
A first cost report of zero is known usage, not missing usage. Persist its high-water mark and attribution so run completeness remains accurate. Co-Authored-By: Codex <noreply@openai.com>
Keep token updates on the shared session write path and clear nullable cost directly, which SessionStore.update cannot represent. Co-Authored-By: Codex <noreply@openai.com>
Keep zero-token synthetic models from making priced fallback usage incomplete. Measure memory while parsing the large transcript fixture. Co-Authored-By: Codex <noreply@openai.com>
Keep headroom between the generated file size and the memory limit. Co-Authored-By: Codex <noreply@openai.com>
Track each non-incremental worker source at its high-water mark while keeping incremental events additive. Cover Claude process changes, Codex thread totals, and event replay with hand-made stream fixtures. Co-Authored-By: Codex <noreply@openai.com>
Accept native ids only for open rows and validate reported costs. Return worker and orchestrator usage with source attributions in usage.get. Co-Authored-By: Codex <noreply@openai.com>
Record transcript cost and token high-water values on release, and revisit older links when an open row receives another native id. Keep release status even when transcript reading fails. Co-Authored-By: Codex <noreply@openai.com>
Schedule transcript reconciliation after recovery without delaying the socket listener. Preserve dead working rows that recovery terminalizes, and keep the reconciliation promise available to tests. Co-Authored-By: Codex <noreply@openai.com>
Carry the stored run id through DriverSession.send so follow-up worker processes keep CODEDECK_RUN_ID. Pass the run id to the initial driver start. Co-Authored-By: Codex <noreply@openai.com>
Retry failed transcript reads for completed or failed open rows after a daemon restart. Keep startup recovery non-blocking and verify the retry. Co-Authored-By: Codex <noreply@openai.com>
Derive Claude source numbering from the session.started count at the current event sequence, matching the worker usage model. Co-Authored-By: Codex <noreply@openai.com>
Attribute pre-upgrade usage to the incoming source when it has no mark, so the next cumulative observation adds only its increase. Keep the seed source when the incoming source already has a mark on another row. Co-Authored-By: Codex <noreply@openai.com>
Prove last-valid sidecar selection and Claude close-time linking order. Co-Authored-By: Codex <noreply@openai.com>
Refresh reconciled transcript totals when an open row is resumed or later released. The ledger high-water marks absorb unchanged cumulative values. Co-Authored-By: Codex <noreply@openai.com>
Place unclassified prior usage under a seed source when the latest usage event came from an earlier Claude process. Co-Authored-By: Codex <noreply@openai.com>
Treat open rows linked to missing or unpriced transcripts as incomplete in usage analytics. Co-Authored-By: Codex <noreply@openai.com>
Recover transcript usage when the watcher misses its final native link. Co-Authored-By: Codex <noreply@openai.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



codedeck opensessions recorded no usage, and they are almost all of the real spend. Worker usage also lost cost across process restarts: harnesses report a cumulative value per native session or process, and the store replaced the row's usage with the latest one.Usage now goes through a ledger:
usage_sourceskeeps a global high-water mark per source key (claude:<nativeId>#<process>,codex:<thread>,session:<id>,claude-open:<nativeId>). A new observation adds only its increase over the mark, so re-reading the same cumulative value adds nothing.usage_attributionsrecords what each row got from each source and is materialized intosessions.usage_*, so existing readers keep working.session_native_linkskeeps every native id anopenrow saw. The session-id hook now appends ids to the sidecar, andopenruns a link watcher that sendssession.linkNativeand flushes beforefinishOpenSessionandsession.release.cost-state, else deduped assistant tokens priced per model) and observes it. Release re-reads every link of the row, which covers resuming an id on a revived row and/clearthen/resumeinside oneopen.codedeck usage <run> --json --observe <session_id>=<cost>.run $is workers plus the orchestrator sources other than the current session plus the live payload cost.RunUsageSummarygainsorchestratorandtotal; the top level stays workers only.codedeck usagegains--by originand--backfill, which imports past orchestrator transcripts asusage_legacyentries with a mark, so a later resume of the same id counts only the increase.seed:<sessionId>otherwise (and when a Claude row's previous usage came from another process), so the upgrade neither drops nor double counts them.Spec, design and tasks:
.specs/features/orchestrator-usage/(ORCH, RUN, SRC, PRICE, BF requirements).The daemon must be restarted after deploy to create the new tables; the migration only adds tables and indexes.
Not covered: the Claude statusline cost after
--resumeand--fork-sessionwas not checked against the real binary, Antigravity and OMP counters across processes are assumed cumulative per session, and the TUI dashboard with--by originwas not exercised. Backfilled legacy entries have no row attribution by design.Tests
Scoped batches, one at a time, on the final HEAD:
npx vitest run tests/orchestrator-usage-daemon.test.ts tests/usage-daemon.test.ts tests/release-interrupted.test.ts tests/session-adopt.test.ts: 56 passednpx vitest run tests/usage-ledger.test.ts tests/native-links.test.ts tests/usage-query.test.ts tests/usage-backfill.test.ts tests/usage-cli.test.ts tests/session-store.test.ts: 39 passednpx vitest run tests/pricing.test.ts tests/usage-source.test.ts tests/claude-transcript.test.ts tests/usage-schema.test.ts tests/power-database.test.ts tests/usage.test.ts: 45 passednpx vitest run tests/statusline.test.ts tests/link-watcher.test.ts tests/session-id-hook.test.ts tests/session-runtime.test.ts tests/run-env.test.ts tests/usage-statusline-contract.test.ts: 36 passednpx vitest run tests/open-action.test.ts tests/open-contract.test.ts: 54 passednpx vitest run tests/open-pty.test.ts tests/open-args.test.ts: 130 passednpx vitest run tests/power-recover.test.ts tests/run-linkage.test.ts tests/opencode-parser.test.ts tests/usage-charts.test.ts: 24 passednpx tsc --noEmit: clean;npm run build: okcodedeck usage <run> --json --observe <id>=<cost>: about 72 ms per call (62 ms before)cost-state, ORCH-16 previous ids, the RUN-06 exclusion, sidecar append, last sidecar id, the close-path flush and the worker-only top level were all caught.sequence <= ?vs<in the process ordinal is equivalent (event sequences are unique).