Fix codex token accounting and remove the two dead codex transports - #66
Merged
Conversation
Two token fields reported by providers were being discarded on their way to costing: 1. The legacy aisdk-codex path (mapCodexUsage) hardcoded cacheWrite: undefined. GPT-5.6 reports input_tokens_details.cache_write_tokens (billed at 1.25x the input rate; openai/codex#32479 is Codex CLI fixing the same drop, and openai-python 2.45 made the field required). Dropped writes ride inside the uncached bucket at 1.0x — undercosted and invisible. The vendor/SSE path already parses the field since the opencode-llm-vendor rewrite; this closes the remaining path, with noCache now subtracting both cache counters. 2. TokenUsageAccumulator derived uncached input by subtraction even though the AI SDK carries the provider's own figure (inputTokenDetails.noCacheTokens). extractUsage now passes it through, add() sums it with poisoning semantics (one call without it makes the model's sum undefined rather than a misleading partial), and snapshot() prefers it over the derived value — so a provider whose breakdown doesn't perfectly telescope (cache-block rounding) is billed on its own accounting. Field evidence from the riptide dev database (agentlayer 0.0.74/75, 8 weeks): 7,012 codex usage rows totalling 12.8B input tokens and 12.1B cache reads carry cache_creation_tokens = 0 on every single row — zero recorded cache writes against 12B cache reads is a dropped field, not telemetry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Docs Agent ReviewAgent finished with reason: error To apply these recommendations, comment: |
…onses transport Four confirmed findings from the fresh-context review of this branch: 1. snapshot() trusted a provider-reported noCacheInputTokens with only a >= 0 clamp: a non-telescoping breakdown billed more prompt-category tokens than the prompt contained (the branch's own test billed 1.05M on a 1M prompt), and a pathological figure was unbounded. The priced categories are now reconciled to PARTITION the prompt total — noCache capped to inputTokens, cacheRead/cacheWrite capped to the remainder — and extractUsage clamps a negative provider figure so it can never drag a summed count below zero. 2. Only the legacy codex transport got the cache_write mapping. The vendored copilot Responses adapter had the identical unpatched shape in all four of its usage sites (doGenerate + streaming, schema and mapping): its zod schema now parses cache_write_tokens, both mappers emit cacheWrite and subtract both counters from noCache. The aisdk_responses transport delegates to upstream @ai-sdk/openai and cannot be fixed here — documented as a known gap in the PR. 3. performCompaction's hardcoded usage-key list silently dropped noCacheInputTokens from the compaction event's summaryUsage; it now accumulates with the same poisoning rule as the accumulator. 4. mapCodexUsage fabricated a "provider-reported" noCache by local subtraction even when the backend sent no input_tokens_details, permanently disabling downstream absence-keyed fallbacks; noCache is now reported only when a breakdown exists to derive it from. One assertion that pinned the fabricated value is updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both verified unused before removal (repo greps in agentlayer and synclayer, GitHub code search across the org, deployment env/infra): - legacy.ts (createCodexProvider and its nine sibling exports) had zero runtime consumers anywhere — it was exported "for backward compat until removed" and only its own tests imported it. - The aisdk_responses transport was selectable via CODEX_PROVIDER but selected nowhere; it delegated SSE parsing to upstream @ai-sdk/openai, whose usage schema drops GPT-5.6's cache_write_tokens — removing it removes this branch's one known token-accounting gap instead of mitigating it. CodexProviderMode narrows to 'sse' | 'websockets'. A daemon still carrying CODEX_PROVIDER=aisdk_responses (documented env escape hatch on a public package) degrades to the default sse transport with a warn rather than crashing. The custom-deployment override path (custom-openai-responses) is untouched — it only shared the diagnostics label. Tests: legacy-only suites deleted (their live-transport coverage already exists in codex-sse-provider.test.ts, including the parametrized cache_write_tokens fixtures); normalizeCodexServiceTier tests salvaged into service-tier.test.ts; agent.test.ts updated for two transports plus a new retired-mode fallback test. READMEs and the docs page now show the two shipping transports. Removing public exports is breaking for the published package — release should bump accordingly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PepijnSenders
changed the title
Ten findings from the fresh-context review of this branch; all applied:
- snapshot() publishes the RECONCILED noCacheInputTokens in byModel and
totals (the figure costing actually used), never the raw report — a
pathological value could exceed inputTokens and contradict the billed
cost for any consumer deriving cached = input − noCache.
- A provider noCache below the prompt total with NO cache counters is no
longer trusted: pricing it would bill the cached remainder at $0. It
falls back to derivation (full input rate, matching pre-noCache
behavior). Trust requires cache counters to price the remainder, or a
figure covering the whole prompt.
- extractUsage treats a NEGATIVE noCacheTokens as absent instead of
clamping to 0 — clamped garbage read as "zero uncached tokens,
provider-vouched" and billed the whole prompt at $0.
- The undefined-poisoning sum rule is one exported sumOrPoison() helper
instead of three hand-written copies (add(), snapshot() totals,
performCompaction).
- The copilot Responses adapter derives noCache through one
deriveNoCacheTokens() used by doGenerate and doStream, gated on cache
COUNTER presence rather than details-object presence — details: {}
no longer fabricates noCache = input_tokens as fact — and the
hasInputTokenDetails flag is gone.
- resolveCodexAuth's expired-oauth refresh-and-persist path (used by
both live transports on every request) regained direct coverage after
its only test died with the legacy suite: codex-auth.test.ts.
- CODEX_PROVIDER= (empty string from an env template) is treated as
unset instead of warning on every model resolution.
- Dead code from the removal: the @ai-sdk/openai dependency (zero
remaining imports — the very SDK whose schema drops
cache_write_tokens), the orphaned wrapSSE watchdog, and
DEFAULT_CHUNK_TIMEOUT_MS. The README no longer documents options of
the removed provider.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PepijnSenders
marked this pull request as ready for review
August 14, 2026 13:36
The "known gap" this branch documented — upstream @ai-sdk/openai dropping GPT-5.6's cache_write_tokens — was fixed upstream within our 3.x line (present in 3.0.96, absent in 3.0.80; we pinned 3.0.69). No issue or PR to vercel/ai needed. This closes the last consumer of the broken schema: the custom-deployment override path (custom-openai-responses), which delegates to createOpenAI().responses(). Its captureResponseUsage helper needed one adaptation — read-once-and-rebuild instead of clone(): the new SDK reads the body in a way that races the clone's tee under Bun, surfacing as "JSON Parse error: Unexpected EOF". The aisdk_responses transport removal earlier in this branch stands on its own grounds (zero users); this bump just means no shipping path is left anywhere that drops cache writes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
The providers.ts fallback comment and the README said upstream 'drops cache_write_tokens' in the present tense; 3.0.96 (bumped in this branch) parses it. Both now scope the claim to the version that held at removal time. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
K-Mistele
reviewed
Aug 14, 2026
Comment on lines
+81
to
+85
| // Read once and rebuild rather than clone(): the SDK (>= @ai-sdk/openai | ||
| // 3.0.96) reads the body in a way that races the clone's tee under Bun, | ||
| // surfacing as "JSON Parse error: Unexpected EOF" from a half-drained | ||
| // stream. A rebuilt Response hands it a fresh, fully-buffered body. | ||
| const text = await response.text() |
Contributor
There was a problem hiding this comment.
ai SDK is not used this doesn't make sense
The removed transport's history lives in git and the PR; the code only needs to say why unknown values degrade instead of throwing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-gated trust, retire the label Independent full-PR review; the blocking finding plus two should-fixes: - snapshot() published RAW cacheRead/cacheWrite in byModel and totals next to the RECONCILED noCacheInputTokens, so the exported breakdown didn't partition the prompt and couldn't re-derive the billed cost (the branch's own test fixture published 1.05M category-tokens for a 1M prompt while billing from 750k). Both now publish the clamped values costing used; the test asserts the counters, not just the cost. - The noCache trust rule gated on cache-counter PRESENCE, so one tiny counter unlocked trust and the uncovered remainder priced at $0 (100-token prompt, read 10, noCache 20 -> 70 tokens free). Trust now requires COVERAGE: reported categories must account for the whole prompt, else derivation. - The custom-deployment diagnostics still labeled its records 'aisdk_responses' — the name of a transport this branch deletes. Renamed to 'custom_responses' in the emit and the CodexDiagnosticTransport union. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
Hardens codex/codelayer token accounting and removes two dead codex transports. Breaking for TWO published packages —
@humanlayer/agentlayer-provider-openai-codexloses 12 exports, and@humanlayer/codelayerre-exportsResolveModelContext, whosecodexProviderModeunion narrowed from 3 to 2 members. Release should bump both accordingly — suggested 0.1.0 (versioning is a manual--versionarg, so this note is the record). The downstream catalog bump is prepared and documented in humanlayer/synclayer#2127, which merges independently and flips to this release once the version exists.State of the world first, so the scope is unambiguous:
cache_write_tokenscorrectly — nothing about the live codex path was broken here.cache_creation_tokens = 0on every row: the private ChatGPT codex backend didn't send the field for our traffic (matches earlier internal findings that some usage fields are absent/different on the private Responses API). openai/codex#32479 shows the backend now emits it to Codex CLI for GPT-5.6 — billed at 1.25× the input rate — so it's likely rolling out, and when it arrives we already capture it.1.
cache_write_tokensplumbed through every remaining path that dropped itderiveNoCacheTokens()gated on cache-counter presence (details: {}no longer fabricates a figure). One GPT-5.6 response with{ input_tokens: 100, cached_tokens: 60, cache_write_tokens: 15 }maps to{ noCache: 25, cacheRead: 60, cacheWrite: 15 }and costs exactly, instead of{ noCache: 40, cacheWrite: undefined }at −7.5%.CODELAYER_CODEX_BASE_URL→createOpenAI().responses()): fixed by bumping@ai-sdk/openai3.0.69 → 3.0.96 — upstream fixed its usage schema within our 3.x line (present in 3.0.96, absent in 3.0.80), so no vercel/ai issue was needed.captureResponseUsageneeded a read-once-and-rebuild adaptation (the new SDK's body reads raceclone()'s tee under Bun).2. The accumulator trusts the provider's own uncached count — carefully
The usage shape the accumulator ingests carries
inputTokenDetails.noCacheTokens;extractUsageused to drop it andsnapshot()re-derived uncached by subtraction. Now: the provider figure flows through with poisoning semantics (one call without it →undefined, never a misleading partial sum — one sharedsumOrPoison()), andsnapshot()prefers it for costing under a trust rule: only when cache counters exist to price the remainder, or the figure covers the whole prompt (a bare partial figure would price the cached remainder at $0). Priced categories are reconciled to partition the prompt total, the publishedbyModel/totalsexpose the reconciled figure (never a raw pathological one), negative reports are treated as absent, andperformCompactioncarries the field with the same rule.3. Two dead codex transports removed
Verified unused before removal — repo greps across agentlayer and synclayer, GitHub code search across the whole org (6 symbols, every hit inside this repo), and deployment env/infra:
legacy.ts(createCodexProvider+ 9 sibling exports, ~1,400 lines): zero runtime consumers anywhere; exported "for backward compat until removed"; only its own tests imported it.aisdk_responses: selectable viaCODEX_PROVIDERbut selected nowhere.CodexProviderModenarrows to'sse' | 'websockets'. A daemon still carryingCODEX_PROVIDER=aisdk_responses(it was a documented env escape hatch) degrades to the defaultssetransport with a warn rather than crashing — covered by a new fallback test. The custom-deployment override path (custom-openai-responses) is kept; later commits renamed its diagnostics label (aisdk_responses->custom_responses, the retired transport name) and adapted its usage capture for the@ai-sdk/openaibump described above. Legacy-only test suites are deleted; their live-transport coverage already exists incodex-sse-provider.test.ts(including parametrizedcache_write_tokensfixtures),resolveCodexAuth's expired-token refresh gained direct coverage (codex-auth.test.ts),normalizeCodexServiceTiertests are salvaged intoservice-tier.test.ts, and both READMEs + the docs page now show the two shipping transports.Downstream consumer: humanlayer/synclayer#2127 stores these fields; once this ships, stored tokens keep re-deriving the persisted cost on every path.
How was this validated?
bun check: 1,247 tests across 111 files, 0 failcache_write_tokensend-to-end fixtures, accumulator trust-rule / partitioning / poisoning / pathological-figure cases, expired-oauth refresh-and-persist, and the retired-transport fallbackagentlayer-corelinked in: wire-truth token counts round-trip exactly through accumulator → daemon → storage, and stored tokens × catalog prices re-derive the persisted cost to 6 decimalsChecklist
🤖 Generated with Claude Code