Skip to content

fix(sdk): take max of OpenRouter cost and upstream_inference_cost - #1175

Open
nordicnode wants to merge 1 commit into
CodebuffAI:mainfrom
nordicnode:oss/credits-upstream-max-1164
Open

fix(sdk): take max of OpenRouter cost and upstream_inference_cost#1175
nordicnode wants to merge 1 commit into
CodebuffAI:mainfrom
nordicnode:oss/credits-upstream-max-1164

Conversation

@nordicnode

Copy link
Copy Markdown

Summary

  • user-facing credits were computed as usage.cost + cost_details.upstream_inference_cost at all three cost sites (sdk/src/impl/llm.ts streaming reportCost, generateText, and structured paths)
  • usage.cost is the total amount charged and already includes the upstream portion reported separately in cost_details, per OpenRouter's usage accounting docs — summing them roughly doubles credits on normal (non-BYOK) OpenRouter routes, which lines up with the "extreme inflation" reports on Discord
  • take the max instead, matching the server-side ledger's documented semantics (the Solar Pro 4 BYOK note in common/src/constants/freebuff-models.ts: cost = 0 there and upstream carries the real spend, so max is correct for both shapes)
  • add a regression test asserting exact credits for cost=0.01, upstream=0.02 (fails under the sum: 3 vs 2)

Fixes #1164

Validation

Re-run on the new main after the history rewrite:

  • bun test sdk/src/impl/__tests__/usage-receipts.test.ts — 5 pass, 0 fail
  • bunx tsc --noEmit -p sdk/tsconfig.json clean

Replaces #1165 — auto-closed when the repo history was force-pushed (the original commits no longer exist in the new history). Rebased onto the new main per the maintainer's request; picking the review up here.

usage.cost is the total amount charged and already contains the upstream
portion reported separately in cost_details.upstream_inference_cost, so
summing the two roughly doubles the credits on normal (non-BYOK)
OpenRouter routes — the shape behind the inflated-usage reports. The
ledger side already takes the max (see the Solar Pro 4 BYOK note in
common/src/constants/freebuff-models.ts, where cost is 0 and upstream
carries the real spend); this fixes the user-facing credit path the
same way at all three sites (stream, generateText, structured).

Fixes CodebuffAI#1164
@codebuff-team

Copy link
Copy Markdown
Contributor

Good instinct and scoped correctly — sdk/src/impl/llm.ts is fair game, and you touched all three cost sites (streaming reportCost, generateText, structured) consistently rather than fixing only one. The added regression test in usage-receipts.test.ts is a real behavioral test (drives an SSE stream through promptAiSdkStream and asserts the emitted credit amount), not just a unit check on a helper, which is the right level for this kind of bug.

One thing worth flagging for the maintainer to weigh: Math.max(cost, upstream) is a heuristic that happens to work for the two cases you cite (normal OpenRouter routes where cost already includes everything and upstream_inference_cost is 0/absent, and BYOK routes where cost is 0 and upstream_inference_cost carries the real spend) but it silently drops any genuine OpenRouter markup fee in a BYOK request where both cost and upstream_inference_cost are simultaneously nonzero — i.e. if OpenRouter ever charges a small fee on top of the upstream cost for BYOK, max will undercount vs the true sum. That's a much smaller and safer failure mode than the current doubling bug, but it's worth the maintainer confirming against OpenRouter's actual accounting semantics for BYOK-with-fee before this ships, rather than assuming max is universally correct.

Overall: focused, in-scope, has a test, addresses a real reported issue (#1164) with a concrete before/after fixture. Worth porting, with the max-vs-sum edge case called out during review.

@codebuff-team codebuff-team added bot:triaged Classified by the community triage bot pr:port-candidate Worth porting into the private source tree labels Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:triaged Classified by the community triage bot pr:port-candidate Worth porting into the private source tree

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SDK: credits double-count OpenRouter's upstream_inference_cost — usage.cost already includes it

2 participants