Record the cost of sessions that use a custom Responses endpoint

[PepijnSenders]

The problem

A custom Responses endpoint (Azure AI Foundry, or OpenAI direct) builds its model with the provider name custom-openai-responses. The pricing catalog has no such provider, so getModelInfo returns early and every lookup misses.

A miss is reported as undefined, not an error. Hosts drop the cost when it is undefined. So these sessions record token counts and no dollars at all.

The failure is silent, and it flatters the numbers: the work still counts in cost-per-unit denominators while adding nothing to the spend. Cost is frozen at ingest, so the loss is permanent.

getModelInfo maps the provider half of the model key onto a catalog provider, then looks the model up under it.

flowchart TD
    K["model key<br/>custom-openai-responses / gpt-5.6-sol"] --> D{"map provider<br/>to a catalog name"}
    D -->|"codex* → openai<br/>(already there)"| O["openai"]
    D -->|"custom-openai-responses → openai<br/>(this PR)"| O
    D -->|"anthropic, openai, … → itself"| S["that same provider"]
    D -.->|"before this PR:<br/>no rule matched it"| M["not a catalog provider<br/>undefined → cost dropped"]
    O --> P["price found<br/>$5 / $30 per Mtok"]
    S --> P

    classDef fixed fill:#1a7f37,stroke:#0b4a1f,color:#fff
    classDef broken fill:#b35900,stroke:#7a3d00,color:#fff
    class O,P fixed
    class M broken

Loading

The fix

The models behind the endpoint are OpenAI catalog models. CODELAYER_CODEX_MODEL renames the model only on the wire, so modelId is still the selected catalog id. Map the provider to openai, the same way Codex providers already map.

Verified against models.json: openai holds all four GPT-5.6 entries with prices (gpt-5.6-sol 5/30, -terra 2.5/15, -luna 1/6). The azure provider has 109 models and zero GPT-5.6 entries, so it is the wrong target.

Also exports the provider name and uses it in CodeLayer, which hardcoded the string in 4 places. A rename on either side used to break pricing with no signal.

One intended side effect

Limits share the same lookup. These sessions now resolve a context window (1,050,000) where they had undefined, so auto-compaction applies to them for the first time. Previously there was no limit-based compaction at all.

The endpoint serves the public Responses API, not the private Codex one, so it stays out of the Codex context-window override.

Checks

• bun test packages/agentlayer-core/test/ — 504 pass, 8 skip
• bun test agents/codelayer/test/providers.test.ts agents/codelayer/test/agent.test.ts — 108 pass
• bun run biome:check — clean
• Pre-existing and untouched: agents/codelayer/src/cli.ts fails typecheck on a zod/compile side-effect import.

Note

This does not repair data already recorded. Cost is frozen at ingest, so existing rows stay empty. The token counts are correct and stored, so a backfill is possible as separate work.

🤖 Generated with Claude Code

[github-actions]

Docs Agent Review

Agent finished with reason: error

---

To apply these recommendations, comment: @docs-agent apply

[PepijnSenders]

Local end-to-end check

Ran the whole path against a fake Azure AI Foundry deployment. Same script both times. Only the new clause in getModelInfo differs.

	pricing found	estimatedCostUsd
before	no — cost dropped	undefined
after	yes	6.2

Token counts were identical in both runs: 600,000 uncached, 400,000 cache read, 100,000 output. That is the failure mode exactly — the tokens arrive, the dollars do not.

$6.20 is exact against list pricing, not an approximation:
600k × $5 + 100k × $30 + 400k × $0.50 per Mtok = 3.00 + 3.00 + 0.20

What it exercises

Only the LLM is fake. Everything else is the real code path:

readCodexResponsesOverride() → createCustomCodexResponsesModel() → generateText() over real HTTP to a real Bun.serve on loopback → getModelKey() → ModelProvider.getModelPricing() → TokenUsageAccumulator.

The request the fake deployment received confirms the Azure shape:

path              /openai/v1/responses
api-key header    fake-azure-key
authorization     (absent, as Azure needs)
model on the wire azure-coding-deployment

The booked key stayed custom-openai-responses/gpt-5.6-sol. That split — deployment name on the wire, catalog id in the key — is what the bug turned on.

Two notes

parseCodexResponsesURL permits plain http on a loopback host. That is what makes a local stub possible with no Azure account.

Price the SDK's normalized usage, not the raw provider result. A first attempt called model.doGenerate() directly and produced NaN, because that returns structured token objects. The agent loop prices generateText output, so the check has to go through it.

Script

// Fake Azure AI Foundry deployment
const server = Bun.serve({
  hostname: '127.0.0.1',
  port: 0,
  async fetch(request) {
    seenWireModel = ((await request.json()) as { model?: string }).model ?? ''
    return Response.json({
      id: 'resp_fake', object: 'response', created_at: 1,
      model: seenWireModel, status: 'completed',
      output: [{ type: 'message', id: 'msg_fake', role: 'assistant', status: 'completed',
                 content: [{ type: 'output_text', text: 'ok', annotations: [] }] }],
      parallel_tool_calls: true, tool_choice: 'auto', tools: [],
      usage: {
        input_tokens: 1_000_000,                              // cache-inclusive
        input_tokens_details: { cached_tokens: 400_000 },
        output_tokens: 100_000,
        output_tokens_details: { reasoning_tokens: 0 },
        total_tokens: 1_100_000,
      },
    })
  },
})

// Configured exactly as the docs tell a user to
process.env.CODELAYER_CODEX_BASE_URL = `http://127.0.0.1:${server.port}/openai/v1`
process.env.CODELAYER_CODEX_API_KEY = 'fake-azure-key'
process.env.CODELAYER_CODEX_API_KEY_HEADER = 'api-key'
process.env.CODELAYER_CODEX_MODEL = 'azure-coding-deployment'

const override = readCodexResponsesOverride()!
const model = createCustomCodexResponsesModel({ override, selectedModelId: 'gpt-5.6-sol' })
const result = await generateText({ model, prompt: 'hello' })

const provider = new ModelProvider()
const modelKey = getModelKey(model)
const accumulator = new TokenUsageAccumulator((key: string) => provider.getModelPricing(key))
accumulator.add(modelKey, extractUsage(result.usage))
console.log(accumulator.snapshot().byModel[modelKey]!.estimatedCostUsd)

Happy to commit this as a test in agents/codelayer/test/ if reviewers want the regression held permanently.