feat: LLM usage and cost accounting, with an in-app pricing editor

Every model call now writes one `llm_usage` row (V119), surfaced at
Settings -> AI Usage & Cost: spend by feature, user and model, a daily
trend, and an editable per-model rate table.

Capture sits at the two provider funnels -- RefreshableChatModel and
EmbeddingService -- rather than in feature code, so a new caller is
accounted for without touching it. LlmUsageRecorder swallows its own
failures by design: accounting must never break the call it measures.
Rows are written through LlmUsageWriter in their own transaction,
because self-invoking a @transactional method bypasses the Spring proxy
and the row would roll back with a failed chat turn.

Rates live in system_config with no bundled defaults -- a stale price
list produces confident wrong totals nobody checks. An unpriced model
stores a NULL cost, never 0, and the UI reports the gap instead of
silently understating spend.

Two pre-existing bugs surfaced while wiring this up:

- ResponsesApiChatModel.buildMetadata discarded the provider's `usage`
  block entirely. Harmless while nothing read it, a silent zero the
  moment accounting summed it: real billed calls recorded 0 tokens and
  $0.00. It now reads both vendor dialects, and the streaming path
  requests usage (stream_options.include_usage) and emits the late
  usage event so a streamed turn is billable at all.

- Attribution cannot ride a ThreadLocal. Chat returns a Flux and does
  its model work later on a CompletableFuture, so the servlet is gone
  by then; every row read feature=unknown while a single-threaded unit
  test of the filter passed. ChatService now re-establishes the scope
  where it already re-establishes the SQL actor.

Verified against the running stack, not inferred: real Azure OpenAI
calls at three configured rates, costs matching to six decimals with no
restart; a DEVELOPER 403'd on all endpoints with the rejected write
leaving rates untouched; and chat still answering with the llm_usage
table deleted outright.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: notSumit25

CLAUDE.md

@@ -311,6 +311,82 @@ embedding model and read through another raises no error — retrieval just degr
 (pgvector's `text`-column fallback has no dimension constraint and cosine similarity still
 returns a number).
 
+## LLM Usage & Cost Accounting
+
+Every model call writes one `llm_usage` row (`V119__create_llm_usage.sql`), surfaced at
+**Settings → AI Usage & Cost** (`LlmUsageTab.jsx`) via `GET /admin/llm-usage/summary`.
+Admin-only and not connection-scoped, so it carries a class-level `@PreAuthorize` rather
+than an `assertCan*` call — the case `Endpoint Authorization Rules` describes as
+"an endpoint with no connection scope at all is admin-only".
+
+- **Recording happens at the two provider funnels, never in feature code.**
+  `RefreshableChatModel.call/stream` and `EmbeddingService.createEmbedding(s)` are the only
+  call sites, so a thirteenth service that reaches a model is accounted for without
+  touching it. `LlmUsageRecorder` swallows its own failures by design: accounting must
+  never be able to break the call it measures.
+- **Rows are written in their own transaction** (`LlmUsageWriter`, `REQUIRES_NEW`). It is a
+  separate bean on purpose — self-invoking a `@Transactional` method bypasses the Spring
+  proxy, and the row would then roll back with a failed chat turn, which is exactly the
+  case it exists to record. Same trap `McpTokenRepository.deleteByUserId` documents.
+- **`estimated` distinguishes metered from derived counts.** Chat providers return real
+  token counts; `LlmEmbeddingProvider` returns vectors only, so embedding tokens are
+  derived from input length at 3 chars/token (schema text is denser than prose — see
+  `EmbeddingService.truncate`). Both land in the same columns so one spend total is
+  possible, but a vendor-invoice reconciliation can tell the halves apart.
+- **An unpriced model stores `NULL` cost, never `0`.** `LlmPricingService` reads rates from
+  `system_config` (`llm.pricing.<model>.{input,output,cached-input}-per-1m`) and ships **no
+  default prices** — a stale bundled price list produces confident wrong totals nobody
+  thinks to check. The UI reports unpriced calls rather than silently understating spend.
+- **Rates are edited in the UI**, in the Model pricing panel of the same tab
+  (`LlmPricingPanel.jsx` → `GET|PUT /admin/llm-usage/pricing`). The list is every model the
+  ledger has seen plus every model with a rate configured, unpriced first. Writes take
+  effect on the next call with no restart, since `LlmPricingService` reads `system_config`
+  per call. Three details are load-bearing:
+  - **A cleared field writes `""`, not a deleted row.** `rate()` already treats blank as
+    absent, and `SystemConfigService` has no delete — adding one for a single caller would
+    widen a shared service. Sending the whole set means an emptied box genuinely clears
+    that rate rather than leaving the old value behind.
+  - **A model name can contain dots** (`gpt-5.4`), so `configuredModels()` strips the known
+    suffix from the *end* of the key. Splitting on the first `.` after the prefix reports
+    `gpt-5` and loses the row. The controller mapping is `{model:.+}` for the same reason.
+  - **A name containing a slash cannot go in the path at all**, even percent-encoded:
+    Spring Security's default `StrictHttpFirewall` rejects `%2F` with a bare 400 before any
+    controller runs — verified in QA, and it applies to every endpoint, not just this one.
+    Self-hosted ids look like `meta-llama/Llama-3-8b`, so `PUT /pricing` (no path segment)
+    takes the name in the body, and `client.js` switches to it when the name contains `/`.
+    Relaxing the firewall would be the wrong trade for a naming convenience.
+  - **A failed save must still say so.** Spring's default 500 body carries no `message`, so
+    the client had nothing to display and a save against a broken config store showed the
+    user *nothing at all* — no error, no toast, silent. The handler now returns a `message`
+    on any non-`IllegalArgumentException` failure, and the panel catches the `mutateAsync`
+    rejection rather than letting it escape the click handler (an uncaught rejection there
+    is what stopped the `isError` banner from rendering). Both found by taking
+    `system_config` away mid-save.
+  - **A negative rate is rejected at write time**, not just ignored on read: a value that
+    silently does nothing after the UI said "Saved" is worse than an error at entry.
+- **Editing a rate does not re-cost recorded calls.** `estimated_cost_usd` is a snapshot
+  written at record time, which is what an audit trail wants — but it means a mid-window
+  price change shows as a step in the daily chart, not a uniform restatement. The panel
+  says so.
+- **Attribution cannot ride a ThreadLocal alone.** `LlmUsageAttributionFilter` labels each
+  request from its URI, but chat returns a `Flux` and does its model work later on a
+  `CompletableFuture` — the servlet, and any thread-local set during it, is gone by then.
+  `ChatService` re-establishes `LlmUsageContext.with(...)` inside `runAsync` exactly where
+  it already re-establishes `QueryActorContextHolder.withActor`; `DashboardAlertService`
+  declares its own scope since scheduled work has no request at all. Verified live: every
+  row read `feature = unknown` until this was added, while a single-threaded unit test of
+  the filter passed.
+- **`ResponsesApiChatModel.buildMetadata` used to discard `usage` entirely**, which cost
+  nothing while nothing read it and became a silent zero the moment accounting summed it —
+  real billed calls recorded 0 tokens and $0.00. It now reads both dialects
+  (`prompt_tokens`/`completion_tokens` and `input_tokens`/`output_tokens`) plus cached-token
+  details, and the streaming path sends `stream_options.include_usage` and emits the late
+  usage event as a text-free chunk. `RefreshableChatModel.meteredStream` records **one row
+  per stream**, taking the last reported usage rather than summing chunks: providers that
+  report running totals would otherwise have every partial added to the final figure.
+- Usage belongs to `QueryActorContextHolder` first and the security principal second, so
+  under **View as** the spend is attributed to the target user, not the admin.
+
 ## Key Rules & Patterns
 
 ### Backend Rules
@@ -853,6 +929,10 @@ DEEPSQL_EMBEDDING_API_KEY=<key>
 DEEPSQL_EMBEDDING_MODEL=text-embedding-3-large
 # Optional chat tuning: DEEPSQL_CHAT_TEMPERATURE, DEEPSQL_CHAT_API_VERSION,
 # DEEPSQL_CHAT_USE_RESPONSES_API (true|false|auto).
+# Model prices are NOT environment variables. They live in system_config and are edited in
+# Settings -> AI Usage & Cost -> Model pricing; there are deliberately no defaults:
+#   llm.pricing.<model>.input-per-1m / .output-per-1m / .cached-input-per-1m
+# An unpriced model still records tokens; its cost is NULL and the UI flags it.
 
 # Only if using Azure AI Search instead of pgvector for the vector store.
 azure.search.api-key=<key>

backend/src/main/java/com/dbaagent/config/LlmConfig.java

@@ -4,6 +4,7 @@
 import com.dbaagent.llm.LlmProviderRegistry;
 import com.dbaagent.llm.spring.ProviderBackedEmbeddingModel;
 import com.dbaagent.service.EmbeddingService;
+import com.dbaagent.service.llm.LlmUsageRecorder;
 import org.springframework.ai.chat.model.ChatModel;
 import org.springframework.ai.embedding.EmbeddingModel;
 import org.springframework.context.annotation.Bean;
@@ -24,8 +25,9 @@ public class LlmConfig {
 
     @Bean
     @Primary
-    public ChatModel chatModel(LlmConfigResolver resolver, LlmProviderRegistry registry) {
-        return new RefreshableChatModel(resolver, registry);
+    public ChatModel chatModel(LlmConfigResolver resolver, LlmProviderRegistry registry,
+                               LlmUsageRecorder usageRecorder) {
+        return new RefreshableChatModel(resolver, registry, usageRecorder);
     }
 
     /**

backend/src/main/java/com/dbaagent/config/RefreshableChatModel.java

@@ -7,15 +7,20 @@
 import com.dbaagent.llm.api.LlmErrorCategory;
 import com.dbaagent.llm.api.LlmNotConfiguredException;
 import com.dbaagent.llm.api.UnsupportedLlmProviderException;
+import com.dbaagent.model.LlmUsageRole;
+import com.dbaagent.service.llm.LlmUsageRecorder;
 import lombok.extern.slf4j.Slf4j;
 import org.springframework.ai.chat.model.ChatModel;
 import org.springframework.ai.chat.model.ChatResponse;
 import org.springframework.ai.chat.prompt.ChatOptions;
 import org.springframework.ai.chat.prompt.Prompt;
+import org.springframework.ai.chat.metadata.Usage;
 import reactor.core.publisher.Flux;
 
 import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
 import java.util.concurrent.locks.ReentrantLock;
+import java.util.function.Supplier;
 
 /**
  * A {@link ChatModel} that resolves its credentials on every call through
@@ -52,28 +57,62 @@ private record CachedDelegate(LlmCredentials key, ChatModel model) {}
     private final LlmConfigResolver resolver;
     private final LlmProviderRegistry registry;
 
+    /**
+     * Nullable so the accounting dependency stays optional. This class is constructed
+     * directly in tests and in {@code LlmConfig}; a hard dependency would make every
+     * existing construction site a compile error for a concern none of them care about.
+     */
+    private final LlmUsageRecorder usageRecorder;
+
     /** Serialises rebuilds only; the read path is a single lock-free volatile read. */
     private final ReentrantLock buildLock = new ReentrantLock();
     private volatile CachedDelegate cached;
 
     public RefreshableChatModel(LlmConfigResolver resolver, LlmProviderRegistry registry) {
+        this(resolver, registry, null);
+    }
+
+    public RefreshableChatModel(LlmConfigResolver resolver, LlmProviderRegistry registry,
+                                LlmUsageRecorder usageRecorder) {
         this.resolver = resolver;
         this.registry = registry;
+        this.usageRecorder = usageRecorder;
     }
 
     @Override
     public ChatResponse call(Prompt prompt) {
         CachedDelegate active = resolveDelegate();
         try {
-            return active.model().call(prompt);
+            return metered(active, () -> active.model().call(prompt));
         } catch (RuntimeException e) {
             if (shouldRetryWithEnvFallback(e, active.key())) {
-                return resolveDelegate().model().call(prompt);
+                CachedDelegate retry = resolveDelegate();
+                return metered(retry, () -> retry.model().call(prompt));
             }
             throw e;
         }
     }
 
+    /**
+     * Runs a chat call and records what it cost.
+     *
+     * <p>A failed call is recorded too. Providers bill for prompt tokens on responses that
+     * error partway, and an operator investigating a spend spike caused by a retry loop
+     * needs to see the failures — a ledger holding only successes hides exactly the
+     * pathology it would be consulted about.
+     */
+    private ChatResponse metered(CachedDelegate active, Supplier<ChatResponse> call) {
+        long startedAt = System.nanoTime();
+        try {
+            ChatResponse response = call.get();
+            recordUsage(active, response, startedAt, null);
+            return response;
+        } catch (RuntimeException e) {
+            recordUsage(active, null, startedAt, e);
+            throw e;
+        }
+    }
+
     /**
      * Streaming delegates report failures asynchronously through the sink rather than by
      * throwing, so the fallback hangs off the error signal as well as off a synchronous
@@ -87,15 +126,65 @@ public Flux<ChatResponse> stream(Prompt prompt) {
             CachedDelegate active = resolveDelegate();
             AtomicBoolean emitted = new AtomicBoolean(false);
             try {
-                return active.model().stream(prompt)
+                return meteredStream(active, active.model().stream(prompt)
                         .doOnNext(chunk -> emitted.set(true))
-                        .onErrorResume(e -> resumeStream(prompt, e, active.key(), emitted.get()));
+                        .onErrorResume(e -> resumeStream(prompt, e, active.key(), emitted.get())));
             } catch (RuntimeException e) {
-                return resumeStream(prompt, e, active.key(), false);
+                return meteredStream(active, resumeStream(prompt, e, active.key(), false));
             }
         });
     }
 
+    /**
+     * Records one row for a whole stream, not one per chunk.
+     *
+     * <p>Usage on a stream arrives on a single late chunk — typically the last, after the
+     * provider has finished counting — while every earlier chunk carries either no
+     * metadata or a zero-filled {@link Usage}. Recording per chunk would write hundreds of
+     * rows for one call and inflate the call count enormously; summing across chunks would
+     * be worse, because providers that report cumulative running totals would have their
+     * final figure added on top of every partial. So the last non-zero usage seen wins, and
+     * exactly one row is written when the stream terminates.
+     *
+     * <p>{@code doFinally} rather than {@code doOnComplete}: a stream that errors or is
+     * cancelled partway still consumed prompt tokens, and a cancelled dashboard build is
+     * precisely the kind of silent spend an operator wants on the ledger.
+     */
+    private Flux<ChatResponse> meteredStream(CachedDelegate active, Flux<ChatResponse> source) {
+        if (usageRecorder == null) {
+            return source;
+        }
+        long startedAt = System.nanoTime();
+        AtomicReference<ChatResponse> lastWithUsage = new AtomicReference<>();
+        AtomicReference<Throwable> failure = new AtomicReference<>();
+        AtomicBoolean recorded = new AtomicBoolean(false);
+
+        return source
+                .doOnNext(chunk -> {
+                    if (hasUsage(chunk)) {
+                        lastWithUsage.set(chunk);
+                    }
+                })
+                .doOnError(failure::set)
+                .doFinally(signal -> {
+                    // doFinally can fire once per subscription; a Flux that is retried or
+                    // resubscribed must not double-bill the same logical call.
+                    if (recorded.compareAndSet(false, true)) {
+                        Throwable error = failure.get();
+                        recordUsage(active, lastWithUsage.get(), startedAt,
+                                error instanceof RuntimeException re ? re : null);
+                    }
+                });
+    }
+
+    private static boolean hasUsage(ChatResponse chunk) {
+        if (chunk == null || chunk.getMetadata() == null) {
+            return false;
+        }
+        Usage usage = chunk.getMetadata().getUsage();
+        return usage != null && zeroIfNull(usage.getTotalTokens()) > 0;
+    }
+
     /**
      * Reports the active delegate's options, or neutral ones when nothing is configured.
      *
@@ -128,6 +217,75 @@ public ChatOptions getDefaultOptions() {
         }
     }
 
+    /**
+     * Records one chat call, taking token counts from the response the provider returned.
+     *
+     * <p>The model name comes from the response metadata when the provider reports it and
+     * falls back to the configured model. Those can legitimately differ — an alias that
+     * resolves to a dated snapshot, for instance — and the served model is the one that
+     * was actually billed, so it wins.
+     *
+     * <p>Never throws. It runs inside the call path of every chat turn in the product;
+     * a defect here must not become a failed conversation.
+     */
+    private void recordUsage(CachedDelegate active, ChatResponse response,
+                             long startedAt, RuntimeException failure) {
+        if (usageRecorder == null) {
+            return;
+        }
+        try {
+            long latencyMs = (System.nanoTime() - startedAt) / 1_000_000L;
+            LlmCredentials key = active.key();
+
+            long prompt = 0;
+            long completion = 0;
+            long total = 0;
+            long cached = 0;
+            String model = key.getOrDefault("model", "unknown");
+
+            if (response != null && response.getMetadata() != null) {
+                Usage usage = response.getMetadata().getUsage();
+                if (usage != null) {
+                    prompt = zeroIfNull(usage.getPromptTokens());
+                    completion = zeroIfNull(usage.getCompletionTokens());
+                    total = zeroIfNull(usage.getTotalTokens());
+                    cached = zeroIfNullLong(usage.getCacheReadInputTokens());
+                }
+                String served = response.getMetadata().getModel();
+                if (served != null && !served.isBlank()) {
+                    model = served;
+                }
+            }
+
+            String errorCategory = failure == null ? null
+                    : String.valueOf(registry.chatProvider(key.providerId()).classify(failure));
+
+            usageRecorder.record(new LlmUsageRecorder.Call(
+                    LlmUsageRole.CHAT,
+                    key.providerId(),
+                    model,
+                    prompt,
+                    completion,
+                    total,
+                    cached,
+                    false,
+                    latencyMs,
+                    failure == null,
+                    errorCategory));
+        } catch (RuntimeException e) {
+            log.warn("RefreshableChatModel: could not record usage; the call was unaffected", e);
+        }
+    }
+
+    private static long zeroIfNull(Integer value) {
+        return value == null ? 0L : value.longValue();
+    }
+
+    /** Cache token accessors are {@code Long} in Spring AI 2.0, unlike prompt/completion. */
+    private static long zeroIfNullLong(Long value) {
+        return value == null ? 0L : value;
+    }
+
     // ── Internal ──────────────────────────────────────────────────────────────
 
     private CachedDelegate resolveDelegate() {