feat(ai): route text generation to Groq Cloud gpt-oss-20b with Gemini fallback and usage tracking

.env.example

@@ -75,6 +75,15 @@ AI_MOCK=false
 OPENAI_API_KEY=
 ANTHROPIC_API_KEY=
 
+# Text Generation provider (Groq Cloud openai/gpt-oss-20b first, Gemini fallback)
+GROQ_API_KEY=
+GROQ_TEXT_MODEL=
+GROQ_TIMEOUT_MS=
+GROQ_REASONING_EFFORT=
+GROQ_MAX_CONCURRENCY=
+TEXT_PROVIDER_FORCE=
+USD_INR=
+
 # DM Classification provider (clef | gemini; defaults to gemini)
 CLASSIFICATION_PROVIDER=
 CLOUDFLARE_ACCOUNT_ID=

README.md

@@ -173,6 +173,52 @@ Every human-readable string comes from one place, and **no raw stored value reac
 | `REDIRECT_URI` | Must match the Meta dashboard entry **exactly**, no trailing slash. |
 | `FRONTEND_URL` / `BACKEND_URL` | Also drive the CORS allowlist. |
 
+### Text Generation Providers (Groq Cloud + Gemini Fallback)
+
+Text generation (auto-reply, Draft AI reply, conversation summaries) runs on **Groq Cloud** (`openai/gpt-oss-20b`) as first-priority with automatic, zero-customer-error fallback to Gemini.
+
+#### Environment Variables
+
+| Variable | Default | Notes |
+|---|---|---|
+| `GROQ_API_KEY` | *(none)* | Groq API Key (Bearer authentication). |
+| `GROQ_TEXT_MODEL` | `openai/gpt-oss-20b` | Text generation model id on Groq. |
+| `GROQ_TIMEOUT_MS` | `15000` | Request timeout in milliseconds. |
+| `GROQ_REASONING_EFFORT` | `low` | Reasoning effort level for reasoning models. |
+| `GROQ_MAX_CONCURRENCY` | `10` | In-process concurrency limiter maximum. |
+| `TEXT_PROVIDER_FORCE` | *(none)* | Kill switch: set to `gemini` to force Gemini and bypass Groq. |
+| `USD_INR` | `84` | USD to INR conversion rate for admin dashboard display. |
+
+#### Provider Precedence & Fallback
+At runtime, text generation resolves the provider in this order:
+1. **Kill switch**: If `TEXT_PROVIDER_FORCE=gemini`, uses Gemini directly.
+2. **Admin runtime override**: Stored in `app_settings` (`text_generation_provider`) via the Super Admin Panel.
+3. **Default**: Groq Cloud first, automatic Gemini fallback on any error (missing key, 401/403, 429, 5xx, timeout, circuit breaker open, empty content).
+
+#### Super Admin Panel Controls (`/admin/ai`)
+- **Runtime Provider Controls**: Switch primary text generator between Groq and Gemini with audit reasoning.
+- **Groq Health Card**: Live status, key configured status, error rate %, circuit breaker state, remaining rate limit tokens and requests.
+- **Probe Test**: Instant live test probe button.
+- **AI Usage Tracking Dashboard**: Token aggregates (prompt, completion, reasoning, cached), cost in USD & INR, daily timeline chart, top customers table, call trace table with filters, and CSV export.
+
+#### Rollback
+If Groq experiences degraded service:
+- **Instant Admin Rollback**: In Super Admin (`/admin/ai`), select **Google Gemini** in the Text Generation Provider control and confirm with an audit reason. The switch applies immediately across all text-generation calls without requiring a service restart.
+- **Environment Kill Switch**: Set `TEXT_PROVIDER_FORCE=gemini` in environment variables.
+
+#### Production Migration Note
+The AI usage tracking migration `20261029100000_add_ai_usage_logs` is **purely additive** (creates table `ai_usage_logs` with indexes; no existing tables are altered or dropped).
+- Apply to production using the owner migration script:
+  ```powershell
+  cd backend
+  powershell -ExecutionPolicy Bypass -File .\scripts\prod-migrate.ps1
+  ```
+- Or execute inside the Northflank API service shell:
+  ```bash
+  npx prisma migrate deploy
+  ```
+- AI usage logging is fire-and-forget: during any gap before migration is applied, text-generation and customer replies continue working uninterrupted without errors.
+
 ### DM Classification Providers
 
 DM classification can run on **Cloudflare Workers AI Clef Flash** (`@cf/cloudflare/clef-flash`) with automatic, fail-open fallback to Gemini. Text generation, draft replies, and conversation summaries remain strictly on Gemini.

backend/prisma/migrations/20261029100000_add_ai_usage_logs/migration.sql

@@ -0,0 +1,50 @@
+-- Additive migration: comprehensive AI usage tracking log (ai_usage_logs).
+-- Metadata ONLY: never stores DM text or prompts.
+--
+-- REVERSE (manual):
+--   DROP TABLE "ai_usage_logs";
+
+-- CreateTable
+CREATE TABLE "ai_usage_logs" (
+    "id" TEXT NOT NULL,
+    "created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    "user_id" TEXT,
+    "tenant_id" TEXT,
+    "feature" TEXT NOT NULL,
+    "provider" TEXT NOT NULL,
+    "model" TEXT NOT NULL,
+    "attempt_no" INTEGER NOT NULL DEFAULT 1,
+    "is_fallback" BOOLEAN NOT NULL DEFAULT false,
+    "fallback_reason" TEXT,
+    "status" TEXT NOT NULL,
+    "http_status" INTEGER,
+    "latency_ms" INTEGER NOT NULL,
+    "prompt_tokens" INTEGER,
+    "completion_tokens" INTEGER,
+    "reasoning_tokens" INTEGER,
+    "cached_tokens" INTEGER,
+    "total_tokens" INTEGER,
+    "estimated_cost_usd" DOUBLE PRECISION,
+    "provider_request_id" TEXT,
+    "error_message" TEXT,
+    "conversation_id" TEXT,
+    "message_id" TEXT,
+
+    CONSTRAINT "ai_usage_logs_pkey" PRIMARY KEY ("id")
+);
+
+-- CreateIndex
+CREATE INDEX "ai_usage_logs_created_at_idx" ON "ai_usage_logs"("created_at");
+
+-- CreateIndex
+CREATE INDEX "ai_usage_logs_user_id_idx" ON "ai_usage_logs"("user_id");
+
+-- CreateIndex
+CREATE INDEX "ai_usage_logs_provider_idx" ON "ai_usage_logs"("provider");
+
+-- CreateIndex
+CREATE INDEX "ai_usage_logs_feature_idx" ON "ai_usage_logs"("feature");
+
+-- CreateIndex
+CREATE INDEX "ai_usage_logs_status_idx" ON "ai_usage_logs"("status");
+

backend/prisma/schema.prisma

@@ -691,6 +691,43 @@ model AiUsageDaily {
   @@map("ai_usage_daily")
 }
 
+// ============================================
+// Comprehensive AI usage tracking log (additive).
+// Metadata ONLY — never stores DM text or prompts.
+// ============================================
+model AiUsageLog {
+  id                String   @id @default(uuid())
+  createdAt         DateTime @default(now()) @map("created_at")
+  userId            String?  @map("user_id")
+  tenantId          String?  @map("tenant_id")
+  feature           String   // auto_reply | draft_reply | summary | classification | other
+  provider          String
+  model             String
+  attemptNo         Int      @default(1) @map("attempt_no")
+  isFallback        Boolean  @default(false) @map("is_fallback")
+  fallbackReason    String?  @map("fallback_reason")
+  status            String   // success | error | timeout | rate_limited | empty
+  httpStatus        Int?     @map("http_status")
+  latencyMs         Int      @map("latency_ms")
+  promptTokens      Int?     @map("prompt_tokens")
+  completionTokens  Int?     @map("completion_tokens")
+  reasoningTokens   Int?     @map("reasoning_tokens")
+  cachedTokens      Int?     @map("cached_tokens")
+  totalTokens       Int?     @map("total_tokens")
+  estimatedCostUsd  Float?   @map("estimated_cost_usd")
+  providerRequestId String?  @map("provider_request_id")
+  errorMessage      String?  @map("error_message")
+  conversationId    String?  @map("conversation_id")
+  messageId         String?  @map("message_id")
+
+  @@index([createdAt])
+  @@index([userId])
+  @@index([provider])
+  @@index([feature])
+  @@index([status])
+  @@map("ai_usage_logs")
+}
+
 // One row per scheduled job run (Gate 3). Purged after 30 days by housekeeping.
 model JobRun {
   id         String    @id @default(uuid())

backend/src/lib/ai/aiPricing.js

@@ -0,0 +1,84 @@
+/**
+ * AI Price Table and Cost Estimation
+ *
+ * Prices are in USD per 1 million tokens.
+ * Groq: console.groq.com/docs/models
+ *   - openai/gpt-oss-20b: $0.075 in / $0.30 out / $0.0375 cached-in, per 1M tokens
+ * Gemini: current prices from lib/ai/models.js
+ *   - gemini-flash-lite-latest: $0.10 in / $0.40 out, per 1M tokens
+ *   - gemini-flash-latest: $0.30 in / $2.50 out, per 1M tokens
+ */
+
+const AI_PRICE_TABLE = Object.freeze({
+  groq: {
+    'openai/gpt-oss-20b': { input: 0.075, output: 0.30, cachedInput: 0.0375 },
+    default: { input: 0.075, output: 0.30, cachedInput: 0.0375 },
+  },
+  gemini: {
+    'gemini-flash-lite-latest': { input: 0.10, output: 0.40, cachedInput: 0.025 },
+    'gemini-flash-latest': { input: 0.30, output: 2.50, cachedInput: 0.075 },
+    default: { input: 0.10, output: 0.40, cachedInput: 0.025 },
+  },
+  cloudflare: {
+    '@cf/cloudflare/clef-flash': { input: 0.0, output: 0.0, cachedInput: 0.0 },
+    default: { input: 0.0, output: 0.0, cachedInput: 0.0 },
+  },
+  clef: {
+    '@cf/cloudflare/clef-flash': { input: 0.0, output: 0.0, cachedInput: 0.0 },
+    default: { input: 0.0, output: 0.0, cachedInput: 0.0 },
+  },
+  openai: {
+    'gpt-4o-mini': { input: 0.15, output: 0.60, cachedInput: 0.075 },
+    'gpt-4o': { input: 2.50, output: 10.0, cachedInput: 1.25 },
+    default: { input: 0.15, output: 0.60, cachedInput: 0.075 },
+  },
+  anthropic: {
+    'claude-haiku-4-5': { input: 1.0, output: 5.0, cachedInput: 0.10 },
+    'claude-sonnet-5': { input: 2.0, output: 10.0, cachedInput: 0.20 },
+    'claude-opus-5': { input: 5.0, output: 25.0, cachedInput: 0.50 },
+    default: { input: 1.0, output: 5.0, cachedInput: 0.10 },
+  },
+});
+
+function getPriceFor(provider, model) {
+  const normProvider = (provider || '').toLowerCase();
+  const providerPrices = AI_PRICE_TABLE[normProvider];
+  if (!providerPrices) return null;
+  return providerPrices[model] || providerPrices.default || null;
+}
+
+/**
+ * Compute estimated USD cost for a given call
+ */
+function estimateCostUsd({ provider, model, promptTokens, completionTokens, cachedTokens = 0 }) {
+  const price = getPriceFor(provider, model);
+  if (!price) return null;
+
+  const prompt = Number.isFinite(promptTokens) ? Math.max(0, promptTokens) : 0;
+  const completion = Number.isFinite(completionTokens) ? Math.max(0, completionTokens) : 0;
+  const cached = Number.isFinite(cachedTokens) ? Math.max(0, Math.min(cachedTokens, prompt)) : 0;
+  const uncachedPrompt = Math.max(0, prompt - cached);
+
+  const cachedRate = price.cachedInput !== undefined ? price.cachedInput : price.input;
+  const totalCost = (uncachedPrompt * price.input + cached * cachedRate + completion * price.output) / 1e6;
+  return totalCost;
+}
+
+function getUsdInrRate() {
+  const envVal = parseFloat(process.env.USD_INR);
+  return Number.isFinite(envVal) && envVal > 0 ? envVal : 84.0;
+}
+
+function usdToInr(usd) {
+  if (usd == null || !Number.isFinite(usd)) return null;
+  return usd * getUsdInrRate();
+}
+
+module.exports = {
+  AI_PRICE_TABLE,
+  getPriceFor,
+  estimateCostUsd,
+  getUsdInrRate,
+  usdToInr,
+};
+

backend/src/lib/ai/aiUsageLogger.js

@@ -0,0 +1,139 @@
+/**
+ * Centralized AI Usage Tracking Logger (AiUsageLog)
+ *
+ * FIRE-AND-FORGET: A logging failure must never break or delay AI generation.
+ * PRIVACY CONTRACT: NEVER stores DM text, prompts, responses, or API keys.
+ * Metadata only (tokens, latency, model, user, feature, cost, errors).
+ */
+
+const prisma = require('../prisma');
+const { estimateCostUsd } = require('./aiPricing');
+
+/**
+ * Truncate error message safely to avoid huge payloads and prevent leaking sensitive data.
+ */
+function sanitizeErrorMessage(err) {
+  if (!err) return null;
+  const msg = typeof err === 'string' ? err : err.message || String(err);
+  // Redact potential api keys or tokens in error message
+  const cleaned = msg.replace(/(?:key|token|bearer|authorization)=?[\s:"]*[a-zA-Z0-9_\-\.]{10,}/gi, '[REDACTED_SECRET]');
+  return cleaned.slice(0, 500);
+}
+
+/**
+ * Normalise status to: success | error | timeout | rate_limited | empty
+ */
+function normalizeStatus({ status, httpStatus, error }) {
+  if (status) return status;
+  if (!error) return 'success';
+  if (httpStatus === 429 || error.status === 429 || error.code === 'RATE_LIMITED') return 'rate_limited';
+  if (
+    error.name === 'AbortError' ||
+    error.code === 'ETIMEDOUT' ||
+    error.code === 'ECONNABORTED' ||
+    /timeout/i.test(error.message || '')
+  ) {
+    return 'timeout';
+  }
+  if (error.aiErrorCode === 'EMPTY_RESPONSE' || error.code === 'EMPTY_RESPONSE') return 'empty';
+  return 'error';
+}
+
+/**
+ * Record an AI usage event into `ai_usage_logs` asynchronously (fire-and-forget).
+ *
+ * @param {object} params
+ * @param {string} [params.userId]
+ * @param {string} [params.tenantId]
+ * @param {string} params.feature - auto_reply | draft_reply | summary | classification | other
+ * @param {string} params.provider - groq | gemini | cloudflare | openai | anthropic
+ * @param {string} params.model
+ * @param {number} [params.attemptNo=1]
+ * @param {boolean} [params.isFallback=false]
+ * @param {string} [params.fallbackReason]
+ * @param {string} [params.status] - success | error | timeout | rate_limited | empty
+ * @param {number} [params.httpStatus]
+ * @param {number} params.latencyMs
+ * @param {number} [params.promptTokens]
+ * @param {number} [params.completionTokens]
+ * @param {number} [params.reasoningTokens]
+ * @param {number} [params.cachedTokens]
+ * @param {number} [params.totalTokens]
+ * @param {number} [params.estimatedCostUsd]
+ * @param {string} [params.providerRequestId]
+ * @param {Error|string} [params.error]
+ * @param {string} [params.errorMessage]
+ * @param {string} [params.conversationId]
+ * @param {string} [params.messageId]
+ */
+function logAiUsage(params) {
+  try {
+    const promptTokens = Number.isFinite(params.promptTokens) ? Math.max(0, Math.round(params.promptTokens)) : null;
+    const completionTokens = Number.isFinite(params.completionTokens) ? Math.max(0, Math.round(params.completionTokens)) : null;
+    const reasoningTokens = Number.isFinite(params.reasoningTokens) ? Math.max(0, Math.round(params.reasoningTokens)) : null;
+    const cachedTokens = Number.isFinite(params.cachedTokens) ? Math.max(0, Math.round(params.cachedTokens)) : null;
+
+    let totalTokens = Number.isFinite(params.totalTokens) ? Math.max(0, Math.round(params.totalTokens)) : null;
+    if (totalTokens == null && (promptTokens != null || completionTokens != null)) {
+      totalTokens = (promptTokens || 0) + (completionTokens || 0);
+    }
+
+    const estimatedCostUsd = params.estimatedCostUsd !== undefined
+      ? params.estimatedCostUsd
+      : estimateCostUsd({
+          provider: params.provider,
+          model: params.model,
+          promptTokens,
+          completionTokens,
+          cachedTokens,
+        });
+
+    const status = normalizeStatus({
+      status: params.status,
+      httpStatus: params.httpStatus,
+      error: params.error,
+    });
+
+    const errorMessage = params.errorMessage || (params.error ? sanitizeErrorMessage(params.error) : null);
+
+    const rawHttpStatus = params.httpStatus ?? (params.error?.status || params.error?.response?.status || null);
+    const safeHttpStatus = typeof rawHttpStatus === 'number' && Number.isInteger(rawHttpStatus) ? rawHttpStatus : null;
+
+    const data = {
+      userId: params.userId || null,
+      tenantId: params.tenantId || null,
+      feature: params.feature || 'other',
+      provider: (params.provider || 'unknown').toLowerCase(),
+      model: params.model || 'unknown',
+      attemptNo: params.attemptNo || 1,
+      isFallback: Boolean(params.isFallback),
+      fallbackReason: params.fallbackReason || null,
+      status,
+      httpStatus: safeHttpStatus,
+      latencyMs: Math.max(0, Math.round(params.latencyMs || 0)),
+      promptTokens,
+      completionTokens,
+      reasoningTokens,
+      cachedTokens,
+      totalTokens,
+      estimatedCostUsd,
+      providerRequestId: params.providerRequestId || null,
+      errorMessage,
+      conversationId: params.conversationId || null,
+      messageId: params.messageId || null,
+    };
+
+    // Non-blocking fire-and-forget write to DB
+    prisma.aiUsageLog.create({ data }).catch((err) => {
+      console.error('[AiUsageLog] Non-blocking write failed:', err.message);
+    });
+  } catch (err) {
+    console.error('[AiUsageLog] Error preparing log entry:', err.message);
+  }
+}
+
+module.exports = {
+  logAiUsage,
+  sanitizeErrorMessage,
+  normalizeStatus,
+};