[agoldis]

Owner: @agoldis

Linear: ENG-2181

Adds the ai-queue-worker deployment, off by default. On-prem webhooks no longer runs AI tasks after the Currents release that contains currents-dev/currents#4874, so an install that uses AI needs this deployment turned on.

This must ship with, not before, the Currents release that contains currents-dev/currents#4874. The image currents/on-prem/ai-queue-worker does not exist before then.

Changes

• New aiQueueWorker values: enabled (default false), replicas (default 1), image currents/on-prem/ai-queue-worker (same registry and tag as the other images), env, probes, resources and scheduling like webhooks.
• Why off by default: the worker does nothing useful until the admin gives it an LLM endpoint. The docs don't yet say what AI features an admin gets, how we use their provider, what token usage to expect, or how to limit it. AI is also off on-prem until an org admin grants consent, so a default install queues no AI tasks.
• templates/ai-queue-worker/deployment.yaml renders only when aiQueueWorker.enabled is true. It is a copy of the webhooks deployment, with the same MongoDB, Redis, object storage and URL env and secret refs from the shared helpers. It keeps the image's own command and uses the same pm2 probes (the process is also named dist).
• With aiQueueWorker.enabled: true, the scheduler also gets AI_QUEUE_WORKER_ENABLED=true. The scheduler builds the MongoDB indexes, and it declares the ones on aiSessions, aiSessionMessages and aiSlackCallbacks (including their TTLs) only with that variable. Turning AI on is therefore one value.
• webhooks and director are unchanged.
• Docs: a quickstart section on turning the worker on, ECR repository in the IAM policy, regenerated docs/configuration.md.
• No chart version bump. Versions change in the release PR (release: x.y.z), as in release: 1.0.1 [ENG-1851] #69.

Upgrade notes for on-prem customers

• Installs that run AI today (AI_QUEUE_WORKER_ENABLED=true on webhooks) lose it at this upgrade until they turn the worker on:
  • set aiQueueWorker.enabled: true;
  • move the LLM_* settings from webhooks.env to aiQueueWorker.env, or to global.env;
  • on webhooks, replace AI_QUEUE_WORKER_ENABLED=true with SLACK_AI_CALLBACK_WORKER_ENABLED=true if they use Slack AI.
• The scheduler builds the AI collection indexes, including the TTL indexes that expire AI sessions, and the chart now gives it the variable for that. The director builds indexes only with CREATE_MONGO_INDEX, which the chart never sets, so AI_QUEUE_WORKER_ENABLED on director does nothing.
• Pull access: to turn the worker on, the ECR policy for the nodes must include the currents/on-prem/ai-queue-worker repository (see docs/eks/iam.md). Mirror the image if you mirror the others.

Follow-up

• After feat: check what an upgrade's index builds cost before any pod rolls [ENG-1881] #72 merges, its pre-upgrade currents-index-check Job should get the same variable when the worker is on. Without it, the Job's plan leaves out the AI indexes the scheduler is about to build. Those collections are small, so the estimate barely moves.
• ENG-2209: the docs that would let AI default to on.

Checked

• helm lint with charts/currents/ci/lint-values.yaml passes.
• helm template with the CI values renders nothing for the worker by default, and the scheduler has no AI_QUEUE_WORKER_ENABLED. With aiQueueWorker.enabled=true it renders the worker Deployment (1 replica, currents/on-prem/ai-queue-worker image), and the scheduler, and no other deployment, gets AI_QUEUE_WORKER_ENABLED=true.
• docs/configuration.md was regenerated with jnorwood/helm-docs, as the docs check runs it.
• The ai-queue-worker:2026-11-10-003 image is in ECR: the customer-images mirror copied it.

🤖 Generated with Claude Code

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}

[baz-reviewer]

Baz Summary

Add an on-premises ai-queue-worker deployment to consume ai-bmq tasks independently from webhooks, restoring AI explanations and chat after AI processing is removed from webhooks. Configure its image, runtime settings, health checks, scheduling, ECR access, and operator documentation for LLM connectivity and upgrades.

Topics

Topic	Details
AI Worker Deployment	Deploy the standalone ai-queue-worker service to process on-premises ai-bmq tasks with configurable replicas, image, environment, probes, resources, storage, and scheduling. Keep AI processing separate from webhooks while preserving shared service connectivity. Modified files (4) charts/currents/templates/_helpers.tpl charts/currents/templates/ai-queue-worker/deployment.yaml charts/currents/values.yaml docs/configuration.md Latest Contributors(2) User Commit Date agoldis@gmail.com feat: add the ai-queue... November 10, 2026 dj@currents.dev fix: fail templating w... November 02, 2026
On-Prem Operations	Document the worker’s LLM configuration and operational requirements, including ECR pull permissions, migration from webhooks settings, and disabling the worker when AI is unused. Modified files (2) docs/eks/iam.md docs/eks/quickstart.md Latest Contributors(2) User Commit Date agoldis@gmail.com docs: describe the web... November 10, 2026 dj@currents.dev release: 1.0.0 [] (#64) October 14, 2026

---

Merger   ✅ Ready to Merge

MERGE: the PR delivers the AI queue worker deployment and scheduler configuration, CI ran successfully, and all review discussions are settled.

---

  _{Activate Skill Maintainer to keep your skills up to date}

  _{This PR would have been improved with Baz Planner - Try it now}

[twk3]

Tested on Docker Desktop Kubernetes at 4c53f75.

Setup:

• The seven release/2026-11-10-003 images, built locally and imported into the node.
• MongoDB, ClickHouse and Redis from .github/ci/dependencies.yaml.
• -f charts/currents/ci/lint-values.yaml plus currents.imageTag: 2026-11-10-003, imagePullPolicy: IfNotPresent and no pull secrets.

1. Install with the defaults

• All six deployments came up Ready, and there's no ai-queue-worker deployment.
• The scheduler has no AI_QUEUE_WORKER_ENABLED. Its startup tasks completed, and the AI collections weren't created.
• I seeded a run with failed tests, granted the org FAILURE_ANALYSIS consent, and sent POST /v1/ai-requests (errorExplanation). The request stayed pending with its job in bull:ai-bmq:wait, which is what should happen with no worker.

2. helm upgrade with the worker on

Values

aiQueueWorker:
  enabled: true
  env:
    - name: LLM_PROVIDER_URL
      value: http://llm.invalid/v1
    - name: LLM_API_KEY
      value: unused
    - name: USE_MOCK_AI
      value: "true"

• The worker pod is 1/1 Ready with 0 restarts, so its startup and readiness probes pass.
• The scheduler restarted with AI_QUEUE_WORKER_ENABLED=true. No other deployment has the variable.
• The scheduler built the indexes on aiSessions, aiSessionMessages and aiSlackCallbacks, including the expiresAt_1 TTL indexes.
• The worker took the queued job ("Processing test error analysis", then "test error analysis completed"). The request went to completed with success: true, and the queue emptied.
• A new testExplanation request completed in 0.1 s.

3. helm upgrade back to the defaults

• The worker deployment was removed, and the scheduler restarted without the variable. Its startup completed with no errors.
• The AI indexes were left in place.

Not covered:

• A real LLM provider. USE_MOCK_AI returns a fixed answer without calling one.
• Pulling the images from ECR.
• The pre-upgrade index check from feat: check what an upgrade's index builds cost before any pod rolls [ENG-1881] #72, which isn't on this branch.