Two scheduled cron workflows fail on every run (~10k/month) -- disable them and remove their matching controllers

[colinwilliams91]

Problem

Two scheduled GitHub Actions workflows have been failing on every single run for an extended period (confirmed continuous over the last 24h in the Actions dashboard):

• Check Queues — .github/workflows/check-queues.yml, cron */5 * * * * (~288 runs/day)
• Clean Every 30 Minutes Before 24h Completed Jobs — .github/workflows/clean-before-24h-complete-jobs.yml, cron 30 * * * * (~48 runs/day)

Combined: ~336 failed runs/day, ~10,000 failed runs/month.

Root cause (from the run logs)

From run 29581536093:

##[group]Run response=$(curl ... https://api.firecrawl.dev/admin//check-queues)
                                                  ^^  double slash = empty BULL_AUTH_KEY
env:
  BULL_AUTH_KEY:           <- empty (secret not configured)
Failed to check queues. Response: 404

Two compounding issues:

1. BULL_AUTH_KEY is not configured as a repo secret. ${{ secrets.BULL_AUTH_KEY }} resolves to empty, the URL becomes admin//check-queues, and the request returns HTTP 404.
2. Wrong target entirely. The workflows curl https://api.firecrawl.dev, which is upstream firecrawl's (Mendable's) own SaaS deployment, NOT this fork's deployment. This repo is a fork of firecrawl/firecrawl; the firecrawl org owns api.firecrawl.dev. These workflows were copied verbatim when the fork diverged and were never adapted to a fork-owned endpoint (and no fork-owned prod endpoint is documented in this repo).

Why disabling (and removing the matching controllers) is safe

Verified against the code. Even if these were re-pointed at a fork-owned deployment with the secret configured, the operational benefit would be nil:

1. check-queues calls checkAlerts() (apps/api/src/services/alerts/index.ts), which is a no-op by default. It is guarded by process.env.ENV === "production" AND process.env.ALERT_NUM_ACTIVE_JOBS AND process.env.ALERT_NUM_WAITING_JOBS — none of which appear in .env.example, docker-compose.yaml, README.md, SELF_HOST.md, or CONTRIBUTING.md. When unset, the controller just logs "Initializing alerts" and returns 200. Slack output was previously stripped (commit b024f08 cleanup: remove SLACK_WEBHOOK_URL env and code); zero SLACK_* env reads remain in the repo. The only consumer of checkAlerts() is checkQueuesController. So a correctly-pointed check-queues cron would emit only local Logger.info/warn lines that nobody reads.

2. clean-before-24h-complete-jobs is largely redundant. BullMQ's queue is created with removeOnComplete: { age: 90000 } (25h) and removeOnFail: { age: 90000 } (apps/api/src/services/queue-service.ts:18-23), so completed/failed jobs auto-evict after ~25h anyway. The controller also caps at 9 batches x 10 = ~90 jobs per call (apps/api/src/controllers/v0/admin/queue.ts:16-27), inadequate for any real backlog.

3. Upstream itself has fully removed both workflows AND the matching controllers. .github/workflows/check-queues.yml, .github/workflows/clean-before-24h-complete-jobs.yml, apps/api/src/controllers/v0/admin/queue.ts, and apps/api/src/services/alerts.ts are all GONE from firecrawl/firecrawl. This fork is carrying cold ash from before upstream's queue/admin refactor.

Proposed fix

• Move both workflows into .github/archive/ (matching the existing precedent — 6 SDK workflows already live there) with on: [] as belt-and-suspenders.
• Remove the two cron-matched controllers (checkQueuesController, cleanBefore24hCompleteJobsController) from apps/api/src/controllers/v0/admin/queue.ts and their route registrations from apps/api/src/routes/admin.ts.
• Delete the now-dead apps/api/src/services/alerts/ directory (its only consumer was checkQueuesController).
• Keep queuesController (the /admin/<KEY>/queues JSON health endpoint — still useful to self-hosters) and redisHealthController (upstream also kept it).
• Keep BULL_AUTH_KEY in .env.example / docker-compose / docs — self-hosters still use it to protect their own admin endpoints to their own deployments.

Cost note (correction)

The repo is public (visibility: public), and GitHub Actions minutes are free and unlimited for public repos — so these failing crons are not directly costing money today. The real harms are:

• ~10k failed runs/month drowning the CI dashboard, hiding real signal from build-docker-images.yml.
• Each run mints a GITHUB_TOKEN with Contents: write, PullRequests: write, etc. — small but unnecessary exposure surface on every run.
• Risk of GitHub's abuse-detection flagging constant scheduled failures.
• If the repo ever flips private, the bill kicks in immediately: ~1.2k Actions min/month from these two crons alone (against the 2k free allowance).

Latent bug (flagged, out of scope)

apps/api/src/routes/admin.ts:11-26 mounts the admin routes unconditionally. If BULL_AUTH_KEY is unset, routes register at literal /admin/undefined/... paths — trivially guessable. Upstream gates the whole block behind if (config.BULL_AUTH_KEY). This fork has not backported that guard. Worth a separate issue; not addressed here.