Skip to content

FEATURE: Add the Cloudflare Queues worker boundary - #33

Open
bmdavis419 wants to merge 3 commits into
review/hosted-03-pg-portfrom
review/hosted-04-queue-foundation
Open

FEATURE: Add the Cloudflare Queues worker boundary#33
bmdavis419 wants to merge 3 commits into
review/hosted-03-pg-portfrom
review/hosted-04-queue-foundation

Conversation

@bmdavis419

@bmdavis419 bmdavis419 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Add the typed Cloudflare Queues boundary: the generated Worker signs each batch, forwards it into the SvelteKit bundle, and applies per-message acknowledgment or retry decisions. Missing decisions retry; invalid jobs are acknowledged and logged. Operation handlers are connected in #36.

The internal endpoint limits streamed UTF-8 bytes before verifying the signature over the unchanged request text. Behavioral tests execute the generated facade and cover signatures, mixed decisions, missing decisions, and failed consumers.

Important files:

  • apps/web/scripts/cloudflare-adapter.mjs: queue entrypoint and signed forwarding.
  • apps/web/src/lib/server/jobs/consumer.ts: typed dispatch and per-message decisions.
  • apps/web/src/routes/api/internal/jobs/+server.ts: authenticated, bounded batch ingestion.
  • apps/web/src/lib/server/services/jobs.ts and apps/web/wrangler.jsonc: producer service and environment bindings.

Validation: full root tests including63 route/Postgres tests; TypeScript/Effect/Svelte checks; formatting; Worker build; final independent Codex review clean. Cloudflare delivery remains a deployment-time check.

Stack layer4/11: depends on #32; followed by #34.

Note

Add Cloudflare Queues worker boundary with HMAC-signed batch forwarding

  • Adds a Cloudflare Queue handler in cloudflare-adapter.mjs that serializes batches and POSTs them to POST /api/internal/jobs with an HMAC signature covering the timestamp and exact body.
  • The internal jobs endpoint verifies the signature, parses the batch against a new shared JobSchema, and returns per-message ack/retry decisions based on a new consumer in consumer.ts.
  • The consumer dispatches recognized jobs but currently only logs them; domain operations are marked with TODOs.
  • Adds Wrangler queue producer bindings and dead-letter queue configuration in wrangler.jsonc.
  • Behavioral Change: The queue handler throws on non-2xx responses and retries messages without explicit ack decisions. Endpoint body reads are now limited to 1 MiB via readBoundedText before signature verification.

Macroscope summarized d460dfd.

RetriggerConfidence Score: 5/5

Safe to merge with respect to blocking issues; the remaining configuration-drift concern is non-blocking.

Fix All in CodexFindings

  1. P2 Check queue configuration drift
Fix with agent prompt
### Issue 1
scripts/check-wrangler-drift.mjs:184-189
This check reduces producers to binding names and does not compare producer destinations or any consumer configuration. A production queue can therefore use a different destination, retry limit, or consumer queue while this release check reports no drift. This is non-blocking, but it removes the intended deployment safeguard and can leave jobs routed or retried differently in production.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

  • This PR adds typed Cloudflare Queues support for background jobs, including signed job delivery into the application, bounded request validation, per-message acknowledgement or retry handling, producer bindings, deployment configuration, migration serialization, and safeguards for route tests.

Reviews (2) · Last reviewed commit: "BUGFIX: Bound queue requests before sign..."

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

On-demand reviews are free for the next 9 days. After that, they cost $0.25 per reviewed file.

Or wait 33 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used all 6 included reviews currently available. Your 49 included PR review attempts over the past 7 days set your current allowance at 6 reviews per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 06b9b612-2a6f-43fb-857d-16f8c6cdec4b

📥 Commits

Reviewing files that changed from the base of the PR and between dd16506 and d460dfd.

📒 Files selected for processing (19)
  • apps/web/scripts/cloudflare-adapter.mjs
  • apps/web/scripts/cloudflare-adapter.test.ts
  • apps/web/src/lib/server/cron-auth.test.ts
  • apps/web/src/lib/server/cron-auth.ts
  • apps/web/src/lib/server/edge.ts
  • apps/web/src/lib/server/jobs/consumer.test.ts
  • apps/web/src/lib/server/jobs/consumer.ts
  • apps/web/src/lib/server/layer.ts
  • apps/web/src/lib/server/request-json.test.ts
  • apps/web/src/lib/server/request-json.ts
  • apps/web/src/lib/server/routes/jobs.test.ts
  • apps/web/src/lib/server/services/bindings.ts
  • apps/web/src/lib/server/services/jobs.ts
  • apps/web/src/routes/api/internal/jobs/+server.ts
  • apps/web/worker-configuration.d.ts
  • apps/web/wrangler.jsonc
  • docs/release.md
  • packages/shared/src/index.ts
  • scripts/check-wrangler-drift.mjs

Comment @coderabbitai help to get the list of available commands.

@bmdavis419
bmdavis419 added this pull request to stack #41 September 11, 2026 04:32
@bmdavis419
bmdavis419 force-pushed the review/hosted-04-queue-foundation branch from 31c4541 to 7efff9d Compare September 11, 2026 05:22
@bmdavis419
bmdavis419 marked this pull request as ready for review September 11, 2026 08:23
Comment on lines +184 to +189
const queueProducers = (block) => bindingNames(block.queues?.producers);
if (!sameJson(queueProducers(config), queueProducers(prod))) {
drift.push(
`queues.producers bindings: local=${JSON.stringify(queueProducers(config))} production=${JSON.stringify(queueProducers(prod))}`
);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Check queue configuration drift

This check reduces producers to binding names and does not compare producer destinations or any consumer configuration. A production queue can therefore use a different destination, retry limit, or consumer queue while this release check reports no drift. This is non-blocking, but it removes the intended deployment safeguard and can leave jobs routed or retried differently in production.

Knowledge Base Used:

Artifacts

Evidence from the check

  • The authored Node script clones the current Wrangler configuration into temporary directories, applies one queue-only mutation per run, and invokes the real drift checker; it reproduces the unchecked queue values.

Command output from the check

  • The real checker was executed against an unmodified temporary copy of the current Wrangler configuration and exited successfully with no drift.

Command output from the check

  • The real checker was executed against three temporary copies differing only in a producer destination, consumer retry setting, or consumer destination, and it passed all three; queue configuration drift is not detected.

View artifacts

T-Rex Ran code and verified through T-Rex

Prompt To Fix With AI
This is a comment left during a code review.
Path: scripts/check-wrangler-drift.mjs
Line: 184-189

Comment:
**Check queue configuration drift**

This check reduces producers to binding names and does not compare producer destinations or any consumer configuration. A production queue can therefore use a different destination, retry limit, or consumer queue while this release check reports no drift. This is non-blocking, but it removes the intended deployment safeguard and can leave jobs routed or retried differently in production.

**Knowledge Base Used:**
- [Runtime configuration and schema](https://app.greptile.com/davis7dotsh/-/custom-context/knowledge-base/davis7dotsh/adrive/-/docs/runtime-configuration-and-schema.md)
- [Release, backup, and safety automation](https://app.greptile.com/davis7dotsh/-/custom-context/knowledge-base/davis7dotsh/adrive/-/docs/release-backup-and-safety-automation.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex

bmdavis419 and others added 3 commits September 11, 2026 01:47
Declares the adrive-jobs queue (producer JOBS plus consumer with a
dead-letter queue) at the top level and in env.production, keeps the
producer bindings covered by the drift check, and regenerates the Worker
types. Adds the shared Job discriminated union and JobSchema, a JobQueue
service (with a null layer) in the request layer, and a consumer that
decodes each message, acks invalid bodies, and retries failed jobs. The
Worker facade forwards queue batches to /api/internal/jobs in-process
under an HMAC signed over the batch body, then acks or retries per
message from the endpoint's decisions. No behaviour moves onto the queue
yet; runJob only logs until D2/D3.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@bmdavis419
bmdavis419 force-pushed the review/hosted-04-queue-foundation branch from 7efff9d to d460dfd Compare September 11, 2026 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant