Email notifications

[ngoiyaeric]

Enable the application to send notifications regarding the system asynchronous tasking.

[ngoiyaeric]

@coderabbitai plan

[coderabbitai]

Coding Plan

Summary

• Build a from-scratch, build-safe email module (lib/email/) around Resend, following the singleton/env-guard pattern of lib/db/index.ts and the app's env-var conventions.
• Add an explicit user opt-in as a new users.email_notifications_enabled column with get/save actions in lib/actions/users.ts and a Switch toggle in the Account settings tab, reusing existing preference patterns.
• Hook email dispatch into the only durable async job (runBackgroundWorker in lib/actions/system-prompt.ts) as a non-blocking, preference-gated, idempotent side effect on its complete/error transitions, resolving the recipient from the user's Clerk-authenticated primary verified email (via clerkClient.users.getUser) rather than the DB users.email column, with safe logged fallbacks — and without introducing any new queue or cron infrastructure.

Design Choices

Design Choice 1: Which email provider/library should power transactional sends?

Options Considered:

1. Resend — modern transactional email API with a lightweight Node SDK, first-class idempotency-key support, and simple env-var configuration.
2. Nodemailer + SMTP — flexible but requires managing SMTP credentials/deliverability manually.
3. SendGrid/Postmark/SES — capable but heavier setup and less aligned with the app's simple integration style.

Chosen Option: 1

Rationale: It matches the codebase's preference for simple, single-SDK integrations (like Composio/Firecrawl), supports idempotency keys to avoid duplicate sends on retries, and aligns with current best-practice guidance.

Design Choice 2: Where should the "email notifications enabled" preference be stored?

Options Considered:

1. New boolean column email_notifications_enabled on users (mirrors selected_model).
2. Store inside existing users.metadata jsonb (no migration needed).

Chosen Option: 1

Rationale: A first-class column is consistent with how selectedModel/systemPrompt preferences are modeled, is easy to query for server-side gating, and keeps the opt-in explicit.

Design Choice 3: How should the email send be dispatched relative to the job's DB update?

Options Considered:

1. Fire-and-forget side effect inside runBackgroundWorker after the updateJob terminal call, wrapped in its own try/catch (mirrors the Clerk webhook's non-blocking secondary-write pattern).
2. Introduce a queue/cron to process pending notifications.

Chosen Option: 1

Rationale: The worker is already a fire-and-forget async function; adding a guarded, non-blocking email side effect matches existing patterns and avoids introducing new queue/cron infrastructure that the ticket does not require.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Email-Sending Infrastructure

Establish a reusable, server-side email capability from scratch, following the codebase's singleton-client and env-var conventions. The goal is a single, well-guarded module any server-side flow can call to send a transactional email, plus the configuration and documentation needed to operate it.

Task 1: Add email provider dependency and configuration

Introduce the email SDK and its credentials following existing env conventions.

• Add the resend package to package.json.
• Add a new grouped block to .env.local.example for email credentials (e.g. RESEND_API_KEY, EMAIL_FROM_ADDRESS) using the existing # Section comment + SERVICE_KEYNAME placeholder convention (server-only, no NEXT_PUBLIC_ prefix).

Task 2: Create an email client singleton and send helper

Provide a lazily-initialized, build-safe email client and a typed send function.

• Create a new module under lib/email/ (e.g. lib/email/index.ts) that lazily instantiates the Resend client, mirroring the env-guard/build-safe pattern used in lib/db/index.ts (return a stub or no-op when credentials are absent so build/import never crashes).
• Expose a small sendEmail({ to, subject, html/text, idempotencyKey? }) helper that centralizes the from address, error handling (log-and-swallow so callers never throw), and idempotency-key passthrough to avoid duplicate sends.

Task 3: Create task-notification email content builders

Encapsulate the notification's subject/body construction separate from transport.

• Add a content-builder helper (in lib/email/) that produces the subject and body for an async-task-completion notification, taking task type, terminal status (complete/error), and any relevant detail (e.g. domain, error message).
• Keep templates as lightweight inline HTML/text strings (no new templating dependency) to match the project's minimal-abstraction style.

🤖 Prompt for AI agents

Implement the email-sending infrastructure from scratch for the app: a reusable,
server-side email capability following the codebase's singleton-client and
env-var conventions.

- Add the `resend` package as a dependency in `package.json`.
- In `.env.local.example`, add a new grouped block for email credentials (e.g.
`RESEND_API_KEY`, `EMAIL_FROM_ADDRESS`) using the existing `# Section` comment +
`SERVICE_KEYNAME` placeholder convention; these must be server-only (no
`NEXT_PUBLIC_` prefix).
- Create a new module under `lib/email/` (e.g. `lib/email/index.ts`) that lazily
instantiates the Resend client, mirroring the env-guard/build-safe pattern used
in `lib/db/index.ts`: if credentials are absent, return a stub or no-op so
build/import never crashes.
- In that module, expose a `sendEmail({ to, subject, html/text, idempotencyKey?
})` helper that centralizes the `from` address, handles errors by logging and
swallowing them (never throw to callers), and passes through the idempotency key
to avoid duplicate sends.
- Add a separate content-builder helper (also under `lib/email/`) that produces
the subject and body text for an async-task-completion notification email,
accepting task type, terminal status (`complete`/`error`), and relevant detail
(e.g. domain, error message).
- Keep the templates as lightweight inline HTML/text strings — do not introduce
a new templating dependency, to match the project's minimal-abstraction style.

Phase 2: User Notification Preference (Opt-In)

Give users control over whether they receive these emails, honoring the existing preference-storage and settings-UI patterns. The goal is an explicit, persisted opt-in that the dispatch logic can check server-side before sending.

Task 1: Add the preference column and migration

Persist the opt-in flag on the user record.

• Add an emailNotificationsEnabled boolean column (default true) to the users table in lib/db/schema.ts, mirroring the selectedModel scalar-preference pattern.
• Hand-write a new migration file drizzle/migrations/0009_add_email_notifications_pref.sql following the 0007-style ALTER TABLE ... ADD COLUMN IF NOT EXISTS convention (note the journal is stale; hand-written migrations are the recent norm).

Task 2: Add preference get/save server actions

Expose read/write access to the preference following the established action pattern.

• Extend lib/actions/users.ts with getEmailNotificationsEnabled() and saveEmailNotificationsEnabled(enabled: boolean), following the exact getSelectedModel/saveSelectedModel shape (getCurrentUserIdOnServer() gate → Drizzle read/update → revalidatePath('/settings')).

Task 3: Add the settings UI toggle

Surface the opt-in in the settings panel using existing form conventions.

• Add an emailNotifications boolean field to settingsFormSchema and defaultValues in components/settings/components/settings.tsx, load it on mount via getEmailNotificationsEnabled(), and include saveEmailNotificationsEnabled(...) in the existing Promise.all save flow.
• Render the toggle using the existing components/ui/switch.tsx inside a FormField, placed in the Account tab (identity/account related), following the sub-form pattern used by system-prompt-form.tsx.

🤖 Prompt for AI agents

Implement a persisted, explicit user opt-in preference for email notifications,
following existing preference-storage and settings-UI patterns.

- In `lib/db/schema.ts`, add an `emailNotificationsEnabled` boolean column to
the `users` table with a default of `true`, mirroring the `selectedModel`
scalar-preference pattern.
- Hand-write a new migration file
`drizzle/migrations/0009_add_email_notifications_pref.sql` following the
`0007`-style `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` convention (note: the
migration journal is stale, so hand-written migrations are the current norm).
- Extend `lib/actions/users.ts` with two new server actions:
`getEmailNotificationsEnabled()` and `saveEmailNotificationsEnabled(enabled:
boolean)`. Follow the exact shape of `getSelectedModel`/`saveSelectedModel`:
gate on `getCurrentUserIdOnServer()`, perform the Drizzle read/update, and call
`revalidatePath('/settings')` on save.
- In `components/settings/components/settings.tsx`, add an `emailNotifications`
boolean field to `settingsFormSchema` and to `defaultValues`. Load its initial
value on mount via `getEmailNotificationsEnabled()`, and include a call to
`saveEmailNotificationsEnabled(...)` in the existing `Promise.all` save flow.
- Render the toggle control using the existing `components/ui/switch.tsx`
component wrapped in a `FormField`, placing it in the Account tab (since it is
identity/account related), following the sub-form pattern already used by
`system-prompt-form.tsx`.

Phase 3: Dispatch Notifications on Async Task Completion

Wire the email capability into the async task lifecycle so users are notified when their background job reaches a terminal state, gated by their preference. The goal is a non-blocking side effect on the existing job's success/error transitions, with the recipient address resolved from Clerk (not the DB users.email column).

Task 1: Trigger emails from the background worker's terminal transitions

Send an email when the async task completes or fails.

• In lib/actions/system-prompt.ts::runBackgroundWorker, after the updateJob(..., { status: 'complete', ... }) success call and after the updateJob(..., { status: 'error', ... }) catch-block call, invoke a notification dispatch step.
• Implement dispatch as a guarded, non-blocking side effect (its own try/catch that logs and swallows errors) so email failures never affect job state recording — mirroring the Clerk webhook's non-blocking secondary-write pattern.

Task 2: Resolve recipient from Clerk and honor opt-in before sending

Ensure emails only go to opted-in users, addressed to their Clerk-authenticated primary verified email (never the DB-synced users.email column).

• Implement recipient resolution as an explicit five-step sequence:
  1. Get the job's userId (the local DB users.id passed into runBackgroundWorker).
  2. Retrieve the corresponding Clerk user via Clerk's server-side backend API — look up the user's clerkUserId (from the local users table) and call clerkClient.users.getUser(clerkUserId) from @clerk/nextjs/server, since the worker is headless and the request-scoped auth()/currentUser() helpers are not usable here.
  3. Select the Clerk primary authenticated email by matching user.primaryEmailAddressId against user.emailAddresses[].id and confirming verification.status === 'verified' — this Clerk primary verified email is the sole delivery address (do not read or use the DB users.email column for delivery; it remains reserved for other/unrelated features only).
  4. Check the DB opt-in flag by reading users.emailNotificationsEnabled via Drizzle for the same userId.
  5. Send only if both conditions are met (a valid Clerk primary verified email exists AND the user is opted in); otherwise skip.
• Add explicit fallback handling as its own step: if there is no primary email, no verified email, or the Clerk lookup fails/throws (e.g. ClerkAPIResponseError, 429), skip the send safely and log the reason — this must never affect job completion or the recorded terminal state (all wrapped in the guarded try/catch from Task 3.1).
• Build the subject/body via the Phase 1 content builder (task type = system-prompt generation, status, domain, error message) and call sendEmail(...) with an idempotency key derived from the job id + terminal status to prevent duplicates (carry forward unchanged from the original plan).

Task 3: Document the email notification setup

Provide operator/developer documentation consistent with existing integration docs.

• Add a short Markdown doc (e.g. docs/EMAIL_NOTIFICATIONS.md) following the COMPOSIO_MIGRATION.md/docs/ALPHAEARTH_SETUP.md structure (Overview → Configuration/env vars → How it hooks into async jobs → Preferences → Troubleshooting).
• Explicitly state in the doc that notification emails are sent to the user's Clerk-authenticated primary verified email, resolved server-side via Clerk's backend API — not the database-synced users.email column — and document the skip/fallback behavior (no primary email, no verified email, or Clerk lookup failure results in a safe, logged skip that does not affect job completion).

🤖 Prompt for AI agents

Wire the email capability into the async task lifecycle so users are notified
when their background job reaches a terminal state, gated by their preference,
as a non-blocking side effect, with the recipient resolved from Clerk (not the
DB `users.email` column).

- In `lib/actions/system-prompt.ts`, inside `runBackgroundWorker`, add a
notification dispatch step after the `updateJob(..., { status: 'complete', ...
})` success call and after the `updateJob(..., { status: 'error', ... })`
catch-block call.
- Implement this dispatch as a guarded, non-blocking side effect wrapped in its
own try/catch that logs and swallows any errors, so email failures never affect
job state recording — mirror the Clerk webhook's non-blocking secondary-write
pattern.
- Implement recipient resolution as an explicit five-step sequence: (1) get the
job's `userId` (the local DB `users.id`); (2) retrieve the corresponding Clerk
user via `clerkClient.users.getUser(clerkUserId)` from `@clerk/nextjs/server`,
using the `clerkUserId` stored in the local `users` table, since the worker is
headless and the request-scoped `auth()`/`currentUser()` helpers cannot be used
here; (3) select the Clerk primary authenticated email by matching
`user.primaryEmailAddressId` against `user.emailAddresses[].id` and confirming
`verification.status === 'verified'` — this Clerk primary verified email is the
sole delivery address; never read or use the DB `users.email` column for
delivery, as it is reserved for other/unrelated features; (4) check the DB
opt-in flag by reading `users.emailNotificationsEnabled` via Drizzle for the
same `userId`; (5) send only if both a valid Clerk primary verified email exists
AND the user is opted in, otherwise skip.
- Add explicit fallback handling: if there is no primary email, no verified
email, or the Clerk lookup fails/throws (e.g. `ClerkAPIResponseError`, `429`),
skip the send safely and log the reason, ensuring this never affects job
completion or the recorded terminal state (all inside the guarded try/catch).
- Build the email subject/body using the Phase 1 content builder, passing task
type = system-prompt generation, the terminal status, domain, and error message
as applicable. Call `sendEmail(...)` from the Phase 1 helper, passing an
idempotency key derived from the job id plus terminal status to prevent
duplicate sends.
- Add a short Markdown documentation file (e.g. `docs/EMAIL_NOTIFICATIONS.md`)
following the structure of `COMPOSIO_MIGRATION.md`/`docs/ALPHAEARTH_SETUP.md`:
Overview → Configuration/env vars → How it hooks into async jobs → Preferences →
Troubleshooting. Explicitly state that notification emails go to the user's
Clerk-authenticated primary verified email (resolved server-side via Clerk's
backend API), not the DB-synced `users.email` column, and document the
skip/fallback behavior for missing/unverified email or Clerk lookup failure.

Research

This is a Next.js 15 (App Router, React 19, Node.js runtime) application using Drizzle ORM over Postgres, Clerk for auth, and sonner for in-app toasts. There is currently no email-sending library, notification table, or job queue. The single true background job is runBackgroundWorker in lib/actions/system-prompt.ts, which scrapes a domain and generates a prompt, recording terminal state via updateJob(...) on the prompt_generation_jobs table (status: complete | error). User preferences follow a simple pattern of scalar columns on users plus get/save server actions in lib/actions/users.ts. Settings UI is a Radix Tabs-based shared react-hook-form shell (components/settings/components/settings.tsx) with per-section sub-forms and a Switch primitive already available. Note on identity: although a database-synced users.email column exists, it is used only for other/unrelated features (Clerk↔DB sync, user identity matching) and is explicitly NOT the source of truth for notification delivery — the recipient address for notifications must be resolved from Clerk's server-side API. clerkClient from @clerk/nextjs/server (backed by @clerk/backend, already a dependency) is not yet used anywhere and would be the context-independent way to fetch a user inside the headless background worker (the existing auth()/currentUser() helpers are request-scoped and won't work there).

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Implement the email-sending infrastructure from scratch for the app: a reusable,
server-side email capability following the codebase's singleton-client and
env-var conventions.

- Add the `resend` package as a dependency in `package.json`.
- In `.env.local.example`, add a new grouped block for email credentials (e.g.
`RESEND_API_KEY`, `EMAIL_FROM_ADDRESS`) using the existing `# Section` comment +
`SERVICE_KEYNAME` placeholder convention; these must be server-only (no
`NEXT_PUBLIC_` prefix).
- Create a new module under `lib/email/` (e.g. `lib/email/index.ts`) that lazily
instantiates the Resend client, mirroring the env-guard/build-safe pattern used
in `lib/db/index.ts`: if credentials are absent, return a stub or no-op so
build/import never crashes.
- In that module, expose a `sendEmail({ to, subject, html/text, idempotencyKey?
})` helper that centralizes the `from` address, handles errors by logging and
swallowing them (never throw to callers), and passes through the idempotency key
to avoid duplicate sends.
- Add a separate content-builder helper (also under `lib/email/`) that produces
the subject and body text for an async-task-completion notification email,
accepting task type, terminal status (`complete`/`error`), and relevant detail
(e.g. domain, error message).
- Keep the templates as lightweight inline HTML/text strings — do not introduce
a new templating dependency, to match the project's minimal-abstraction style.
===============================================================================

Task: 2

Implement a persisted, explicit user opt-in preference for email notifications,
following existing preference-storage and settings-UI patterns.

- In `lib/db/schema.ts`, add an `emailNotificationsEnabled` boolean column to
the `users` table with a default of `true`, mirroring the `selectedModel`
scalar-preference pattern.
- Hand-write a new migration file
`drizzle/migrations/0009_add_email_notifications_pref.sql` following the
`0007`-style `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` convention (note: the
migration journal is stale, so hand-written migrations are the current norm).
- Extend `lib/actions/users.ts` with two new server actions:
`getEmailNotificationsEnabled()` and `saveEmailNotificationsEnabled(enabled:
boolean)`. Follow the exact shape of `getSelectedModel`/`saveSelectedModel`:
gate on `getCurrentUserIdOnServer()`, perform the Drizzle read/update, and call
`revalidatePath('/settings')` on save.
- In `components/settings/components/settings.tsx`, add an `emailNotifications`
boolean field to `settingsFormSchema` and to `defaultValues`. Load its initial
value on mount via `getEmailNotificationsEnabled()`, and include a call to
`saveEmailNotificationsEnabled(...)` in the existing `Promise.all` save flow.
- Render the toggle control using the existing `components/ui/switch.tsx`
component wrapped in a `FormField`, placing it in the Account tab (since it is
identity/account related), following the sub-form pattern already used by
`system-prompt-form.tsx`.
===============================================================================

Task: 3

Wire the email capability into the async task lifecycle so users are notified
when their background job reaches a terminal state, gated by their preference,
as a non-blocking side effect, with the recipient resolved from Clerk (not the
DB `users.email` column).

- In `lib/actions/system-prompt.ts`, inside `runBackgroundWorker`, add a
notification dispatch step after the `updateJob(..., { status: 'complete', ...
})` success call and after the `updateJob(..., { status: 'error', ... })`
catch-block call.
- Implement this dispatch as a guarded, non-blocking side effect wrapped in its
own try/catch that logs and swallows any errors, so email failures never affect
job state recording — mirror the Clerk webhook's non-blocking secondary-write
pattern.
- Implement recipient resolution as an explicit five-step sequence: (1) get the
job's `userId` (the local DB `users.id`); (2) retrieve the corresponding Clerk
user via `clerkClient.users.getUser(clerkUserId)` from `@clerk/nextjs/server`,
using the `clerkUserId` stored in the local `users` table, since the worker is
headless and the request-scoped `auth()`/`currentUser()` helpers cannot be used
here; (3) select the Clerk primary authenticated email by matching
`user.primaryEmailAddressId` against `user.emailAddresses[].id` and confirming
`verification.status === 'verified'` — this Clerk primary verified email is the
sole delivery address; never read or use the DB `users.email` column for
delivery, as it is reserved for other/unrelated features; (4) check the DB
opt-in flag by reading `users.emailNotificationsEnabled` via Drizzle for the
same `userId`; (5) send only if both a valid Clerk primary verified email exists
AND the user is opted in, otherwise skip.
- Add explicit fallback handling: if there is no primary email, no verified
email, or the Clerk lookup fails/throws (e.g. `ClerkAPIResponseError`, `429`),
skip the send safely and log the reason, ensuring this never affects job
completion or the recorded terminal state (all inside the guarded try/catch).
- Build the email subject/body using the Phase 1 content builder, passing task
type = system-prompt generation, the terminal status, domain, and error message
as applicable. Call `sendEmail(...)` from the Phase 1 helper, passing an
idempotency key derived from the job id plus terminal status to prevent
duplicate sends.
- Add a short Markdown documentation file (e.g. `docs/EMAIL_NOTIFICATIONS.md`)
following the structure of `COMPOSIO_MIGRATION.md`/`docs/ALPHAEARTH_SETUP.md`:
Overview → Configuration/env vars → How it hooks into async jobs → Preferences →
Troubleshooting. Explicitly state that notification emails go to the user's
Clerk-authenticated primary verified email (resolved server-side via Clerk's
backend API), not the DB-synced `users.email` column, and document the
skip/fallback behavior for missing/unverified email or Clerk lookup failure.

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- @coderabbitai You can skip phase 3. Add a simple unit test case for phase 2.
- @coderabbitai For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!

[ngoiyaeric]

@coderabbitai plan use the clerk authenticated email.

[coderabbitai]

I've updated the plan above based on your feedback. Ready to proceed when you are!