Send ramp completion and verification outcome emails - #1295
Conversation
✅ Deploy Preview for vortexfi ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for vrtx-dashboard canceled.
|
✅ Deploy Preview for vortex-sandbox ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
There was a problem hiding this comment.
🟡 Not ready to approve
Queue durability, retry limits, opt-out handling, KYC copy, and KYB reconciliation contain unresolved correctness and reliability issues.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
Adds durable, localized email notifications for ramp completion and Avenia verification outcomes, delivered asynchronously through Resend.
Changes:
- Adds the email queue, dispatcher, templates, retries, and Resend transport.
- Adds signed Avenia webhooks plus KYB polling reconciliation.
- Adds configuration, migration, tests, scripts, and security documentation.
File summaries
| File | Description |
|---|---|
supabase/config.toml |
Documents local Resend SMTP setup. |
packages/shared/src/services/brla/types.ts |
Adds webhook and verification types. |
packages/shared/src/services/brla/mappings.ts |
Maps Avenia webhook endpoints. |
packages/shared/src/services/brla/brlaApiService.ts |
Adds webhook and key APIs. |
docs/security-spec/README.md |
Indexes the Resend specification. |
docs/security-spec/05-integrations/resend.md |
Defines email security invariants. |
docs/security-spec/05-integrations/brla.md |
Documents webhook security behavior. |
docs/features/alfredpay-kyc-notification-gap.md |
Records the Alfredpay follow-up. |
docs/architecture/email-notifications.md |
Describes notification architecture. |
apps/api/src/scripts/register-avenia-webhook.ts |
Registers Avenia webhooks. |
apps/api/src/scripts/preview-emails.ts |
Generates transactional previews. |
apps/api/src/scripts/auth-email-templates.ts |
Generates Supabase templates. |
apps/api/src/models/kycCase.model.ts |
Adds the entity association helper. |
apps/api/src/models/index.ts |
Registers notification associations. |
apps/api/src/models/emailNotification.model.ts |
Defines queued-email records. |
apps/api/src/index.ts |
Starts notification workers. |
apps/api/src/database/migrations/055-create-email-notifications-table.ts |
Creates the queue table. |
apps/api/src/config/vars.ts |
Adds email and webhook configuration. |
apps/api/src/config/express.ts |
Mounts the raw-body webhook route. |
apps/api/src/api/workers/notification-dispatch.worker.ts |
Schedules queued delivery. |
apps/api/src/api/workers/kyb-status.worker.ts |
Reconciles KYB outcomes. |
apps/api/src/api/services/phases/phase-processor.ts |
Triggers completion notifications. |
apps/api/src/api/services/email/types.ts |
Defines locales and payloads. |
apps/api/src/api/services/email/templates/verification-status.ts |
Renders verification emails. |
apps/api/src/api/services/email/templates/ramp-completed.ts |
Renders completion emails. |
apps/api/src/api/services/email/templates/layout.ts |
Provides the shared layout. |
apps/api/src/api/services/email/templates/index.ts |
Dispatches template rendering. |
apps/api/src/api/services/email/resend.transport.ts |
Sends mail through Resend. |
apps/api/src/api/services/email/ramp-completion.ts |
Builds completion payloads. |
apps/api/src/api/services/email/notification.service.ts |
Implements queue processing. |
apps/api/src/api/services/email/index.ts |
Exports email services. |
apps/api/src/api/services/avenia/webhook-signature.ts |
Verifies webhook signatures. |
apps/api/src/api/services/avenia/webhook-signature.test.ts |
Tests signature verification. |
apps/api/src/api/services/avenia/verification-notifications.ts |
Maps terminal outcomes. |
apps/api/src/api/services/avenia/avenia-customer.service.ts |
Resolves notification owners. |
apps/api/src/api/services/avenia/__tests__/fixtures.ts |
Provides cryptographic fixtures. |
apps/api/src/api/services/auth/supabase.service.ts |
Resolves user email locale. |
apps/api/src/api/routes/v1/avenia-webhook.route.ts |
Defines the webhook endpoint. |
apps/api/src/api/controllers/brla.controller.ts |
Persists KYB attempt IDs. |
apps/api/src/api/controllers/avenia-webhook.controller.ts |
Handles signed events. |
apps/api/src/api/controllers/avenia-webhook.controller.test.ts |
Tests webhook handling. |
apps/api/package.json |
Adds email utility scripts. |
apps/api/.gitignore |
Ignores generated previews. |
apps/api/.env.example |
Documents required variables. |
Review details
Comments suppressed due to low confidence (5)
apps/api/src/api/services/email/notification.service.ts:96
- Delivery ignores the existing
notification_preferences.email_enabledsetting, so a user who explicitly disables email through the preferences API still receives these notifications. Check the profile's global email preference (and the relevant per-type preference, if applicable) before rendering/sending, and record opted-out rows as skipped.
async function deliver(notification: EmailNotification): Promise<void> {
const user = await User.findByPk(notification.userId);
apps/api/src/api/services/email/notification.service.ts:119
- The database claim prevents concurrent workers from sending simultaneously, but it does not prevent a duplicate when Resend accepts the request and the process dies (or the following DB update fails) before
Sentis persisted. Stale-claim recovery will send the same row again. Pass a deterministic provider idempotency key derived fromnotification.idthrough the transport so retries cannot create a second email.
const messageId = await sendEmail({ ...rendered, to: user.email });
apps/api/src/api/services/email/notification.service.ts:153
- Every stale claim is reset to
failedregardless of its attempt count, while the claim query has noattempts < MAX_ATTEMPTSguard. A process that repeatedly dies during the fifth send can therefore be claimed for attempts 6, 7, and beyond and never reachesabandonedor triggers the Slack alert. Mark stale rows at the cap as abandoned/alerted and only retry rows below the cap.
{ nextAttemptAt: new Date(), status: NotificationStatus.Failed },
apps/api/src/api/workers/kyb-status.worker.ts:81
- A terminal provider response only enqueues mail; it never changes this case's pending status or otherwise excludes it from the next query. Unless the user separately calls the status endpoint, the worker polls the same settled attempt every hour for up to 60 days. Persist the provider-confirmed terminal state after enqueueing, or exclude attempts that already have a terminal notification.
await enqueueVerificationNotification(attempt, profileId);
apps/api/src/api/services/email/notification.service.ts:163
- The core queue state machine has no automated coverage: claiming/locking, allowlist skips, missing-key behavior, backoff/exhaustion, and stale-claim recovery can regress without detection. Add focused service tests, including a stale fifth attempt and a send accepted before persistence, since these are the feature's reliability guarantees.
export async function dispatchPendingNotifications(): Promise<void> {
- Files reviewed: 44/45 changed files
- Comments generated: 6
- Review effort level: Medium
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
Resolved 7 conflicts, all where staging's refactors met the email notification work: - models/index.ts, vars.ts, phase-processor.ts, alfredpay-customer.service.ts: both sides added adjacent declarations; kept both. Dropped the TaxId associations, since staging removed that model in the provider_customers cutover. - security-spec/README.md: kept the Resend row, dropped the Stellar Anchors row that staging deleted along with the spec file. - alfredpay.md: both sides added an invariant 24; kept staging's at 24 and renumbered the notification invariants to 25 and 26. - brla.md: took staging's anonymous-quote row, which matches the merged registerRamp (rejects only when quote and request users both exist and differ); this branch's row described pre-cutover behavior. Kept the four added webhook threat rows.
There was a problem hiding this comment.
🟡 Changes recommended
Delivery idempotency, retry limits, provider validation, preference enforcement, and worker-bounding issues remain unresolved.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Review details
Suppressed comments (14)
apps/api/src/api/workers/kyb-status.worker.ts:83
- Validate that Avenia returned the requested attempt before notifying. This worker currently accepts any
attempt.id, so a malformed/misrouted provider response can send another attempt's outcome and rejection reason to this case's profile. The authenticated status path already rejects this atbrla.controller.ts:868-871; apply the same check here.
apps/api/src/api/workers/kyb-status.worker.ts:52 - Settled unattended cases remain selected every hour: this worker enqueues the email but deliberately never updates
KycCase.status, and deduplication only suppresses another row, not another provider request. A terminal case can therefore be polled roughly 1,440 times over the 60-day window; additionally thisfindAllis unbounded. Exclude attempt IDs that already have a verification notification (or persist an observation marker) and cap each sweep.
packages/shared/src/services/brla/brlaApiService.ts:426 - This fetch has no timeout. On a cold cache or signature miss, the public webhook awaits it, and all concurrent verifications coalesce onto the same potentially hung promise; an Avenia/network stall can therefore hold webhook connections for minutes and block key refresh. Add an abort timeout consistent with other provider clients.
apps/api/src/api/services/email/ramp-completion.ts:39 - This records enqueue time, not completion time. The recovery sweep can run up to 24 hours after the ramp completed, so recovered emails show a materially incorrect date. Read the
completeentry's timestamp fromphaseHistory(and select that field in the reconciliation query), withupdatedAtonly as a legacy fallback.
completedAt: new Date().toISOString(),
apps/api/src/api/workers/alfredpay-status.worker.ts:30
- This hourly job can overlap itself during a provider outage. A cycle may perform up to 750 sequential calls, each with a 30-second Alfredpay timeout, while this CronJob leaves
waitForCompletiondisabled; later hourly ticks then poll the same nonterminal accounts concurrently. Enable overlap prevention/a running guard and choose a batch size that finishes within the schedule interval.
docs/architecture/email-notifications.md:4 - This new one-file
docs/architecture/tree conflicts withdocs/README.md:8-15,55-69, which requires general architecture documents directly underdocs/, indexed there, and warns against duplicating security-spec content and implementation inventories. Move this to a supporteddocs/architecture-*.mdlocation, trim duplicated canonical material, update its links, and add it to the index.
docs/features/alfredpay-kyc-notification-gap.md:4 - This one-off follow-up file violates the documentation lifecycle in
docs/README.md:55-76: general docs belong at the root under a supported kind, and completed implementation summaries/open-gap journals should not be retained as standalone feature files. Fold the durable locale gap into the canonical product/architecture document or an explicitly scoped proposal, then remove this file and update links.
docs/architecture/email-notifications.md:304 - The lifecycle diagram is off by one relative to the implementation and the preceding table: attempt 5 schedules the 180-minute retry, and attempt 6 is abandoned. As written, this architecture document says the final backoff is never used.
docs/security-spec/05-integrations/brla.md:118 - These additions restart numbering at 25 even though invariant 25 is immediately above, making references ambiguous. Renumber the webhook invariants 26–29.
apps/api/src/api/services/email/notification.service.ts:175 - The dispatcher's tests cover only
nextRetryAt; none exercise claiming/locking, stale recovery, missing-key handling, allowlist enforcement, delivery state transitions, or concurrent workers. These are the feature's load-bearing security and reliability invariants, and the uncapped crash retry and uncertain-send duplicate paths currently pass unnoticed. Add database-backed dispatch tests covering success, failure, crash recovery, and concurrent claims.
export async function dispatchPendingNotifications(): Promise<void> {
if (!config.integrations.resend.apiKey) {
logger.warn("RESEND_API_KEY is not set; leaving pending notifications queued");
return;
}
apps/api/src/api/services/email/templates/ramp-completed.ts:48
- The linked acceptance ticket #1144 specifies a personalized
Hi Name,greeting (and sign-off), but this template starts directly with the transaction body and the new architecture document explicitly defers the greeting. Either implement a trusted name source now or clarify with the ticket owner that this acceptance criterion is intentionally removed before marking the feature ready.
const body: EmailBody = {
details: [
docs/security-spec/05-integrations/resend.md:27
- This code map points auditors to the wrong producer:
ramp.service.tsdoes not enqueue completion mail. The trigger is inphase-processor.ts, and payload construction/enqueueing is inemail/ramp-completion.ts; list those paths so the audit checklist traces the real side effect.
docs/architecture/email-notifications.md:94 - “All fire-and-forget” is incorrect and obscures important failure semantics: the webhook awaits enqueue before acknowledging, the KYB poll awaits it per attempt, and Alfredpay deliberately awaits it before persisting terminal status. Only ramp completion detaches the promise. Describe these as durable queue producers that never send inline instead.
docs/architecture/email-notifications.md:195 - The diagram still names the removed
TaxId.kycAttemptstorage, while this PR's worker readskyc_cases.provider_case_id. Update the state label so the architecture does not contradict the implementation described later in this section.
- Files reviewed: 54/55 changed files
- Comments generated: 4
- Review effort level: Balanced
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
…ed ramps The 055 prefix collides with 055-create-financial-operations on staging -- the exact duplicate-prefix situation MIGRATION_RENAMES in migrator.ts was added to clean up. Renumbering now, while branch-only, avoids another rename shim entry. The backfill closes a first-deploy hazard: reconcileMissedRampCompletedEmails deliberately has no age cutoff, so an empty table would re-enqueue every historical completed ramp and mass-mail users about months-old completions. Tombstoning them as skipped at table creation keeps the sweep anomaly-only.
getEffectiveUserId fills rampState.userId with the credential's linked profile on API-key requests, so the null-userId check alone mailed the partner once per end-customer ramp -- the opposite of the documented behavior. A ramp whose quote carries an api_credential_id now records a skipped tombstone instead, which also stops the hourly reconcile sweep from re-surfacing it forever. The reconciliation integration test now seeds real ramps and asserts the anti-join's semantics instead of running it against an empty database.
quote amounts are DECIMAL(38,18) and come back as "1250.000000000000000000"; the API response path already trims them, the email path did not.
bun's mock.module patches are process-global and leak into every file loaded
afterwards. The two Avenia suites and the dispatch suite left @vortexfi/shared,
the logger, the transport, Slack, and the template renderer stubbed for the
rest of the run. Each now restores the real namespaces in afterAll, matching
the priceFeed convention.
The canary's name check cannot see this stub shape -- an object literal's
{ getInstance: () => ... } infers the name "getInstance" -- so it now also
asserts the singletons are classes, which an object-literal stub is not.
New coverage for the properties the feature leans on but nothing verified:
the transactional SKIP LOCKED claim shape, the non-production recipient
allowlist (absent recipient, empty list, production bypass), the failure
path (backoff scheduling, abandon-at-cap with Slack alert, error-text cap),
the Avenia terminal-status mapping and reason cap (previously only its
Alfredpay twin was tested), the HTML-escaping invariant from the security
spec, and shared's {attemptId} path templating that the KYB poller's URLs
depend on. Also drops a tautological constant self-assertion.
…uristic The status route's catch handled provider errors and local enqueue/persist errors alike, so an enqueue failure whose message contained "not found" reset the customer to Consulted and wiped the observed state. The local writes now carry their own catch. The service-path swallow is raised from info to warn with the stack, since it also hides enqueue failures that block terminal persistence. The universally-fresh-submission-id dedupe claim is softened to match what the retry endpoints actually guarantee.
The poller never writes terminal outcomes back to kyc_cases, so every settled attempt cost one Avenia request per hour until it aged out of the 60-day window. The queue row the enqueue writes is the natural retirement marker: an anti-join against email_notifications excludes those attempts before any provider call. The batch is also capped at 250 cases (oldest writes first), matching the Alfredpay worker's bound.
Duplicates are the common case -- webhook replays and re-polled settled attempts -- and each one paid a Supabase admin API call to resolve a locale that was never used. A key lookup now short-circuits first. Also restores the claim-consumed attempt when the not-configured guard requeues a row, stubs getUserLocale in the fake auth world so integration enqueues stop dialing the neutralized Supabase host, and stubs findOne in the enqueue unit tests to keep them off the database.
- The unauthenticated Avenia webhook route no longer buffers the API's full 20mb body limit before the signature check; events are a few KB, 100kb is generous. - Supabase auth-mail subjects and the Terms/Privacy links now localize for pt-BR (the Dashboard subject field needs a self-contained conditional, and the site serves /pt/ legal pages). - The Avenia contract test sweeps stale contractRun webhooks left by crashed runs before checking the 3-slot cap, and a failed cleanup delete no longer masks the error that failed the test.
docs/README.md allows directory trees only for security-spec/ and api/; general documents live flat as docs/<kind>-<topic>.md and must be indexed. The architecture doc moves accordingly, the alfredpay-kyc-notification-gap follow-up note folds into its open-follow-ups section (it was a 40-line progress journal, which the conventions disallow as a standalone file), and the index and spec links are repaired in the same change.
…path AlfredpayCustomerView exposes alfredPayId, not id; the new error log in the 404-heuristic split referenced a field the view does not have.
Empirically (2-file probe): mock.module mutates already-imported namespace objects in place, so spreading the namespace at restore time copies the stubs back -- the restore was a silent no-op. Snapshots are now plain-object copies taken before the first mock.module call, and the new Avenia enqueue suite resolves its subject in beforeAll because the webhook controller test mocks that exact module path and bun's file execution order is not the CLI order.
Same in-place-mutation flaw fixed for the email suites earlier: spreading a live namespace at restore time copies the stub back, so the fund-ephemeral and offramp-subsidy suites and the FakeSquidRouter harness restore() were silent no-ops. All three now restore from a plain-object snapshot taken before the first mock.module call. The canary additionally checks that shared's getRoute is not a leftover FakeSquidRouter stub, which the name and class-shape checks cannot see.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 71 out of 72 changed files in this pull request and generated 3 comments.
Suppressed comments (6)
apps/api/src/api/workers/kyb-status.worker.ts:74
- This excludes a KYB outcome if
getKybAttemptStatuspersisted the case as Approved/Rejected first (brla.controller.ts:891-900), but that path does not enqueue an email. If the webhook was missed and the client polls first, the reconciliation worker can never recover the notification. Enqueue before that terminal write, or keep these cases eligible while separately preventing pre-deployment backfill mail.
apps/api/src/api/workers/kyb-status.worker.ts:60 - “Oldest writes first” does not make this batch drain: pending attempts are neither updated nor removed, so the same first 250 can be selected every hour for up to 60 days. During a webhook outage, later settled attempts can therefore be starved indefinitely. Use a stable keyset cursor (with wraparound), like
AlfredpayStatusWorker, rather than repeatedly taking the same prefix.
apps/api/src/api/services/email/ramp-completion.ts:90 - This recovery query is unbounded. After a prolonged queue/database outage, it loads every missing completion and then performs a quote lookup plus locale lookup for each one sequentially, so a single hourly cycle can become arbitrarily large. Keep the no-age-cutoff guarantee, but process a stable bounded batch per cycle.
const completed = await RampState.findAll({
docs/architecture-email-notifications.md:8
- This relative link resolves outside
docs/to/security-spec/..., so the architecture document’s security-spec link is broken.
docs/security-spec/05-integrations/resend.md:97 - These audit checks reference identifiers that do not exist in this change: the function is
enqueueRampCompletedEmail, and the migration is062. As written, reviewers following the normative checklist will search the wrong symbols.
docs/architecture-email-notifications.md:435 - The table is created by migration 062, not 055 (the same document correctly names 062 in §4). This file map currently points readers to the wrong migration.
The route persisted Approved/Rejected without ever queuing an email. Once a case is terminal both this route's short-circuit and the KYB worker stop observing the attempt, so a client polling ahead of a lost webhook lost the notification forever. The outcome is queued before the terminal writes -- mirroring the Alfredpay enqueue-before-persist invariant -- and a failed enqueue fails the request, leaving the case pollable.
…empts Three review findings on the hourly sweep: partner-owned cases passed the row filter and could permanently occupy the 250-slot batch (now excluded in the join); a capped oldest-first prefix re-selected the same rows every cycle since pending cases are never modified (now an id-ordered keyset like the Alfredpay sweep); and the returned attempt id was trusted, letting a malformed provider response enqueue another attempt's outcome and reason for this case's profile (now discarded, mirroring the authenticated route's guard). Spec updated in the same change, including its stale enqueueCompletionEmail/055 identifiers.
sendRequest used a bare fetch, so one hung Avenia connection stalled its caller indefinitely -- for cron workers with waitForCompletion that means no future cycle ever runs. Timeouts land in the existing transport-failure path (BrlaApiError status 0) that callers already normalize to a 502.
The recovery query had no limit, so a prolonged outage made a single hourly cycle arbitrarily large (one quote plus locale lookup per missing ramp). 250 oldest-first per cycle, with a log line when the cap is hit; processed ramps gain a queue row and leave the anti-join, so the backlog drains across cycles without giving up the no-age-cutoff guarantee.
The architecture doc's security-spec link broke when the doc moved into docs/ (its ../ now resolved outside the tree), and the section 7 comparison table still named migration 055.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 72 out of 73 changed files in this pull request and generated no new comments.
Suppressed comments (4)
packages/shared/src/services/brla/brlaApiService.ts:414
sendRequestonly casts provider JSON (and even returnsundefinedwhen parsing fails), so this production wrapper does not use the newly addedaveniaWebhooksListSchema. A malformed or partial Avenia response can therefore make the registration script miss an existing subscription, attempt a duplicate registration, or fail with an unrelated property error. Parse the response with the schema here and fail closed before exposing it.
packages/shared/src/services/brla/brlaApiService.ts:449- This assertion does not validate the public-key response at runtime. If Avenia or an intermediary returns a truthy non-string
publicKey, it is cached and every genuine webhook fails signature verification until refresh. Check that the field is a non-empty string before returning it.
apps/api/src/api/services/email/notification.service.ts:231 - The stale-row read and the subsequent abandonment update are not one atomic claim. Because both flow-variant backends run this worker, they can both read the same exhausted
sendingrow and each emit a Slack abandonment alert even though only one transition occurred. Claim/transition stale rows transactionally (for example with a row lock plusSKIP LOCKEDorUPDATE ... RETURNING) and alert only for rows claimed by that transaction.
const exhausted = await EmailNotification.findAll({
where: { ...staleClaim, attempts: { [Op.gte]: MAX_ATTEMPTS } }
});
apps/api/src/database/migrations/062-create-email-notifications-table.ts:6
- The PR deployment notes identify this as migration
055-create-email-notifications-table.ts, but the actual migration is numbered 062 and the in-repository docs also reference 062. Update the PR description so deployers and reviewers do not look for or gate deployment on a nonexistent migration 055.
Users now get an email when a ramp completes and when their KYC/KYB verification is approved or rejected. Previously they had to sit on the page or check back manually.
Two emails, both bilingual (en-US / pt-BR, chosen from the user's locale):
How it works
Nothing sends mail inline. Every trigger just writes a row to a new email_notifications table, and a cron worker is the only sender:
trigger → enqueue row → dispatch worker (every minute) → Resend
That way a failed send never fails the thing that succeeded (a completed ramp, a verification), and nothing is lost if the API restarts mid-send.
Recipient addresses always come from profiles.email for the notification's user — never from a request body or a partner-supplied field, so partner API ramps don't email anyone.
New: Avenia webhook receiver
POST /v1/webhooks/avenia, authenticated by Avenia's RSA-PSS signature over the raw body (mounted ahead of the JSON parser, since the signature covers the exact bytes). Their public key is cached an hour and
refetched once on a verification miss because they rotate it without notice. Register the subscription with bun register:avenia-webhook.
Deploying
Set these before merging to an environment that should actually send:
RESEND_API_KEY= # absent → rows stay queued, nothing sends, nothing lost
EMAIL_FROM_ADDRESS=Vortex Finance support@vortexfinance.co
AVENIA_WEBHOOK_URL= # only for the registration script
Also: one migration (055-create-email-notifications-table.ts), and vortexfinance.co needs to be verified in Resend before anything leaves.