v0.6.92: enrichment table column type, table run fixes, scheduled jitter, hosted-key queueing

[TheodoreSpeaks]

• feat(tables): Add enrichment table column type (feat(tables): Add enrichment table column type #4752)
• fix(tables): workflow-column run fixes + bounded run-N-rows (fix(tables): workflow-column run fixes + bounded run-N-rows #4754)
• improvement(schedules): jitter scheduled execution starts by 0-30s (improvement(schedules): jitter scheduled execution starts by 30s #4750)
• feat(tools): queue hosted-key tool calls instead of failing with 429 (feat(tools): queue hosted-key tool calls instead of failing with 429 #4416)
• log(db): Add db failure cause log message (log(db): Add db failure cause log message #4749)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	May 27, 2026 9:29am

[cursor]

PR Summary

High Risk
Touches core table execution (cascade locking, dispatch limits, enrichment billing via hosted keys) and new paid data paths; regressions could mis-run rows, over/under-bill, or leave stale cell state.

Overview
Adds Clay-style table enrichments: a registry-driven catalog (work-email, phone-number, company-domain, company-info) that creates enrichment workflow groups, maps inputs to columns, and runs a hosted-tool provider cascade per row (with usage billing) instead of a workflow. The UI gets an Enrichments slideout, + New column → Enrichments, group meta headers with enrichment icons, and completed empty hits shown as Not found.

Workflow / run behavior is tightened: optional inputMappings from Start-block fields to table columns, cell-scoped context-menu run/re-run for a single output group, Run N empty rows caps (RunLimit) through API/SSE/dispatcher, and a post-cascade loop that only re-drives queued markers after lock release. Enrichment and workflow fixes include skipping optimistic stamps for capped runs, ignoring capped dispatches in the ahead-of-cursor overlay, and syncing autoRun when adding groups so staged groups don’t flash a no-op dispatch.

Ops / polish: scheduled executions get 0–30s random enqueue jitter; Copilot gains list_enrichments / add_enrichment; shared CollapsibleCard and an internal /add-enrichment command doc the pattern.

^{Reviewed by Cursor Bugbot for commit 92fd17c. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 92fd17c. Configure here.}

[greptile-apps]

Greptile Summary

This PR bundles five distinct changes: a new enrichment table column type (registry-backed provider cascade with billing), workflow-column run fixes (bounded dispatches, inputMappings rename cascade, enrichment isDone semantics), a 0–30 s schedule execution jitter, Redis-backed FIFO queuing for hosted-key 429 recovery, and a DB failure-cause log helper.

• Enrichment column type: a code-defined enrichment catalog runs a provider cascade per row, writes output columns, and bills hosted-key cost via recordUsage; the isDone enrichment guard prevents billable re-runs on no-match rows by treating completed (even with empty outputs) as terminal.
• Hosted-key queuing: HostedKeyQueue (Redis list + per-ticket heartbeat) wraps acquireKey in a FIFO wait loop with heartbeat-aware sleeping and AbortSignal-bounded timeouts, replacing the previous immediate 429.
• Dispatcher bounded runs: DispatchLimit + processedCount on tableRunDispatches supports "run first N rows" dispatch; budget resets to zero on batch-enqueue failure to avoid over-counting transient errors.

Confidence Score: 4/5

Safe to merge; all three findings are edge cases unlikely to trigger in normal usage and do not affect correctness of the happy path.

The queue list TTL is only refreshed on new enqueues, not during the active head waiter's heartbeat cycle. Under a very long wait with no new callers the list can expire, causing all waiters to bypass FIFO ordering simultaneously. The interruptibleSleep TOCTOU window is real but bounded to one extra sleep period by design. The stale enrichmentId field carried across outer-loop iterations is currently harmless but could mislead future code that inspects the payload.

apps/sim/lib/core/rate-limiter/hosted-key/queue.ts (queue list TTL refresh) and apps/sim/background/workflow-column-execution.ts (stale enrichmentId in re-drive payload)

Important Files Changed

Filename	Overview
apps/sim/lib/core/rate-limiter/hosted-key/queue.ts	New Redis-backed FIFO queue for hosted-key acquisitions; Lua EVAL script atomically reaps dead heads. Queue list TTL (600 s) is only refreshed on enqueue, not during active waiting, which can collapse ordering under long waits with no new callers.
apps/sim/lib/core/rate-limiter/hosted-key/hosted-key-rate-limiter.ts	Replaces immediate 429 returns with a FIFO-queued wait loop backed by the new HostedKeyQueue; heartbeat-aware sleeping, interruptible sleeps, and telemetry events added. Minor TOCTOU window exists in interruptibleSleep between AbortSignal check and addEventListener.
apps/sim/background/workflow-column-execution.ts	Adds outer re-drive loop for queued markers after cascade lock release; new enrichment-group fast path calls registry + runEnrichment instead of a full workflow. Stale enrichmentId can carry forward in currentPayload across iterations but is harmless under current routing logic.
apps/sim/lib/table/dispatcher.ts	Adds DispatchLimit/processedCount support for bounded-row runs; row-cap accounting resets correctly on batch-enqueue failure to prevent budget exhaustion from transient errors.
apps/sim/app/api/schedules/execute/route.ts	Adds 0-30 s random jitter to scheduled execution enqueues; concurrencyKey prevents duplicate runs so the jitter does not introduce double-execution.
apps/sim/enrichments/run.ts	Clean provider cascade runner: 404 treated as no-match, other failures fall through, hosted-key cost accumulated across providers.
apps/sim/lib/table/workflow-columns.ts	Enrichment isDone logic correctly prevents billable no-match re-runs; pickNextEligibleGroupForRow now correctly propagates isManualRun flag from queued markers.
packages/db/schema.ts	Adds limit (JSONB) and processedCount columns to tableRunDispatches; new enrichment value added to usageLogSourceEnum. Migration files present.
apps/sim/lib/workflows/executor/execution-core.ts	Adds describeErrorCause to walk Drizzle/driver error chains and surface PostgreSQL diagnostics alongside the top-level execution failure log.

Sequence Diagram

sequenceDiagram
    participant Caller as Tool Caller
    participant RL as HostedKeyRateLimiter
    participant Q as HostedKeyQueue (Redis)
    participant Bucket as Token Bucket

    Caller->>RL: acquireKey(provider, prefix, config, workspaceId, signal)
    RL->>Q: enqueue(provider, workspaceId, ticketId)
    Q-->>RL: position and enabled

    loop Poll until head
        RL->>Q: checkHead(provider, workspaceId, ticketId)
        Note over Q: Lua EVAL reap dead head check position
        Q-->>RL: waiting or head or missing
        RL->>Q: refreshHeartbeat every 10s
    end

    RL->>Bucket: waitForActorCapacity loop until token available
    Bucket-->>RL: capacity granted

    RL-->>Caller: success true key

    Note over RL,Q: finally dequeue ticketId
    RL->>Q: dequeue(provider, workspaceId, ticketId)

Loading

_{Reviews (1): Last reviewed commit: "fix(tables): workflow-column run fixes +..." | Re-trigger Greptile}

[greptile-apps]

Queue list TTL not extended by active head waiter

QUEUE_LIST_TTL_SECONDS (600 s) is only refreshed via pipeline.expire inside enqueue. The refreshHeartbeat method extends the per-ticket key (30 s TTL, refreshed every 10 s) but never touches the queue list key. If the head waiter's execution budget exceeds 10 minutes (ABSOLUTE_MAX_QUEUE_WAIT_MS derives from getMaxExecutionTimeout(), which can be much longer for enterprise async) and no new caller arrives to refresh the list TTL, the queue list expires. Every waiter's next checkHead call sees "missing" (list gone → LINDEX returns nil → script returns "missing") and all proceed to the bucket simultaneously, collapsing FIFO ordering into concurrent bucket racing. Extending the queue list TTL inside refreshHeartbeat (alongside the ticket key) would close this gap.

[greptile-apps]

TOCTOU window in interruptibleSleep between signal.aborted check and addEventListener

If the signal fires between the if (signal.aborted) guard at the top and the signal.addEventListener('abort', onAbort, { once: true }) call at the bottom, the abort event will not be delivered to the listener (it already fired before the listener was registered). The sleep then runs to full ms duration instead of resolving early. The call-site comment notes callers re-check their budget after each sleep, so the practical impact is bounded to one extra sleep period (at most HEARTBEAT_REFRESH_INTERVAL_MS = 10 s in the bucket-wait path). Adding a signal.aborted re-check immediately after addEventListener would eliminate the window entirely.

[greptile-apps]

Stale enrichmentId carried forward when re-driving a workflow group after an enrichment group

The outer re-drive loop spreads ...currentPayload when building the next iteration's payload, which means enrichmentId from a prior enrichment group persists if the next group is a workflow group (next.enrichmentId is undefined and is never explicitly cleared). The current runWorkflowAndWriteTerminal implementation routes on group.type === 'enrichment' && group.enrichmentId (using the schema-fresh group object, not payload.enrichmentId), so this stale field does not cause incorrect routing today. However, if any future code path reads payload.enrichmentId as a signal for "this is an enrichment run", the mis-attribution would silently trigger the wrong branch. Explicitly setting enrichmentId: next.enrichmentId (or enrichmentId: undefined) when constructing currentPayload would keep the payload consistent with the target group.

[TheodoreSpeaks]

Greptile and bugbot issues are legit but not actually that impactful. Will take these in a followup pr.