Skip to content

[core] Pin correlation-id draw order to event-log order (#3700) (backport to stable) - #3719

Draft
VaguelySerious wants to merge 1 commit into
backport/pr-3406-to-stablefrom
backport/pr-3700-to-stable
Draft

[core] Pin correlation-id draw order to event-log order (#3700) (backport to stable)#3719
VaguelySerious wants to merge 1 commit into
backport/pr-3406-to-stablefrom
backport/pr-3700-to-stable

Conversation

@VaguelySerious

Copy link
Copy Markdown
Member

Manual backport of #3700 (cherry-pick 9b1b8c71) to stable. The automated job could not do it: every Backport to stable run since 2026-08-21 18:31 fails because the AI Gateway key is over its budget (API key budget exceeded. Current spend: $10001.21, limit: $10000.00), including the forced workflow_dispatch run, which resolved the SHA and cherry-picked correctly before dying at conflict resolution.

Stacked on #3718 (backport of #3406), which this commit cannot terminate without. Merge that first; this PR's diff is against it.

Why the prerequisite

#3700's turnstile waits on any lower armed barrier, a wider relation than gatesOn, and its deadlock-freedom argument rests on #3406 having removed computeResolvesOnItsOwn's unconditional step -> true shortcut. Cherry-picked onto plain stable, this commit's own turnstile termination pin fails with Unconsumed event: step_created: the step spinner is reported self-resolving, so hasParkedCommittedDelivery blocks the dispenser and the parked chain never opens.

Adaptations

  • workflow.ts: stable has no generateUlid const and installs the raw ulid factory as the STABLE_ULID global (upstream installs the counting generateUlid). Wrapped as countingUlid(seedTime?), preserving the factory signature, so mintCount counts every draw from the run's sequence including serialization-driven stream ids (the termination argument for quiesceEarlierCascades depends on that) without changing which seed time stream ids get on stable.
  • private.ts: the gatesOn doc hunk carries over via [core] Keep step results ordered behind waits parked on unread hook payloads (#3406) (backport to stable) #3718. isLogOrderDrawsEnabled, quiescenceTurn, quiesceEarlierCascades and awaitEarlierDeliveries are byte-identical to main.
  • test-support/orchestrator-context.ts does not exist on stable (that extraction is main-only) and is dropped. The mintCount wiring it carries applies to the local setupWorkflowContext copies in the four test files the commit already touches, which is where the cherry-pick put it.
  • docs/content/docs/v5/configuration/runtime-tuning.mdx does not exist on stable, whose docs tree has no v5/configuration directory. Dropped, matching Backport #3554: [core] Pin draw order to event-log order: retire delivery barriers through one ordered dispenser #3629 which carried no docs files for the same reason. WORKFLOW_LOG_ORDER_DRAWS stays documented on main.
  • .gitignore: took only this commit's addition (the e2e-diagnostics-*.json sidecar entry, whose writer does exist on stable), not the main-only event-log-race-repro-* block the conflict dragged in.
  • Changeset downgraded minor -> patch for the maintenance line, matching Backport #3554: [core] Pin draw order to event-log order: retire delivery barriers through one ordered dispenser #3629.

Verification

Not run locally: the e2e lanes (they need a built workbench) and the Event Log Race Repro lane, which is the measurement that matters for this change. CI on this PR is the check.

@changeset-bot

changeset-bot Bot commented Aug 21, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: cbe086c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
Name Type
workflow Patch
@workflow/core Patch
@workflow/world-testing Patch
@workflow/builders Patch
@workflow/cli Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/vitest Patch
@workflow/web-shared Patch
@workflow/web Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, v0 Aug 21, 2026 8:26pm
example-nextjs-workflow-webpack Ready Ready Preview, v0 Aug 21, 2026 8:26pm
example-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workbench-astro-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workbench-express-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workbench-fastify-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workbench-hono-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workbench-nestjs-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workbench-nitro-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workbench-nuxt-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workbench-python-workflow Error Error v0 Aug 21, 2026 8:26pm
workbench-sveltekit-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workbench-tanstack-start-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workbench-vite-workflow Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workflow-docs Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workflow-swc-playground Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workflow-tarballs Ready Ready Preview, v0 Aug 21, 2026 8:26pm
workflow-web Ready Ready Preview, v0 Aug 21, 2026 8:26pm

@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

Summary

Passed Failed Skipped Total
✅ ▲ Vercel Production 1099 0 78 1177
✅ 💻 Local Development 1198 0 86 1284
✅ 📦 Local Production 1198 0 86 1284
✅ 🐘 Local Postgres 1198 0 86 1284
✅ 🪟 Windows 107 0 0 107
❌ 🌍 Community Worlds 83 105 9 197
✅ 📋 Other 606 0 36 642
Total 5489 105 381 5975

❌ Failed Tests

🌍 Community Worlds (105 failed)

redis (20 failed):

  • hookWorkflow is not resumable via public webhook endpoint | wrun_01M0JZY8PG683DHP2RAZEAMXBN
  • parallelStepsThenWebhookWorkflow - no hook_conflict from same-tick replay race | wrun_01M0JZYMAWNVRX7HJT9GP6TQ7H
  • sleepingWorkflow | wrun_01M0JZZHQ8RJWRRKMTE3PG7QQ0
  • outputStreamWorkflow negative startIndex (reads from end)
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns correct index after stream completes
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns -1 before any chunks are written
  • outputStreamWorkflow - getTailIndex and getStreamChunks getStreamChunks returns same content as reading the stream
  • error handling serialization failures step-argument serialization failure is catchable in workflow code
  • error handling serialization failures uncaught step-argument serialization failure fails the run as USER_ERROR without redelivery retries
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01M0K07QH3YDQAGK4JNR0RP4CF
  • hookGetConflictWorkflow - awaiting hook.getConflict() registers hook without payload | wrun_01M0K084KKRZZJZ5TST2H1YZKG
  • hookGetConflictThenStepParallelWorkflow - hook.getConflict() continuation step runs alongside other steps | wrun_01M0K08DQJPKWR4FW21JHKXSV3
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_01M0K08V9B0R6JK0CT49K4HX05
  • hookClaimOnlyMutexWorkflow - hook works as a pure run mutex without payload data | wrun_01M0K09Q7PCDZ3WV7BN0QD37S2
  • hookAdoptOwnerResultWorkflow - duplicate adopts the owner result via conflict.returnValue | wrun_01M0K09TZ5YA2M2ADEWZQ56GHY
  • hookSignalOwnerWorkflow - duplicate forwards its payload to the owner via resumeHook | wrun_01M0K0A3G12JJT1J0M77X8AH9Y
  • hookSupersedeOwnerWorkflow - duplicate cancels the owner and claims the released token | wrun_01M0K0A8Y7XWVN07MN0GPEQEW2
  • resume-or-start route pattern - resumeHook retried after start() reaches the new run | wrun_01M0K0AH65SKVW4TMGENTJDPVG
  • pages router sleepingWorkflow via pages router
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_01M0K0G38TA9BNEA1QDC7Y5NPD

turso (85 failed):

  • addTenWorkflow | wrun_01M0JZX259EV4WP3Y9RRGZGBXW
  • addTenWorkflow | wrun_01M0JZX259EV4WP3Y9RRGZGBXW
  • deploymentId: 'latest' is a no-op in non-Vercel worlds
  • wellKnownAgentWorkflow (.well-known/agent) | wrun_01M0JZZ6Q5XZHR7Q4DB657N38F
  • should work with react rendering in step
  • promiseAllWorkflow | wrun_01M0JZX8TWKHTVP35YMDZXNDR8
  • promiseRaceWorkflow | wrun_01M0JZXDJ2SMAT55E5YY8Q6S5Z
  • promiseAnyWorkflow | wrun_01M0JZXHB0ZRZ12KEGRNXEYA5E
  • importedStepOnlyWorkflow | wrun_01M0JZZJKSNPCAMNY3WMKBS9BC
  • readableStreamWorkflow | wrun_01M0JZXKNKHPEANXWX8RH64FCB
  • hookWorkflow | wrun_01M0JZY0491DF4N627B5HWZTPW
  • hookWorkflow is not resumable via public webhook endpoint | wrun_01M0JZY8PG683DHP2RAZEAMXBN
  • webhookWorkflow | wrun_01M0JZYD2Q5TCJZEZYFNVAKF0G
  • parallelStepsThenWebhookWorkflow - no hook_conflict from same-tick replay race | wrun_01M0JZYMAWNVRX7HJT9GP6TQ7H
  • sleepingWorkflow | wrun_01M0JZZHQ8RJWRRKMTE3PG7QQ0
  • parallelSleepWorkflow | wrun_01M0K000XSD6XKX8TPHYQHA0T0
  • sleepWinsRaceWorkflow | wrun_01M0K004F6W32F5CFBEY4PYJV9
  • stepWinsRaceWorkflow | wrun_01M0K007ZEYKWB5K70SG939VCJ
  • nullByteWorkflow | wrun_01M0K00BJXNV06ZCDBCBE1GNQ6
  • workflowAndStepMetadataWorkflow | wrun_01M0K00DXZWT193WHHQ8MQQS20
  • outputStreamWorkflow no startIndex (reads all chunks)
  • outputStreamWorkflow positive startIndex (skips first chunk)
  • outputStreamWorkflow negative startIndex (reads from end)
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns correct index after stream completes
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns -1 before any chunks are written
  • outputStreamWorkflow - getTailIndex and getStreamChunks getStreamChunks returns same content as reading the stream
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions | wrun_01M0K02T1CJYTE6T77C3D26TB2
  • writableForwardedFromWorkflowWorkflow | wrun_01M0K037CEPPMR0NVEE7STM0SH
  • writableForwardedFromStepWorkflow | wrun_01M0K03BEFT517BBVHGXBVRC0C
  • fetchWorkflow | wrun_01M0K03EXGT0P6Z25DM0P0EFK9
  • promiseRaceStressTestWorkflow | wrun_01M0K03JD1ET78ZQJMMVQM249S
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • error handling serialization failures step-argument serialization failure is catchable in workflow code
  • error handling serialization failures uncaught step-argument serialization failure fails the run as USER_ERROR without redelivery retries
  • error handling not registered WorkflowNotRegisteredError fails the run when workflow does not exist
  • error handling not registered StepNotRegisteredError fails the step but workflow can catch it
  • error handling not registered StepNotRegisteredError fails the run when not caught in workflow
  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_01M0K07BPMPWDGTPPYBVG2PPT7
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01M0K07QH3YDQAGK4JNR0RP4CF
  • hookGetConflictWorkflow - awaiting hook.getConflict() registers hook without payload | wrun_01M0K084KKRZZJZ5TST2H1YZKG
  • 'hookGetConflictWithPriorStepWorkflow' - hook.getConflict() does not block step execution | wrun_01M0K087B1JH0F9K47ABYQSMGA
  • 'hookGetConflictWithParallelStepWorkfl…' - hook.getConflict() does not block step execution | wrun_01M0K08B3CQQ6BH580W7DX2FV2
  • hookGetConflictThenStepParallelWorkflow - hook.getConflict() continuation step runs alongside other steps | wrun_01M0K08DQJPKWR4FW21JHKXSV3
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_01M0K08V9B0R6JK0CT49K4HX05
  • hookClaimOnlyMutexWorkflow - hook works as a pure run mutex without payload data | wrun_01M0K09Q7PCDZ3WV7BN0QD37S2
  • hookAdoptOwnerResultWorkflow - duplicate adopts the owner result via conflict.returnValue | wrun_01M0K09TZ5YA2M2ADEWZQ56GHY
  • hookSignalOwnerWorkflow - duplicate forwards its payload to the owner via resumeHook | wrun_01M0K0A3G12JJT1J0M77X8AH9Y
  • hookSupersedeOwnerWorkflow - duplicate cancels the owner and claims the released token | wrun_01M0K0A8Y7XWVN07MN0GPEQEW2
  • resume-or-start route pattern - resumeHook retried after start() reaches the new run | wrun_01M0K0AH65SKVW4TMGENTJDPVG
  • hookDisposeTestWorkflow - hook token reuse after explicit disposal while workflow still running | wrun_01M0K0AVJ64E383F2YCX7WF1V6
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars) | wrun_01M0K0BB3EXPNCE7318MFKSKR6
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument | wrun_01M0K0BKZBB2TS9DAQR7S4FXAT
  • closureVariableWorkflow - nested step functions with closure variables | wrun_01M0K0BSTAVRW1R3RVA9G5VKTG
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step | wrun_01M0K0BW9HJ45D322PJ9E2EWNQ
  • health check (queue-based) - workflow and step endpoints respond to health check messages
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly | wrun_01M0K0CAPMDB7A5BMGM1XWT6RK
  • Calculator.calculate - static workflow method using static step methods from another class | wrun_01M0K0CGBB4KKKW7KTMC9VKQB8
  • AllInOneService.processNumber - static workflow method using sibling static step methods | wrun_01M0K0CQD6ED5F782AETR97RX0
  • ChainableService.processWithThis - static step methods using this to reference the class | wrun_01M0K0CWW90ZKK51V4FR0WQ5S2
  • thisSerializationWorkflow - step function invoked with .call() and .apply() | wrun_01M0K0D3QRY4J40KMCJ2JBX7NH
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE | wrun_01M0K0DAFW6S7CKR6KPPAWF5RQ
  • instanceMethodStepWorkflow - instance methods with "use step" directive | wrun_01M0K0DH4YX5C5DJNR8A8BVBKQ
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context | wrun_01M0K0DW27M33NR37JSGYJY6QR
  • stepFunctionAsStartArgWorkflow - step function reference passed as start() argument | wrun_01M0K0E42JJ333PXJFGAKC62CW
  • cancelRun - cancelling a running workflow | wrun_01M0K0EAQCWHSCWPQ5ZSZZC8JG
  • cancelRun via CLI - cancelling a running workflow | wrun_01M0K0EFYRHMTTA4C0KBBYH6VX
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router
  • hookWithSleepWorkflow - hook payloads delivered correctly with concurrent sleep | wrun_01M0K0EQ3KB6MFAK98AN7YARDC
  • hookWithSleepFinalStepWorkflow - step only on final payload | wrun_01M0K0F37NQJJ21P30KM21CAN8
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_01M0K0FCP5DVJ2PH27Y8PX1DVW
  • sleepWithSequentialStepsWorkflow - sequential steps work with concurrent sleep (control) | wrun_01M0K0FQTJ3FYG3NYBMKBHEYEE
  • importMetaUrlWorkflow - import.meta.url is available in step bundles | wrun_01M0K0FYD3WQAMXAHAZTNR7R71
  • metadataFromHelperWorkflow - getWorkflowMetadata/getStepMetadata work from module-level helper (#1577) | wrun_01M0K0G0TZ1D099TC2MCT10KS1
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_01M0K0G38TA9BNEA1QDC7Y5NPD

Details by Category

✅ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 99 0 8
✅ example 99 0 8
✅ express 99 0 8
✅ fastify 99 0 8
✅ hono 99 0 8
✅ nextjs-turbopack 104 0 3
✅ nextjs-webpack 104 0 3
✅ nitro 99 0 8
✅ nuxt 99 0 8
✅ sveltekit 99 0 8
✅ vite 99 0 8
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 101 0 6
✅ express-stable 101 0 6
✅ fastify-stable 101 0 6
✅ hono-stable 101 0 6
✅ nextjs-turbopack-canary 88 0 19
✅ nextjs-turbopack-stable 107 0 0
✅ nextjs-webpack-canary 88 0 19
✅ nextjs-webpack-stable 107 0 0
✅ nitro-stable 101 0 6
✅ nuxt-stable 101 0 6
✅ sveltekit-stable 101 0 6
✅ vite-stable 101 0 6
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 101 0 6
✅ express-stable 101 0 6
✅ fastify-stable 101 0 6
✅ hono-stable 101 0 6
✅ nextjs-turbopack-canary 88 0 19
✅ nextjs-turbopack-stable 107 0 0
✅ nextjs-webpack-canary 88 0 19
✅ nextjs-webpack-stable 107 0 0
✅ nitro-stable 101 0 6
✅ nuxt-stable 101 0 6
✅ sveltekit-stable 101 0 6
✅ vite-stable 101 0 6
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 101 0 6
✅ express-stable 101 0 6
✅ fastify-stable 101 0 6
✅ hono-stable 101 0 6
✅ nextjs-turbopack-canary 88 0 19
✅ nextjs-turbopack-stable 107 0 0
✅ nextjs-webpack-canary 88 0 19
✅ nextjs-webpack-stable 107 0 0
✅ nitro-stable 101 0 6
✅ nuxt-stable 101 0 6
✅ sveltekit-stable 101 0 6
✅ vite-stable 101 0 6
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 107 0 0
❌ 🌍 Community Worlds
App Passed Failed Skipped
✅ mongodb-dev 4 0 3
✅ redis-dev 4 0 3
❌ redis 68 20 0
✅ turso-dev 4 0 3
❌ turso 3 85 0
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 101 0 6
✅ e2e-local-dev-tanstack-start-stable 101 0 6
✅ e2e-local-postgres-nest-stable 101 0 6
✅ e2e-local-postgres-tanstack-start-stable 101 0 6
✅ e2e-local-prod-nest-stable 101 0 6
✅ e2e-local-prod-tanstack-start-stable 101 0 6

📋 View full workflow run

@VaguelySerious
VaguelySerious marked this pull request as ready for review August 21, 2026 20:31
@VaguelySerious
VaguelySerious requested review from a team and ijjk as code owners August 21, 2026 20:31
@pranaygp
pranaygp marked this pull request as draft August 21, 2026 23:57
@pranaygp

Copy link
Copy Markdown
Contributor

Converting to draft so we can confirm that it doesn't cause regression in main beta branch before we re confident about rolling this backport over to stable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants