Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/global-singleton-helper.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/utils': minor
---

Add `globalSingleton()`, which parks a package's process-wide state on `globalThis` so bundled copies of a module in one process share it.
5 changes: 5 additions & 0 deletions .changeset/module-scope-lint-hardening.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/world-testing': patch
---

Annotate the test server's per-run invocation counter as deliberately per-copy, so it passes the module-scope state rule.
8 changes: 8 additions & 0 deletions .changeset/module-scope-state-all-bundled-packages.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
'@workflow/core': patch
'@workflow/world': patch
'@workflow/ai': patch
'@workflow/nest': patch
---

Hold process-wide state on `globalThis` rather than at module scope in the packages that get bundled into the host application's server build, where a bundler compiles one copy of each module per layer. Covers warn-once latches, lazy caches, the VM script and QuickJS asset caches, the dev-server port cache, and step single-flight, whose per-copy map was not actually single-flight. State that is deliberately per-copy is annotated with the reason.
5 changes: 5 additions & 0 deletions .changeset/reuse-runtime-world-for-route-handlers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/core': patch
---

Build the workflow entrypoint's queue handler from the runtime World (`getWorld()`) instead of `getWorldHandlers()`, so a process creates one World rather than two. A stateful World no longer gets duplicate connection pools or queue workers.
5 changes: 5 additions & 0 deletions .changeset/utils-side-effects-free.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/utils': patch
---

Declare `sideEffects: false` so bundlers can drop the unused parts of the barrel from a host application's build.
6 changes: 6 additions & 0 deletions .changeset/world-module-scope-state.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@workflow/world-vercel': patch
'@workflow/world-local': patch
---

Hold process-wide state (the WebSocket transport registry, HTTP connection pools, ULID factories, caches, log-once latches) on `globalThis` instead of at module scope. This de-duplicates state across bundled packages. Fixes WebSocket transport, which was registered in one module state but looked up in another.
39 changes: 39 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -478,6 +478,45 @@ The `executionContext` field on workflow runs is a flexible JSONB/CBOR object th
### Observability data hydration
`packages/core/src/observability.ts` contains `hydrateResourceIO` which strips certain fields (like `executionContext`) before UI display. If you need to display data from stripped fields, extract it before the stripping occurs.

### World packages must not hold mutable module state

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Note

Worth a sentence here on why the rule stops at packages/world-*. @workflow/core is statically imported into the same server build and has always been bundled, and this PR's own reasoning (a run started from a Server Component and consumed in a route handler puts live copies in both the ssr and app-route graphs) applies to it unchanged. It reports 26:

$ node scripts/lint/module-scope-state.mjs packages/core
TOTAL 26
packages/core/src/runtime/step-single-flight.ts:32  const inFlightSteps  (`.set()`)
packages/core/src/vm/script-cache.ts:63             const scriptCache    (`.delete()`)
packages/core/src/serialization/workflow-vm.ts:20   let _encoder         (reassigned)
packages/core/src/runtime/start.ts:83               let hasWarnedLatestNoOp (reassigned)
...

I spot-checked several and they look wasteful rather than wrong: registeredSteps (private.ts:28) is already globalThis-backed, the compile and single-flight caches are only reached from /flow so they stay in one layer, and the duplicated cbor encoders and warn-once latches cost memory and a repeated log. So I am not asking for core in this PR. But as written, someone reading "enforces this across every published packages/world-*" alongside "the class of bug behind it" in the PR description will assume core is covered. packages/next reports 7 and packages/cli 4 for the same reason.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added. AGENTS.md now states where the sweep stops and why, rather than leaving 'every published packages/world-*' to be read as covering core.

I used your spot-check verbatim, since it's more useful than a bare caveat: the step registry is already globalThis-backed, the compile and single-flight caches are only reached from /flow so they stay in one layer, and the remainder cost a duplicated encoder or a repeated warn-once log — wasteful rather than wrong, which is why core isn't gated. @workflow/next and @workflow/cli noted as the same case. Widening the sweep is tracked in #3729, and the note ends by saying a new mutable module-scope binding in core should be treated as suspect even though nothing fails the build.

The globalThis fix from your other comment also brings core down to 22 and next to 6, so those numbers are now closer to the genuine ones.


`@workflow/world-local` and `@workflow/world-vercel` are bundled into the host
application's server build (see `VERCEL_WORLD_DEPENDENCY_PACKAGES` in
`packages/next/src/index.ts`). Bundlers key module identity on
`(resource, layer)`, and Next.js alone compiles `instrument`, app-route, `ssr`
and `edge` as separate module graphs, so one process holds one copy of every
module in these packages **per bundler layer**. A top-level `let`, or a `const`
holding a `Map`, is per-copy state, not the process singleton it reads as. A
duplicated mutex stops mutually excluding; a duplicated registry is a
deterministic miss; duplicated ID generators can fork a sequence.

Hold such state on the World instance where it is per-World, or on `globalThis`
via `globalSingleton()` from `@workflow/utils` where it is genuinely
process-wide. State that is deliberately per-copy needs a
`// per-copy-ok: <why>` annotation. `scripts/lint/module-scope-state.mjs`
enforces this across every published `packages/world-*`, run from
`@workflow/utils`'s test suite (with a local mirror in each world package), so
adding a new world package is covered automatically.

Custom worlds loaded through `WORKFLOW_TARGET_WORLD` are deduped by Node's
module cache and are safe today, but that is a property of how they are loaded,
not of how they are written, and it changed for world-vercel in #3493. Keep them
clean too. The author-facing version of this rule is in
`docs/content/worlds/{v4,v5}/building-a-world.mdx`; keep both versions in sync.

The sweep covers every package that ends up inside the host application's
server build: all published `packages/world-*` (discovered at runtime, so a new
world is covered the day it is added) plus `core`, `world`, `ai` and `nest`,
which are named in `BUNDLED_RUNTIME_PACKAGES` in
`packages/utils/src/module-scope-state.test.ts`. Adding a package that runs in
the host server means adding it to that list: "does this run inside the host's
server bundle" is a judgement, not something to infer from a directory name.

Deliberately outside the sweep, because a single module graph makes the hazard
impossible: `next`, `builders` and `sveltekit` (build-time code), `cli` (its own
process), `web` and `web-shared` (the observability UI), `vitest` (the test
runner's process), and private packages such as `world-sim`.

### Trace context propagation (world-vercel HTTP requests)
Every outgoing HTTP request from `@workflow/world-vercel` to workflow-server (or the queue) MUST explicitly inject W3C trace context so the server can parent its spans to the caller and traces stay correlated end to end. Call `injectTraceContextIntoHeaders(headers)` (from `packages/world-vercel/src/telemetry.ts`) on the outgoing headers, inside the client span when one exists. `makeRequest` in `utils.ts` is the reference implementation. It is a no-op when no OpenTelemetry SDK is registered.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,10 @@ type WorldHandlers = Pick<World, "createQueueHandler" | "specVersion">;
```

<Callout type="warn">
This is SDK infrastructure used by framework adapters and the workflow entrypoint. Application code should use [`getWorld()`](/docs/api-reference/workflow-runtime/get-world) instead.
This is SDK infrastructure used by framework adapters at build time. Runtime routes and application code should use [`getWorld()`](/docs/api-reference/workflow-runtime/get-world) instead.
</Callout>

## Related functions

- [`getWorld()`](/docs/api-reference/workflow-runtime/get-world): Resolve the full World instance at runtime.
- [`workflowEntrypoint()`](/docs/api-reference/workflow-runtime/workflow-entrypoint): The route handler factory built on these handlers.
- [`workflowEntrypoint()`](/docs/api-reference/workflow-runtime/workflow-entrypoint): Create the runtime route handler that shares the full World instance.
Original file line number Diff line number Diff line change
Expand Up @@ -38,5 +38,6 @@ Returns a fetch-style request handler: `(req: Request) => Promise<Response>`.

## Related functions

- [`getWorldHandlers()`](/docs/api-reference/workflow-runtime/get-world-handlers): The build-time World access this handler is built on.
- [`getWorld()`](/docs/api-reference/workflow-runtime/get-world): Resolve the runtime World instance this handler shares with workflow execution.
- [`getWorldHandlers()`](/docs/api-reference/workflow-runtime/get-world-handlers): Access build-time-safe World handlers for framework tooling.
- [`healthCheck()`](/docs/api-reference/workflow-runtime/health-check): Verify the entrypoint processes queue messages end-to-end.
59 changes: 59 additions & 0 deletions docs/content/worlds/v4/building-a-world.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,65 @@ Streams are identified by a combination of `runId` and `name`. Each workflow run

`getChunks` returns a paginated snapshot of currently available chunks (unlike `get` which returns a live `ReadableStream` that waits for new chunks). `getInfo` returns the tail index (last chunk index, 0-based, or `-1` when empty) and whether the stream is complete, which is useful for resolving negative `startIndex` values into absolute positions.

## Process-wide state

Hold state that must be process-wide on `globalThis`, not at module scope.

A World is loaded in one of two ways, and only one of them gives your package a
single module instance:

- **Loaded at runtime.** `WORKFLOW_TARGET_WORLD=@your-org/world-foo` is resolved
with `require()` at runtime, so Node's module cache dedupes it and one process
holds one copy.
- **Bundled.** The host application's bundler compiles your package into its
server build. Bundlers key module identity on `(resource, layer)`, and a
framework routinely builds several server layers. Next.js compiles
`instrument`, app-route, `ssr` and `edge` as separate module graphs. Your
package is then compiled into each one, so a single process holds several
copies of every one of your modules, each with its own module scope.

The two built-in worlds are bundled. A custom world is not today, but that is a
property of how it is loaded rather than of how it is written, and it can change
under you. `@workflow/world-vercel` was external until it wasn't, and every
module-scope variable in it silently became per-copy state.

So a top-level `let` or a `const` holding a `Map` is not the singleton it looks
like:

```typescript
// Wrong: one Map per copy. Writes from one part of the app are invisible to
// another, and a mutex like this simply stops mutually excluding.
const locks = new Map<string, Promise<void>>();
```

Reach for `globalThis` under a `Symbol.for()` key instead, so every copy shares
one object:

```typescript
type WorldState = { locks: Map<string, Promise<void>> };

const StateKey = Symbol.for('@your-org/world-foo//locks/v1');
const store = globalThis as typeof globalThis &
Record<symbol, WorldState | undefined>;

const state: WorldState = (store[StateKey] ??= { locks: new Map() });
```

Version the key. Two releases of your package can end up in one process, and a
key without a version lets an older copy read a state object it does not
understand.

Inside this repository, `globalSingleton()` from `@workflow/utils` does exactly
this and is what the first-party worlds use; the hand-rolled form above is
written out so a world published outside this repository does not need the
dependency. `scripts/lint/module-scope-state.mjs` accepts either.

Better still, keep the state on the World instance your `createWorld()` returns.
Connection pools, caches, and open channels are usually per-World rather than
per-process, and instance state cannot be duplicated by a bundler. Reserve the
global for the few things that are genuinely process-wide: ID generators whose
sequence must not fork, and log-once latches.

## Reference implementations

Study these implementations for guidance:
Expand Down
59 changes: 59 additions & 0 deletions docs/content/worlds/v5/building-a-world.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -373,6 +373,65 @@ If you implement this namespace, observe the following requirements:

See the [Analytics API reference](/docs/api-reference/workflow-runtime/world/analytics) for per-method parameters, row shapes, and `pageInfo` semantics.

## Process-wide state

Hold state that must be process-wide on `globalThis`, not at module scope.

A World is loaded in one of two ways, and only one of them gives your package a
single module instance:

- **Loaded at runtime.** `WORKFLOW_TARGET_WORLD=@your-org/world-foo` is resolved
with `require()` at runtime, so Node's module cache dedupes it and one process
holds one copy.
- **Bundled.** The host application's bundler compiles your package into its
server build. Bundlers key module identity on `(resource, layer)`, and a
framework routinely builds several server layers. Next.js compiles
`instrument`, app-route, `ssr` and `edge` as separate module graphs. Your
package is then compiled into each one, so a single process holds several
copies of every one of your modules, each with its own module scope.

The two built-in worlds are bundled. A custom world is not today, but that is a
property of how it is loaded rather than of how it is written, and it can change
under you. `@workflow/world-vercel` was external until it wasn't, and every
module-scope variable in it silently became per-copy state.

So a top-level `let` or a `const` holding a `Map` is not the singleton it looks
like:

```typescript
// Wrong: one Map per copy. Writes from one part of the app are invisible to
// another, and a mutex like this simply stops mutually excluding.
const locks = new Map<string, Promise<void>>();
```

Reach for `globalThis` under a `Symbol.for()` key instead, so every copy shares
one object:

```typescript
type WorldState = { locks: Map<string, Promise<void>> };

const StateKey = Symbol.for('@your-org/world-foo//locks/v1');
const store = globalThis as typeof globalThis &
Record<symbol, WorldState | undefined>;

const state: WorldState = (store[StateKey] ??= { locks: new Map() });

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Blocking

This snippet fails the lint rule added in this same PR. The rule accepts exactly two things: a globalSingleton(...) initializer, or a // per-copy-ok: annotation. store[StateKey] ??= { ... } is an element-access assignment, so collectDeclarations records store and assignment() reports it:

$ cat probe/src/docs-pattern.ts     # copied verbatim from this section
type WorldState = { locks: Map<string, Promise<void>> };
const StateKey = Symbol.for('@your-org/world-foo//locks/v1');
const store = globalThis as typeof globalThis &
  Record<symbol, WorldState | undefined>;
const state: WorldState = (store[StateKey] ??= { locks: new Map() });

$ node scripts/lint/module-scope-state.mjs probe
probe/src/docs-pattern.ts:4  const store  (field written)
TOTAL 1

packages/utils/src/module-scope-state.test.ts discovers every published packages/world-* at runtime, so a world added to this repo that follows this section verbatim fails its own test, and the failure text tells the author to reach for globalSingleton(), which this section deliberately does not mention. AGENTS.md:446 prescribes globalSingleton(); this page prescribes the hand-rolled key. Same block at docs/content/worlds/v4/building-a-world.mdx:261.

Either teach globalSingleton() here (it is already exported from the published @workflow/utils), or teach the rule to recognize a write rooted at a globalThis alias. The second looks worth doing on its own: packages/core/src/private.ts:23 and packages/next/src/index.ts:58 are already correct globalThis-backed code and the rule flags both.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, and I took the second option you suggested — the rule now recognizes state rooted at globalThis, following one alias hop, so the documented two-statement shape (const store = globalThis as … then store[StateKey] ??= …) passes.

You were right that it's worth doing on its own: packages/core/src/private.ts:23 and packages/next/src/index.ts:58 were both correct globalThis-backed code being flagged. Core drops from 26 findings to 22 and next from 7 to 6, purely false positives removed.

I also did the first thing, in a smaller way: the docs section now says outright that globalSingleton() from @workflow/utils does exactly this and is what the first-party worlds use, with the hand-rolled form written out so a world published outside this repo doesn't need the dependency, and notes the rule accepts either. That closes the AGENTS.md/docs inconsistency you flagged without pushing a @workflow/utils dependency onto third-party world authors.

```

Version the key. Two releases of your package can end up in one process, and a
key without a version lets an older copy read a state object it does not
understand.

Inside this repository, `globalSingleton()` from `@workflow/utils` does exactly
this and is what the first-party worlds use; the hand-rolled form above is
written out so a world published outside this repository does not need the
dependency. `scripts/lint/module-scope-state.mjs` accepts either.

Better still, keep the state on the World instance your `createWorld()` returns.
Connection pools, caches, and open channels are usually per-World rather than
per-process, and instance state cannot be duplicated by a bundler. Reserve the
global for the few things that are genuinely process-wide: ID generators whose
sequence must not fork, and log-once latches.

## Reference implementations

Study these implementations for guidance:
Expand Down
1 change: 1 addition & 0 deletions packages/ai/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@
"dependencies": {
"@ai-sdk/provider": "^3.0.0",
"@workflow/serde": "workspace:^",
"@workflow/utils": "workspace:*",
"zod": "catalog:"
},
"optionalDependencies": {
Expand Down
55 changes: 32 additions & 23 deletions packages/ai/src/agent/telemetry.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { globalSingleton } from '@workflow/utils';
import type { TelemetrySettings } from './durable-agent.js';

// Minimal OTel type shims so we don't depend on @opentelemetry/api at compile time.
Expand Down Expand Up @@ -38,24 +39,30 @@ interface OtelApi {
SpanStatusCode: { ERROR: number };
}

// Lazy-loaded OTel API: self-initializes on first use (item 5)
let otelApi: OtelApi | null = null;
let otelLoadAttempted = false;
// Lazy-loaded OTel API: self-initializes on first use (item 5).
//
// On `globalThis` rather than at module scope because this package is bundled
// into the host application's server build, which gives one copy of this module
// per bundler layer; per-copy state would re-attempt the import once per layer.
const otel = globalSingleton('@workflow/ai//agentTelemetry', 1, () => ({
api: null as OtelApi | null,
loadAttempted: false,
}));

async function ensureOtelApi(): Promise<OtelApi | null> {
if (otelLoadAttempted) return otelApi;
otelLoadAttempted = true;
if (otel.loadAttempted) return otel.api;
otel.loadAttempted = true;
try {
// Dynamic import, since @opentelemetry/api is an optional peer dependency.
// Use Function() to hide the import from bundlers that would fail at
// compile time when the package is absent.
otelApi = await (Function(
otel.api = await (Function(
'return import("@opentelemetry/api")'
)() as Promise<OtelApi>);
} catch {
otelApi = null;
otel.api = null;
}
return otelApi;
return otel.api;
}

/**
Expand All @@ -64,9 +71,9 @@ async function ensureOtelApi(): Promise<OtelApi | null> {
* don't need a separate init step.
*/
function getTracer(telemetry?: TelemetrySettings): Tracer | null {
if (!telemetry?.isEnabled || !otelApi) return null;
if (!telemetry?.isEnabled || !otel.api) return null;
if (telemetry.tracer) return telemetry.tracer as Tracer;
return otelApi.trace.getTracer('ai');
return otel.api.trace.getTracer('ai');
}

// ── Attribute helpers ──────────────────────────────────────────────────
Expand Down Expand Up @@ -130,11 +137,11 @@ function recordErrorOnSpan(span: Span, error: unknown): void {
stack: error.stack,
});
span.setStatus({
code: otelApi?.SpanStatusCode.ERROR ?? 2,
code: otel.api?.SpanStatusCode.ERROR ?? 2,
message: error.message,
});
} else {
span.setStatus({ code: otelApi?.SpanStatusCode.ERROR ?? 2 });
span.setStatus({ code: otel.api?.SpanStatusCode.ERROR ?? 2 });
}
}

Expand Down Expand Up @@ -172,12 +179,12 @@ export async function recordSpan<T>(options: {
fn: (span?: Span) => PromiseLike<T> | T;
}): Promise<T> {
// Self-initialize on first call (item 5)
if (!otelLoadAttempted) {
if (!otel.loadAttempted) {
await ensureOtelApi();
}

const tracer = getTracer(options.telemetry);
if (!tracer || !otelApi) {
if (!tracer || !otel.api) {
return options.fn(undefined);
}

Expand All @@ -192,11 +199,13 @@ export async function recordSpan<T>(options: {
{ attributes: attrs },
async (span) => {
// Capture current context so nested spans parent correctly (item 4).
// otelApi is guaranteed non-null here (checked before startActiveSpan).
const ctx = otelApi!.context.active();
// otel.api is guaranteed non-null here (checked before startActiveSpan).
const ctx = otel.api!.context.active();

try {
const result = await otelApi!.context.with(ctx, () => options.fn(span));
const result = await otel.api!.context.with(ctx, () =>
options.fn(span)
);
span.end();
return result;
} catch (error) {
Expand Down Expand Up @@ -228,12 +237,12 @@ export async function createSpan(options: {
telemetry?: TelemetrySettings;
attributes?: Attributes;
}): Promise<SpanHandle | undefined> {
if (!otelLoadAttempted) {
if (!otel.loadAttempted) {
await ensureOtelApi();
}

const tracer = getTracer(options.telemetry);
if (!tracer || !otelApi) return undefined;
if (!tracer || !otel.api) return undefined;

const attrs = buildAttributes(
options.name,
Expand All @@ -243,9 +252,9 @@ export async function createSpan(options: {

// Capture the active context so the span parents under the caller's
// current span, matching how recordSpan uses context.with().
const parentCtx = otelApi.context.active();
const parentCtx = otel.api.context.active();
const span = tracer.startSpan(options.name, { attributes: attrs }, parentCtx);
const context = otelApi.trace.setSpan(parentCtx, span);
const context = otel.api.trace.setSpan(parentCtx, span);
return { span, context };
}

Expand All @@ -263,8 +272,8 @@ export function runInContext<T>(
handle: SpanHandle | undefined,
fn: () => T
): T {
if (!handle || !otelApi) return fn();
return otelApi.context.with(handle.context, fn);
if (!handle || !otel.api) return fn();
return otel.api.context.with(handle.context, fn);
}

/**
Expand Down
Loading
Loading