docs(temporal) WB-529 submission test plan

packages/execution-core/replay-audit.md

@@ -13,18 +13,19 @@ This document audits every code path reachable from `runGraph` in the sandbox, e
 
 ## Files in the sandbox
 
-The sandbox entry `workflow.ts` re-exports the following. Only `runGraph`, the error module, and the redaction module reach the bundle as runtime code; the rest are types (erased at compile time).
+The sandbox entry `workflow.ts` re-exports the following. Four modules reach the bundle as runtime code: `runGraph`, the start-node resolver, the error module and the redaction module. The rest are types, erased at compile time.
 
-| Source                                             | Kind     | Runtime? |
-| -------------------------------------------------- | -------- | -------- |
-| `packages/execution-core/src/graph-runner.ts`      | function | yes      |
-| `packages/execution-core/src/errors.ts`            | classes  | yes      |
-| `packages/execution-core/src/redact.ts`            | function | yes      |
-| `packages/execution-core/src/execution-context.ts` | type     | no       |
-| `packages/execution-core/src/ports/*.port.ts`      | types    | no       |
-| `packages/types/.../execution-model.ts`            | types    | no       |
+| Source                                              | Kind     | Runtime? |
+| --------------------------------------------------- | -------- | -------- |
+| `packages/execution-core/src/graph-runner.ts`       | function | yes      |
+| `packages/execution-core/src/resolve-start-node.ts` | function | yes      |
+| `packages/execution-core/src/errors.ts`             | classes  | yes      |
+| `packages/execution-core/src/redact.ts`             | function | yes      |
+| `packages/execution-core/src/execution-context.ts`  | type     | no       |
+| `packages/execution-core/src/ports/*.port.ts`       | types    | no       |
+| `packages/types/.../execution-model.ts`             | types    | no       |
 
-The audit therefore focuses on `graph-runner.ts` + `errors.ts` + `redact.ts` (a pure, depth-capped walk over plain objects — no clock, no random, no I/O; see rule 8). Activities and adapters live outside the sandbox — they are covered only as ports the runner calls into.
+The audit therefore focuses on `graph-runner.ts` + `resolve-start-node.ts` + `errors.ts` + `redact.ts` (a pure, depth-capped walk over plain objects — no clock, no random, no I/O; see rule 8). The start-node resolver only filters and maps over `definition.nodes` and reads the caller's in-degree `Map`, so it is deterministic given a deterministic input. Activities and adapters live outside the sandbox — they are covered only as ports the runner calls into.
 
 ## Sources of non-determinism reviewed
 
@@ -45,7 +46,7 @@ The audit therefore focuses on `graph-runner.ts` + `errors.ts` + `redact.ts` (a
 | `Array.prototype.shift` on BFS queue     | Yes             | ✅ Safe — FIFO order is deterministic given a deterministic push order. The push order in `propagate` comes from iterating `successors` (a `Map` value), which is insertion-deterministic.                                                                                                                                                                                                                |
 | Throwing for control flow                | No              | ✅ Safe — the runner does not throw for control flow; failures are reported by return value (`RunGraphOutcome`), which is fully determined by the input. `NodeExecutionError` is a plain `Error` subclass with no side effects in its constructor.                                                                                                                                                        |
 | External clock / wall time               | No              | ✅ Safe — runner does not read time. `events.emitEvent('execution_started', ...)` etc. are activities; the timestamp is recorded by the activity outside the sandbox.                                                                                                                                                                                                                                     |
-| Iteration over `Object.keys`/`values`    | No              | ✅ Safe — runner uses `Map` for stateful collections; `nodeOutputs` is an object but never iterated for control flow (only `{ ...nodeOutputs }` for context cloning, which preserves order).                                                                                                                                                                                                              |
+| Iteration over `Object.keys`/`values`    | Yes             | ✅ Safe — `Object.keys(context.nodeOutputs)` builds the `visibleNodeIds` payload on `node_started`. Own string keys enumerate in insertion order per ECMA-262, and the runner inserts in `definition.nodes` order, so the array is deterministic. No control flow branches on it, and stateful collections use `Map` instead.                                                                             |
 | Module-level initialization side effects | No              | ✅ Safe — `graph-runner.ts` exports only function declarations; no top-level statements that read environment or instantiate stateful objects.                                                                                                                                                                                                                                                            |
 | `errors.ts` `NodeExecutionError`         | Yes             | ✅ Safe — constructor only calls `super(message, { cause })` and sets `this.name`. No `Date.now()` in the message, no UUID minting, no env reads. The `Permanent`/`Transient` subclasses add a literal field and their own name, nothing else.                                                                                                                                                            |
 | `errors.ts` `classifyNodeError`          | Yes             | ✅ Safe — reads two fields off the error and returns a literal. Shape-based on purpose (`instanceof` cannot work across bundled copies of this module), so it never depends on which copy of the class the value came from.                                                                                                                                                                               |

packages/temporal/README.md

@@ -203,6 +203,10 @@ Pre-1.0 the API surface may still move between minor versions. It is reviewed de
 
 Two notes on the moving parts underneath: Temporal's plugin API is marked experimental upstream, and this package is deliberately a thin layer over `SimplePlugin` to keep that exposure small. The package ships as ESM only.
 
+## Testing
+
+[`TESTING.md`](TESTING.md) covers what is tested, what is not, and how CI gates it.
+
 ## License
 
 Apache-2.0

packages/temporal/TESTING.md

@@ -0,0 +1,48 @@
+# Testing
+
+The tested unit is the plugin. That is `WorkflowBuilderPlugin`, the three Activities it registers, and the workflow-side code that its `./workflow` entry point runs in Temporal's V8 sandbox. The three Activities are `executeNode`, `emitEvent` and `updateStatus`.
+
+The reference backend and worker in `apps/` are sample consumers of the plugin. What a node executor does is consumer code. No test covers the full path from HTTP request to database write.
+
+```bash
+pnpm --filter @workflowbuilder/temporal test
+pnpm --filter @workflowbuilder/temporal typecheck        # gates the contract check below
+pnpm --filter @workflow-builder/execution-core test
+```
+
+No Docker, no Temporal cluster, no API key and no database. The replay suite starts a dev server binary. `@temporalio/testing` downloads that binary into the system temp directory, and downloads it again when the cached copy is more than one day old. The suite therefore needs network access on a clean machine, and on every CI runner.
+
+## Replay safety
+
+**Deterministic sandbox code.** [`replay-audit.md`](../execution-core/replay-audit.md) gives a verdict for each reviewed source of non-determinism. It also lists the rules that new sandbox code must follow. [`graph-runner.replay-determinism.test.ts`](../execution-core/src/graph-runner.replay-determinism.test.ts) enforces this. It runs each graph shape several times against the same deterministic mocks, then compares the sequence of port calls. The sequences must be identical. The test needs no server and completes in milliseconds.
+
+**A real run.** [`test/replay/replay.test.ts`](test/replay/replay.test.ts) runs a graph on a real Temporal dev server. It uses Temporal's own bundler, a real `Worker`, and the plugin's three Activities. The store and the node executors are test fixtures. The test counts the scheduled Activities of each type. It expects one `executeNode` per node, one `emitEvent` per emitted event, and one `updateStatus`. Then it runs the same graph with the workflow cache off and expects the same counts. `maxCachedWorkflows: 0` turns the cache off. Every workflow task then replays from the first event instead of continuing from a cached state. A side effect that repeats on replay changes one of the counts.
+
+**A history recorded by older code.** The test replays [`histories/v0-parallel-wave.json`](test/replay/histories/) against the current code. This check fails when the current code would issue a command that the recorded run never made. A Workflow Execution can wait for days, so this check is what makes it safe to change the package between releases. [`test/replay/README.md`](test/replay/README.md) describes how to record a history, and what to do when this check fails. The correct response is different before and after the first release.
+
+**Where the guarantee stops.** Replay does not run an Activity again if Temporal recorded its result. A retry does run it again. The plugin schedules one Activity per node, on a profile that allows two attempts, so a node executor must accept that it can run more than once. `emitEvent` and `updateStatus` run on a profile that allows five attempts. The workflow assigns each event its own `sequence` number, so a store can make that write idempotent. The reference worker does this with `ON CONFLICT (execution_id, sequence) DO NOTHING`. No test covers that write.
+
+## Limitations
+
+- **Cancellation is untested.** No test runs the cancellation branch in `run-workflow.ts`.
+- **Node failure is untested in the sandbox.** The fixture executors in the replay test never fail. The failure branch of the workflow therefore runs only in the `execution-core` tests, and in the Activity tests.
+- **Only one history is recorded.** It covers the parallel-wave shape. No other graph shape has a cross-version check.
+- **No CI job tests against the newest `@temporalio/*`.** Every job installs the pinned versions. A regression in the SDK therefore appears when someone upgrades, not on a pull request.
+- **Temporal's `patched()` is not called anywhere.** There are two documented responses to a change in the emitted command sequence. You can put the new behaviour behind `patched()`, or you can release a major version and drain the runs that are in flight first. Because no `patched()` call exists, draining is the response available today. See [`README.md`](README.md) § "Versioning and replay".
+
+## What the suites pin
+
+| Suite                                                                                          | Pins                                                                                                                                                                                                                                                                                                                                                                                                                | Gated by    |
+| ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- |
+| [`test/replay/`](test/replay/)                                                                 | The replay checks in the replay safety section above                                                                                                                                                                                                                                                                                                                                                                | `test`      |
+| [`test/bundling.test.ts`](test/bundling.test.ts)                                               | That the `./workflow` entry loads in the sandbox. Temporal's bundler rejects a worker-side or client-side import, so such an import fails here instead of in a consumer's worker                                                                                                                                                                                                                                    | `test`      |
+| [`test/activities.test.ts`](test/activities.test.ts)                                           | Executor dispatch, and which failures Temporal may retry. A permanent error becomes a non-retryable `ApplicationFailure`. Also plugin construction: the plugin's Activities do not displace the consumer's, the name stays `workflowbuilder.WorkflowBuilderPlugin`, a bad activity profile fails `Worker.create` instead of the first workflow activation, and a profile with no matching executor writes a warning | `test`      |
+| [`test/api-surface.test.ts`](test/api-surface.test.ts)                                         | The exact runtime exports of all three entry points, and the values of the default profiles. Types are absent on purpose. The last row covers the exports map                                                                                                                                                                                                                                                       | `test`      |
+| [`src/workflow/profile-validation.test.ts`](src/workflow/profile-validation.test.ts)           | Which activity profiles are accepted and which are rejected. A profile is copied at the boundary, so a later change to the caller's map has no effect. The profile keys that have no matching executor are named                                                                                                                                                                                                    | `test`      |
+| [`src/workflow/node-activity-options.test.ts`](src/workflow/node-activity-options.test.ts)     | Timeout, retry and Summary resolution for each node type, the fallback when a type has no profile, and safe handling of a node type named like an `Object.prototype` key                                                                                                                                                                                                                                            | `test`      |
+| [`src/workflow/sequenced-event-emitter.test.ts`](src/workflow/sequenced-event-emitter.test.ts) | Event numbering and write order, driven through the real `runGraph`. Rows must become visible in ascending `sequence` order, or the SSE cursor in the reference backend skips one. One failed write must not drop the rest of the run                                                                                                                                                                               | `test`      |
+| [`test/core-contract.test.ts`](test/core-contract.test.ts)                                     | That the types this package restates match `execution-core`. It checks assignability in both directions, so adding or removing a field breaks the build. The assertions are at type level only                                                                                                                                                                                                                      | `typecheck` |
+| [`execution-core`](../execution-core/)                                                         | The graph runner itself: topological scheduling, error policy, start-node resolution, template resolution, redaction and error classification                                                                                                                                                                                                                                                                       | `test`      |
+| The published package                                                                          | `publint` and `arethetypeswrong` check the package as it would publish. Three subpaths with their own types is where an exports map breaks without warning                                                                                                                                                                                                                                                          | CI only     |
+
+[`pr-check.yml`](../../.github/workflows/pr-check.yml) runs these in two jobs. The `temporal` job covers this package. The `execution` job covers `execution-core` and the reference apps. Both jobs trigger only on pull requests into `main`, so they do not gate a `release/*` pull request. [`release-temporal.yml`](../../.github/workflows/release-temporal.yml) runs the same sequence again on the release tag, before it publishes.

packages/temporal/test/replay/README.md

@@ -6,7 +6,7 @@ A workflow can wait for days, so this is the guard that lets the package be edit
 between releases at all.
 
 `replay.test.ts` is the harness. It starts a real Temporal (an in-memory dev server via
-`@temporalio/testing`), runs a graph, and then does three separate things with what came
+`@temporalio/testing`), runs a graph, and then does four separate things with what came
 back:
 
 1. **Counts the scheduled activities per type.** One `executeNode` per node, one
@@ -18,6 +18,9 @@ back:
 3. **Replays a committed history from `histories/`.** The cross-version guard. This is
    the one that fails when today's code would issue commands a run recorded on older code
    never made.
+4. **Runs the graph again with the workflow cache off** (`maxCachedWorkflows: 0`), so
+   every workflow task replays from the first event instead of resuming, and re-checks
+   the counts from (1). No sticky queue is used at all in this mode.
 
 Only (3) survives a change to the runner, which is why (3) is the one that matters at
 review time. (2) passes even on a broken change, because the history it checks was
@@ -26,9 +29,9 @@ recorded by the same broken code.
 ## The graph the harness runs
 
 `start → (left, right) → join`, defined in `../fixtures/graph.ts`. The fan-out is the
-point: it is the only shape that puts two commands in a single workflow task, which is
-where the runner's `Promise.all` becomes visible to Temporal. A straight line replays
-green while leaving that path untested.
+point: it is the only shape this runner can be given that puts more than one command in a
+single workflow task, which is where the runner's `Promise.all` becomes visible to
+Temporal. A straight line replays green while leaving that path untested.
 
 ## Recording a history