A compiler that writes a pipeline as plain TypeScript

.github/workflows/ci.yml

@@ -93,6 +93,35 @@ jobs:
         working-directory: packages/codeupipe-lint-ts
         run: npm run lint
 
+  typescript-compile:
+    name: TypeScript compiler
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.12"
+      - uses: actions/setup-node@v5
+        with:
+          node-version: "24"
+      - name: Install the core package
+        working-directory: packages/codeupipe-ts
+        run: npm ci
+      - name: Install
+        working-directory: packages/codeupipe-compile-ts
+        run: npm ci
+      - name: Type check
+        working-directory: packages/codeupipe-compile-ts
+        run: npm run typecheck
+      - name: Tests
+        working-directory: packages/codeupipe-compile-ts
+        run: npm test
+      - name: Build
+        working-directory: packages/codeupipe-compile-ts
+        run: npm run build
+      - name: Conformance
+        run: python conformance/run.py typescript:static
+
   export:
     name: Export against dynamic run
     runs-on: ubuntu-latest

README.md

@@ -29,14 +29,15 @@ If you know Unix, these are close to things you know.
 | Hook | A git hook: code that runs before and after a step |
 | State | `$?` and `PIPESTATUS` |
 
-This repository has four packages.
+This repository has five packages.
 
 | Package | Contents | Reference |
 |---|---|---|
 | `codeupipe` | The eight primitives. An export command that writes a pipeline as plain Python with no import of codeupipe. A config file reader. No dependencies | [conformance/API.md](conformance/API.md), [docs/static-export.md](docs/static-export.md), [docs/config.md](docs/config.md) |
 | `codeupipe-lint` | Two linters for pipeline code. Depends on `codeupipe` only | [packages/codeupipe-lint/README.md](packages/codeupipe-lint/README.md) |
 | `codeupipe` for TypeScript | The eight primitives. No dependencies | [packages/codeupipe-ts/README.md](packages/codeupipe-ts/README.md) |
 | `codeupipe-lint` for TypeScript | A linter for pipeline code in TypeScript | [packages/codeupipe-lint-ts/README.md](packages/codeupipe-lint-ts/README.md) |
+| `codeupipe-compile` for TypeScript | Writes a pipeline as plain TypeScript with no import of codeupipe | [packages/codeupipe-compile-ts/README.md](packages/codeupipe-compile-ts/README.md) |
 
 ## Install
 
@@ -153,7 +154,7 @@ An editable install needs pip 21.3 or later.
 | Command | What it does |
 |---|---|
 | `python -m pytest tests -q` | Runs the tests |
-| `python conformance/run.py` | Runs the 27 conformance cases against the Python dynamic run and the three exports, and against the TypeScript run when the machine has Node 22.18 or later |
+| `python conformance/run.py` | Runs the 27 conformance cases against the Python dynamic run and the three exports, and against the TypeScript targets that the machine can run |
 | `python -m codeupipe_lint.anonymous_steps` | Runs the anonymous-step linter. Paths come from `cup.config.json` |
 | `python -m codeupipe_lint.invariants` | Runs the invariant linter. Paths come from `cup.config.json` |
 | `python -m codeupipe.lower examples.pricing.pricing_pipeline:PricingPipeline --out build/pricing` | Exports the example to `build/pricing/src/steps/` |
@@ -168,6 +169,7 @@ An editable install needs pip 21.3 or later.
 | `packages/codeupipe-lint/` | The `codeupipe-lint` package: `anonymous_steps/`, `invariants/` |
 | `packages/codeupipe-ts/` | The TypeScript runtime, with its tests and its example |
 | `packages/codeupipe-lint-ts/` | The TypeScript linter, with its tests |
+| `packages/codeupipe-compile-ts/` | The TypeScript compiler, with its tests |
 | `conformance/` | The contract ([API.md](conformance/API.md)), the cases, the runners, the benchmark |
 | `tests/` | The tests for the two Python packages |
 | `examples/pricing/` | The example, with its test in `tests/examples/` |
@@ -180,8 +182,8 @@ The root `pyproject.toml` is not a package. It holds the test settings.
 
 The contract and the conformance cases do not depend on a language. Python
 is the first runtime and TypeScript is the second
-([packages/codeupipe-ts/](packages/codeupipe-ts/)), with its own linter. The
-export is for Python only. A runtime in another language must implement
+([packages/codeupipe-ts/](packages/codeupipe-ts/)), with its own linter and
+its own compiler. A runtime in another language must implement
 the contract and add a conformance runner.
 
 ## License

conformance/API.md

@@ -213,7 +213,7 @@ map and fails the comparison.
 | Run | `run` is sync. `run_async` is opt-in | One `run`. It returns a Promise. A step can be sync or async |
 | Absent | `None` | `null` |
 | Label of a step | `type(step).__name__` | `step.constructor.name` |
-| Export | `codeupipe.lower` | none |
+| Export | `codeupipe.lower` | `packages/codeupipe-compile-ts` |
 | Linter | `packages/codeupipe-lint` | `packages/codeupipe-lint-ts` |
 
 Notes on TypeScript:

conformance/MATRIX.md

@@ -6,48 +6,50 @@ if any target has a ❌.
 
 **Every target passes every case, and nothing is unsupported.** The targets
 are the Python dynamic run, async run and three static exports, and the
-TypeScript run.
+TypeScript run and its compiled form.
 
-The two fast exports (`static-steps`, `static-inline`) expose no State or
-Hooks, so they pass those cases as ✅\*. `static-observe` (export with
+The fast exports (`python:static-steps`, `python:static-inline`,
+`typescript:static`) expose no State or Hooks, so they pass those cases as
+✅\*. `static-observe` (export with
 `observe=True`) records both and passes every case outright. See
 [docs/static-export.md](../docs/static-export.md).
 
-| case | python:run | python:run_async | python:static-steps | python:static-inline | python:static-observe | typescript:run |
-|---|---|---|---|---|---|---|
-| `hook/lifecycle-on-success` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `hook/on-error-when-a-filter-fails` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `hook/not-notified-of-a-tap-failure` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `hook/on-error-names-a-failing-valve` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `payload/insert-adds-a-key` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `payload/insert-overwrites` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `payload/missing-key-reads-as-null` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `payload/nested-input-is-never-mutated` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `payload/json-value-types-round-trip` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `payload/remove-drops-a-key` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `payload/remove-of-a-missing-key-is-a-no-op` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `payload/merge-right-side-wins` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `payload/has-reports-presence-even-for-null` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `payload/mutable-escape-hatch-sets-in-place` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `pipeline/empty-returns-input` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `pipeline/steps-run-in-order` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `pipeline/filter-error-halts-the-run` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `pipeline/nested-pipeline-is-a-step` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `tap/observes-without-changing-output` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `tap/failure-never-halts-the-pipeline` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `tap/failure-does-not-stop-later-taps` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `tap/sees-payload-after-a-skipped-valve` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| `valve/runs-when-predicate-true` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `valve/skips-when-predicate-false` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `valve/missing-key-is-falsy` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `valve/sees-upstream-changes` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
-| `valve/independent-gates` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ |
+| case | python:run | python:run_async | python:static-steps | python:static-inline | python:static-observe | typescript:run | typescript:static |
+|---|---|---|---|---|---|---|---|
+| `hook/lifecycle-on-success` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `hook/on-error-when-a-filter-fails` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `hook/not-notified-of-a-tap-failure` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `hook/on-error-names-a-failing-valve` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `payload/insert-adds-a-key` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `payload/insert-overwrites` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `payload/missing-key-reads-as-null` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `payload/nested-input-is-never-mutated` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `payload/json-value-types-round-trip` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `payload/remove-drops-a-key` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `payload/remove-of-a-missing-key-is-a-no-op` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `payload/merge-right-side-wins` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `payload/has-reports-presence-even-for-null` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `payload/mutable-escape-hatch-sets-in-place` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `pipeline/empty-returns-input` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `pipeline/steps-run-in-order` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `pipeline/filter-error-halts-the-run` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `pipeline/nested-pipeline-is-a-step` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `tap/observes-without-changing-output` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `tap/failure-never-halts-the-pipeline` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `tap/failure-does-not-stop-later-taps` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `tap/sees-payload-after-a-skipped-valve` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| `valve/runs-when-predicate-true` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `valve/skips-when-predicate-false` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `valve/missing-key-is-falsy` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `valve/sees-upstream-changes` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
+| `valve/independent-gates` | ✅ | ✅ | ✅* | ✅* | ✅ | ✅ | ✅* |
 
 - **python:run**: 27 pass, 0 pass*, 0 fail, 0 unsupported
 - **python:run_async**: 27 pass, 0 pass*, 0 fail, 0 unsupported
 - **python:static-steps**: 11 pass, 16 pass*, 0 fail, 0 unsupported
 - **python:static-inline**: 11 pass, 16 pass*, 0 fail, 0 unsupported
 - **python:static-observe**: 27 pass, 0 pass*, 0 fail, 0 unsupported
 - **typescript:run**: 27 pass, 0 pass*, 0 fail, 0 unsupported
+- **typescript:static**: 11 pass, 16 pass*, 0 fail, 0 unsupported
 
 ✅* passed everything the target can observe; it does not expose State or Hooks (the fast static export). ➖ the target cannot express the case.

conformance/README.md

@@ -43,6 +43,7 @@ load a number can leave the band.
 | `python:static-steps`, `python:static-inline` | `codeupipe.lower` fast export, loaded from disk (no State or Hooks: ✅\*) |
 | `python:static-observe` | `codeupipe.lower` export with `observe=True`: State and Hooks judged too |
 | `typescript:run` | `packages/codeupipe-ts` Pipeline. Node runs the `.ts` runner as it is (Node 22.18 or later). With no arguments, `run.py` leaves this target out when the machine has no such Node |
+| `typescript:static` | `packages/codeupipe-compile-ts`: the pipeline as plain code, written to a file and imported (no State or Hooks: ✅\*). Needs `npm ci` in that package and a built core package; `run.py` leaves it out when they are not there |
 
 ## Statuses
 

conformance/run.py

@@ -71,12 +71,25 @@ def _node_runs_typescript():
     "python:static-inline": _python("static-inline"),
     "python:static-observe": _python("static-observe"),
     "typescript:run": _typescript("run"),
+    "typescript:static": _typescript("static"),
+}
+
+# The static target runs the compile package, which must be installed, and
+# that package uses the built core package.
+_TYPESCRIPT_NEEDS = {
+    "typescript:static": [REPO / "packages/codeupipe-compile-ts/node_modules",
+                          REPO / "packages/codeupipe-ts/dist/index.js"],
 }
 
 
 def default_targets():
-    """Every target that this machine can run. TypeScript needs Node."""
-    return [name for name in TARGETS if not name.startswith("typescript:") or _node_runs_typescript()]
+    """Every target that this machine can run. TypeScript needs Node, and the
+    static TypeScript target needs its package to be installed."""
+    def can_run(name):
+        if not name.startswith("typescript:"):
+            return True
+        return _node_runs_typescript() and all(path.exists() for path in _TYPESCRIPT_NEEDS.get(name, []))
+    return [name for name in TARGETS if can_run(name)]
 
 
 def load_cases():

conformance/runners/typescript/cup-ops/record-tap.ts

@@ -1,16 +1,15 @@
 import type { Payload, Tap } from "../../../../packages/codeupipe-ts/src/index.ts";
+import { TAPS } from "./sink.ts";
 
 /** append get(key) to the runner's tap log */
 export class RecordTap implements Tap {
   readonly key: string;
-  readonly log: unknown[];
 
-  constructor(key: string, log: unknown[]) {
+  constructor(key: string) {
     this.key = key;
-    this.log = log;
   }
 
   observe(payload: Payload): void {
-    this.log.push(payload.get(this.key));
+    TAPS.push(payload.get(this.key));
   }
 }

conformance/runners/typescript/cup-ops/sink.ts

@@ -0,0 +1,2 @@
+/** Where the Taps of the runner record what they see. The runner empties it before each case. */
+export const TAPS: unknown[] = [];