feat(runtime): add Generic WASM and Python Reactor execution

README.md

@@ -202,6 +202,15 @@ The shim keeps the evaluation function running as a persistent process and commu
 | `tcp` | Raw TCP connection. |
 | `ws` | WebSocket connection. Experimental — custom dialer configuration is not yet supported. |
 
+Generic WASM and Python Reactor are explicit opt-in execution paths. See
+[Execution paths](docs/execution-paths.md) for their environment contracts,
+lifecycle behavior, and compatibility boundaries.
+
+To try Python Reactor without assembling requests by hand, follow the
+[`wasi-eval-python` three-command quick start](examples/wasi-eval-python/README.md#start-here-first-successful-evaluation).
+It includes runnable base, NumPy, and SymPy fixtures plus both passing and
+failing student-code examples.
+
 The shim injects the following environment variables into the evaluation function process so it can identify the transport it should listen on:
 
 | Variable | Value |
@@ -246,16 +255,24 @@ For example, a Wolfram Language evaluation function in `evaluation.wl` would be
 wolframscript -file evaluation.wl /tmp/shimmy/abc/request-data-123 /tmp/shimmy/abc/response-data-456
 ```
 
-### Sandboxed Execution (Linux only, experimental)
+### Sandboxed Execution (Linux host/container only, experimental)
 
-Shimmy can wrap each worker process in an [nsjail](https://github.com/google/nsjail) sandbox to safely execute arbitrary, untrusted code. The sandbox provides:
+On supported Linux hosts, Shimmy can wrap each worker process in an [nsjail](https://github.com/google/nsjail) sandbox to execute untrusted code with an additional OS boundary. The sandbox provides:
 
 - **Filesystem confinement** — the worker can only access explicitly bind-mounted paths
 - **Resource limits** — CPU time, memory, and file descriptor caps
 - **Network isolation** — optional; disables all outbound connections
 - **Unprivileged UID** — worker runs as `nobody` (uid 65534) inside the jail
 
-Sandboxing requires Linux and the `nsjail` binary. The Docker image built from the project's `Dockerfile` includes nsjail at `/usr/sbin/nsjail`. On the host, install it with `sudo apt install nsjail` (Ubuntu 22.04+) or build from source.
+Sandboxing requires Linux, the `nsjail` binary, and permission to create the
+required namespaces/capabilities. The Docker image built from the project's
+`Dockerfile` includes nsjail at `/usr/sbin/nsjail`. On the host, install it with
+`sudo apt install nsjail` (Ubuntu 22.04+) or build from source.
+
+> **AWS Lambda:** Lambda does not grant the namespace/capability controls needed
+> to enable this nsjail path. Shipping the binary in a Lambda container image
+> does not make it an available security boundary. Use the in-process WASM
+> execution profiles for Lambda-compatible isolation.
 
 Enable sandboxing with `--sandbox` and configure it with the flags below:
 

build/python-reactor/producer/README.md

@@ -0,0 +1,54 @@
+# Shimmy Python Runtime Producer
+
+This directory is the source of truth for Shimmy's CPython/WASI guest artifacts.
+The producer is intentionally self-contained: Shimmy-authored guest and build code
+is combined only with digest-locked official upstream sources and tools.
+
+## Contract
+
+`contract/shimmy-python-runtime-v1.json` defines the complete Host/Guest seam.
+The module is a `wasm32-wasip1` reactor, imports only WASI Preview 1, and exposes
+an explicit Shimmy identity plus bounded `init`, `prepare`, allocation, and
+request evaluation functions.
+
+The Guest receives a prepared evaluator before the snapshot boundary. Requests
+contain only a method and params object. Responses are bounded, length-prefixed
+JSON objects copied into Host-owned memory before the Host restores or discards
+the instance.
+
+## Profiles
+
+- `base`: CPython and the selected standard library only.
+- `numpy-core`: `base` plus a source-built, statically registered NumPy subset.
+- `sympy`: `base` plus pinned pure-Python SymPy and mpmath packages in the
+  read-only artifact VFS. No native port or runtime package installation is
+  involved.
+
+Each artifact manifest declares its importable top-level Python modules.
+SciPy and Pandas remain outside the Reactor profiles and use the Pyodide
+compatibility path.
+
+No profile grants environment variables, filesystem preopens, networking,
+or custom Host calls.
+
+## Provenance rules
+
+- `sources.lock.json` is strict and offline-verifiable.
+- Mutable URLs and unpinned package resolution are rejected.
+- Prebuilt Python runtimes from other product repositories are not accepted.
+- Generated artifacts are CI outputs, not tracked Git blobs or releases.
+- Artifact, manifest, benchmark binary, schemas, and source receipt must bind one
+  clean Shimmy commit before benchmark promotion.
+
+## Cheap gate
+
+```bash
+python3 -m unittest discover \
+  -s build/python-reactor/producer/tests -p 'test_*.py' -v
+python3 build/python-reactor/producer/tools/verify_sources_lock.py \
+  build/python-reactor/producer/sources.lock.json
+```
+
+Real Linux Guest builds and Host execution are manual-only gates. The
+`shimmy-python` workflow lane builds all profiles and runs the base profile
+through Shimmy's production container before uploading the exact bundle.

build/python-reactor/producer/build/build-base.sh

@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)
+exec python3 "$ROOT/build/python-reactor/producer/tools/build_runtime.py" \
+  --profile base "$@"

build/python-reactor/producer/build/build-numpy-core.sh

@@ -0,0 +1,5 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)
+exec python3 "$ROOT/build/python-reactor/producer/tools/build_numpy_profile.py" "$@"

build/python-reactor/producer/build/build-sympy.sh

@@ -0,0 +1,5 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)
+exec python3 "$ROOT/build/python-reactor/producer/tools/build_sympy_profile.py" "$@"

build/python-reactor/producer/build/link-reactor.mk

@@ -0,0 +1,49 @@
+# Included after CPython's generated WASI Makefile.
+
+.PHONY: shimmy-python-runtime
+
+SHIMMY_RUNTIME_CFLAGS :=
+SHIMMY_LINKER := $(LINKCC)
+SHIMMY_NUMPY_LINK :=
+SHIMMY_NUMPY_LIBC_LINK :=
+SHIMMY_CXX_LIBS :=
+ifneq ($(strip $(SHIMMY_NUMPY_ARCHIVES)),)
+SHIMMY_RUNTIME_CFLAGS += -DSHIMMY_NUMPY_CORE=1
+SHIMMY_LINKER := $(CXX)
+SHIMMY_NUMPY_LINK := -Wl,--whole-archive $(SHIMMY_NUMPY_ARCHIVES) -Wl,--no-whole-archive
+SHIMMY_NUMPY_LIBC_LINK := -lc-printscan-long-double
+SHIMMY_CXX_LIBS := -lc++ -lc++abi
+endif
+
+shimmy-python-runtime:
+	@test -n "$(SHIMMY_RUNTIME_SOURCE)"
+	@test -n "$(SHIMMY_RUNTIME_INCLUDE)"
+	@test -n "$(SHIMMY_GENERATED_INCLUDE)"
+	@test -n "$(SHIMMY_WASI_VFS_LIBRARY)"
+	@test -n "$(SHIMMY_OUTPUT)"
+	$(CC) $(PY_CORE_CFLAGS) \
+		$(SHIMMY_RUNTIME_CFLAGS) \
+		-I$(SHIMMY_RUNTIME_INCLUDE) \
+		-I$(SHIMMY_GENERATED_INCLUDE) \
+		-c $(SHIMMY_RUNTIME_SOURCE) \
+		-o shimmy_python_runtime.o
+	$(SHIMMY_LINKER) $(PY_CORE_LDFLAGS) $(LINKFORSHARED) \
+		-mexec-model=reactor \
+		-Wl,-z,stack-size=16777216 \
+		-Wl,--stack-first \
+		-Wl,--initial-memory=268435456 \
+		-Wl,--max-memory=2147483648 \
+		-Wl,--export-memory \
+		-Wl,--export=shimmy_python_runtime_identity \
+		-Wl,--export=shimmy_python_init \
+		-Wl,--export=shimmy_python_prepare \
+		-Wl,--export=alloc \
+		-Wl,--export=dealloc \
+		-Wl,--export=evaluate \
+		-o $(SHIMMY_OUTPUT) \
+		shimmy_python_runtime.o \
+		$(SHIMMY_NUMPY_LINK) \
+		$(SHIMMY_NUMPY_LIBC_LINK) \
+		$(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) \
+		$(SHIMMY_WASI_VFS_LIBRARY) \
+		$(SHIMMY_CXX_LIBS)

build/python-reactor/producer/contract/shimmy-python-runtime-v1.json

@@ -0,0 +1,58 @@
+{
+  "schema": "shimmy-python-runtime-contract/v1",
+  "artifact_contract": "shimmy-python-runtime/v1",
+  "target": "wasm32-wasip1",
+  "execution_model": "reactor",
+  "profiles": [
+    "base",
+    "numpy-core",
+    "sympy"
+  ],
+  "profile_python_modules": {
+    "base": [],
+    "numpy-core": [
+      "numpy"
+    ],
+    "sympy": [
+      "mpmath",
+      "sympy"
+    ]
+  },
+  "identity_u32": 1397772849,
+  "allowed_import_modules": [
+    "wasi_snapshot_preview1"
+  ],
+  "forbidden_import_modules": [
+    "agent_runtime_v1"
+  ],
+  "required_exports": [
+    "memory",
+    "_initialize",
+    "shimmy_python_runtime_identity",
+    "shimmy_python_init",
+    "shimmy_python_prepare",
+    "alloc",
+    "dealloc",
+    "evaluate"
+  ],
+  "request_envelope": {
+    "required": [
+      "method",
+      "params"
+    ],
+    "methods": [
+      "eval",
+      "preview"
+    ],
+    "unknown_fields": "reject"
+  },
+  "response_layout": "u32le-length-prefixed-json",
+  "request_max_bytes": 1048576,
+  "response_max_bytes": 1048576,
+  "capabilities": {
+    "environment": false,
+    "filesystem_preopens": false,
+    "network": false,
+    "host_calls": false
+  }
+}

build/python-reactor/producer/guest/bootstrap/runtime.py

@@ -0,0 +1,87 @@
+"""Trusted bootstrap for the Shimmy Python/WASI guest.
+
+This source is embedded at build time and executed once during Guest init.
+Evaluator source is supplied separately at the trusted preparation boundary.
+"""
+
+from __future__ import annotations
+
+import json as _json
+
+
+_prepared_eval = None
+_prepared_preview = None
+
+
+def _shimmy_prepare(source: str) -> None:
+    global _prepared_eval, _prepared_preview
+    if not isinstance(source, str):
+        raise TypeError("evaluator source must be text")
+    namespace = {"__builtins__": __builtins__, "__name__": "__shimmy_evaluator__"}
+    exec(compile(source, "<shimmy-evaluator>", "exec"), namespace, namespace)
+    evaluation = namespace.get("evaluation_function")
+    preview = namespace.get("preview_function")
+    if not callable(evaluation):
+        raise ValueError("evaluator must define evaluation_function")
+    if preview is not None and not callable(preview):
+        raise ValueError("preview_function must be callable when defined")
+    _prepared_eval = evaluation
+    _prepared_preview = preview
+
+
+def _json_default(value):
+    item = getattr(value, "item", None)
+    if callable(item):
+        return item()
+    tolist = getattr(value, "tolist", None)
+    if callable(tolist):
+        return tolist()
+    raise TypeError(f"value of type {type(value).__name__} is not JSON serializable")
+
+
+def _error(exc: BaseException) -> str:
+    payload = {
+        "status": "error",
+        "error": {
+            "type": type(exc).__name__,
+            "message": str(exc)[:4096],
+        },
+    }
+    return _json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
+
+
+def _shimmy_handle_request(request_json: str) -> str:
+    try:
+        if _prepared_eval is None:
+            raise RuntimeError("evaluator has not been prepared")
+        request = _json.loads(request_json)
+        if not isinstance(request, dict):
+            raise ValueError("request must be an object")
+        if set(request) != {"method", "params"}:
+            raise ValueError("request must contain exactly method and params")
+        method = request["method"]
+        params = request["params"]
+        if method not in {"eval", "preview"}:
+            raise ValueError("method must be eval or preview")
+        if not isinstance(params, dict):
+            raise ValueError("params must be an object")
+
+        if method == "preview" and _prepared_preview is not None:
+            result = _prepared_preview(
+                params.get("response"),
+                params.get("params", {}),
+            )
+        else:
+            result = _prepared_eval(
+                params.get("response"),
+                params.get("answer"),
+                params.get("params", {}),
+            )
+        return _json.dumps(
+            {"status": "ok", "result": result},
+            default=_json_default,
+            ensure_ascii=False,
+            separators=(",", ":"),
+        )
+    except Exception as exc:
+        return _error(exc)

build/python-reactor/producer/guest/include/shimmy_python_runtime_v1.h

@@ -0,0 +1,18 @@
+#ifndef SHIMMY_PYTHON_RUNTIME_V1_H
+#define SHIMMY_PYTHON_RUNTIME_V1_H
+
+#include <stdint.h>
+
+#define SHIMMY_PYTHON_RUNTIME_IDENTITY 0x53505231u
+#define SHIMMY_REQUEST_MAX_BYTES (1u << 20)
+#define SHIMMY_RESPONSE_MAX_BYTES (1u << 20)
+#define SHIMMY_RESPONSE_PREFIX_BYTES 4u
+
+uint32_t shimmy_python_runtime_identity(void);
+int32_t shimmy_python_init(void);
+int32_t shimmy_python_prepare(uint32_t source_ptr, uint32_t source_len);
+uint32_t alloc(uint32_t size);
+void dealloc(uint32_t ptr);
+uint32_t evaluate(uint32_t request_ptr, uint32_t request_len);
+
+#endif