feat(runtime): add Generic WASM and Python Reactor execution

[bkmashiro]

Summary

Add two opt-in wazero execution paths:

• Generic WASM for modules that implement Shimmy's memory / alloc / dispatch ABI.
• Python Reactor for prebuilt CPython/WASI runtimes with a trusted evaluator script.

The PR also includes a safe Python evaluator example, Linux HTTP end-to-end coverage, and a guided quick start.

What changed

• Added the generic profile, including request/response transfer through linear memory, prepared-memory reset, and explicit host path and environment allowlists.
• Added the python-reactor profile and its prepared instance pool.
• Added startup checks for the runtime artifact digest, ABI, imports, exports, and declared Python module capabilities.
• Python Reactor instances initialize once, then restore their full linear memory after every request.
• Discarded instances that grow memory during a request, since WebAssembly memory cannot shrink back to the prepared size.
• Returned request failures and timeouts before replacing discarded slots. Refill runs asynchronously and a failed refill leaves the pool closed rather than reusing a bad instance.
• Added examples/safe-eval-python with demo, stdin/stdout, unit test, and preview modes.
• Bounded stdout, stderr, traceback, and test details as they are written, including multibyte UTF-8 output.
• Added serve.sh and try.sh for starting a profile and exercising it over Shimmy's HTTP interface.

Runtime boundaries

The runtime manifest controls which Python modules an artifact may provide. The evaluator's AST import checks are useful diagnostics, but they are not the security boundary.

Python Reactor does not expose host filesystem access, networking, or process creation by default. Artifact hashes confirm that the deployed files match the manifest; provenance still depends on how the runtime artifact is built and distributed.

Testing

The test suite covers Generic WASM dispatch, capability allowlists, prepared-memory reset, and instance discard after memory growth.

The Linux E2E used the exact Producer base artifact and exercised artifact and ABI validation, Python Reactor over HTTP, all safe evaluator modes, timeout recovery, and the guided base-profile quick start.

A timed-out request returned HTTP 500 in 5.74 seconds. The discarded slot was replaced in the background and a later request completed successfully. Runtime artifacts and receipts were checked against their SHA-256 hashes before the run.

[Copilot]

Pull request overview

This PR adds two opt-in in-process WebAssembly execution paths to Shimmy (Generic WASM and Python Reactor) backed by wazero, including artifact/ABI validation, snapshot-based state reset, and accompanying end-to-end fixtures and onboarding docs/scripts.

Changes:

• Introduces a Generic WASM dispatcher that uses a memory/alloc/dispatch ABI and restores a prepared linear-memory snapshot between requests.
• Adds a Python Reactor dispatcher that validates a Producer artifact + manifest, supports multiple lifecycle modes (snapshot/single-use/fresh), and discards/refills instances after failures or memory drift.
• Adds an artifact checker CLI, Linux HTTP E2E scripts, and a safe-eval-python example (with tests + guided quick start) to demonstrate the Python Reactor profile.

Reviewed changes

Copilot reviewed 44 out of 46 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/e2e/python-reactor/evaluator.py	Linux E2E evaluator fixture for Python Reactor HTTP verification.
scripts/e2e-safe-eval-python.sh	End-to-end script exercising safe-eval-python over HTTP, including timeout recovery.
scripts/e2e-python-reactor.sh	End-to-end script exercising Python Reactor HTTP path and snapshot reset.
README.md	Documents opt-in WASM/Python Reactor paths and clarifies sandboxing constraints (incl. Lambda).
internal/execution/wasm/testhelpers_test.go	Adds shared wazero/fixture helpers for WASM tests.
internal/execution/wasm/testdata/echo.wat	Adds WAT source for the echo ABI fixture used in WASM tests.
internal/execution/wasm/supervisor.go	Implements per-instance snapshot/restore with unhealthy marking on restore failure or memory growth.
internal/execution/wasm/snapshot.go	Adds SnapshotStrategy + FullMemcpyStrategy for linear-memory snapshotting.
internal/execution/wasm/snapshot_test.go	Unit tests for snapshot strategy behavior (roundtrip, drift, ownership, idempotence).
internal/execution/wasm/robustness_test.go	Robustness tests for malformed ABI modules (alloc/dispatch signature issues).
internal/execution/wasm/python_reactor_artifact.go	Inspects and verifies Python Reactor module shape against manifest contract.
internal/execution/wasm/python_preload_config_test.go	Tests default and validation for Python preload mode config.
internal/execution/wasm/pool.go	Adds generic pool drain helper with context cancellation to avoid deadlocks.
internal/execution/wasm/json_util.go	Adds JSON response parsing helper with bounded raw context in errors.
internal/execution/wasm/dispatcher.go	Adds Generic WASM dispatcher with pool management, shutdown coordination, and replacement logic.
internal/execution/wasm/dispatcher_test.go	Comprehensive tests for generic dispatcher behavior, concurrency, shutdown races, and memory-grow handling.
internal/execution/wasm/config.go	Extends WASM config for sandbox allowlists and Python Reactor options.
internal/execution/wasm/artifact_check.go	Adds artifact compilation + ABI validation entrypoint (generic + python-reactor).
internal/execution/wasm/artifact_check_test.go	Tests for artifact checker generic ABI validation behavior.
internal/execution/wasm/agent_python.go	Implements Python Reactor dispatcher with lifecycle modes, snapshot reset, discard/refill, and bounded diagnostics.
internal/execution/wasm/agent_python_test.go	Tests for manifest validation, request/response contracts, lifecycle behavior, and real-artifact compatibility hooks.
internal/execution/wasm/agent_python_protocol.go	Adds pinned consumer-side protocol/manifest parsing + request/response encoding/decoding.
internal/execution/wasm/agent_python_observer.go	Adds phase observer types and emission for lifecycle instrumentation.
internal/execution/wasm/agent_python_lifecycle_config_test.go	Tests lifecycle defaults and invalid lifecycle/capacity bounds.
internal/execution/wasm/adapter.go	Implements the generic alloc/dispatch adapter (host ↔ linear memory transfer).
internal/execution/supervisor/models.go	Adds wasm IO interface constant.
internal/execution/supervisor/config.go	Documents the new wasm interface behavior and selection contract.
internal/execution/dispatcher.go	Routes FUNCTION_INTERFACE=wasm to generic or python-reactor dispatchers based on FUNCTION_WASM_PROFILE.
go.sum	Adds wazero module checksums.
go.mod	Adds wazero dependency.
examples/safe-eval-python/try.sh	Guided HTTP driver for the safe-eval-python example requests.
examples/safe-eval-python/serve.sh	Launcher to validate artifact/manifest and start Shimmy configured for Python Reactor.
examples/safe-eval-python/safe_eval.py	Trusted evaluator implementing demo/io_test/unit_test/preview with bounded output handling.
examples/safe-eval-python/safe_eval_test.py	Unit tests for safe_eval behavior and output bounding.
examples/safe-eval-python/requests/unit-tests.json	Example request payload for unit-test mode.
examples/safe-eval-python/requests/sympy.json	Example request payload for SymPy profile behavior.
examples/safe-eval-python/requests/preview-blocked.json	Example request payload demonstrating preview rejection of blocked imports.
examples/safe-eval-python/requests/numpy-core.json	Example request payload for NumPy profile behavior.
examples/safe-eval-python/requests/io-tests-pass.json	Example request payload for passing IO tests (incl. hidden).
examples/safe-eval-python/requests/io-tests-fail.json	Example request payload for failing IO tests.
examples/safe-eval-python/requests/demo.json	Example request payload for demo mode.
examples/safe-eval-python/README.md	Full quick start + role separation + security boundary explanation for safe-eval-python.
docs/execution-paths.md	Adds end-user docs for Generic WASM and Python Reactor execution paths and verification steps.
cmd/shimmy-artifact-check/main.go	New CLI tool to validate WASM artifacts/manifests without starting Shimmy.
cmd/root.go	Updates CLI help for --interface / --command to include WASM usage.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[m-messer]

Please allocate to me when you want a review.

Thanks,
Marcus

[bkmashiro]

Hi @m-messer ,This is ready for review.

This PR is an opt-in technical experiment and does not change the existing execution paths. Its scope is to establish the Generic WASM and initial Python Reactor path against the current Shimmy interfaces. Broader Python compatibility and fallback paths will be handled in separate PRs later.

Cheers,
Dylan

[m-messer]

Thanks for the work on the WASM implementation. A few questions and suggestions in the comments.

Has this been tested end-to-end on a containerised environment? And has it been tested with a real evaluation function like evaluatePython?

I am happy to merge this into a separate LF branch, so we can deploy and test in our staging environment if you think that will be useful?

Thanks,
Marcus

[bkmashiro]

Thanks for the review. The Python Reactor path has now been tested end to end in a containerised environment. The run built the base runtime artifact from locked sources, built the production Shimmy image, and exercised evaluation and preview requests through Shimmy's HTTP interface.:

https://github.com/bkmashiro/shimmy-wasm-go/actions/runs/32724088562

The container workflow is kept in my own repository for verification and is not included in this PR.

I have not run the existing evaluatePython implementation unchanged because it is incompatible with the current Reactor capability model. Although it exposes a similar Python callable interface, its evaluation modes execute student code through temporary files and child Python processes. Plot handling also relies on filesystem access and optionally S3. The Reactor deliberately provides no process creation, Host filesystem access or networking, so adapting the callable interface alone would not make the existing implementation work.

Existing evaluators will need different levels of migration depending on the capabilities they use. Pure evaluator functions should be relatively straightforward to adapt when their dependencies are packaged in the artifact. Evaluators that rely on subprocesses, Host files, networking or external services will need a separate compatibility design. I suggest postponing that work until the restricted Python path is stable and we have staging evidence from representative evaluators.

The Reactor in this PR is therefore an explicit, opt-in restricted execution lane. It is not an automatic fallback, and the existing Shimmy execution paths remain unchanged. Since the initial review, I have also included the Producer source in the PR, moved the physical artifact ABI to shimmy-python-runtime/v1, and removed the previous external Agent Python host-call dependency.

A separate LF branch and staging deployment would be very useful. I suggest starting with a small experimental evaluator, perhaps wasiEvalPython, implemented as a self-contained Python package without external resource dependencies. This would give us a controlled way to test representative LF workloads, measure startup and payload behaviour, and determine which parts of evaluatePython should later be migrated or handled through a separate execution path.

I have also included an experimental wasiEvalPython evaluator as a concrete staging target. It intentionally implements a small, verifiable subset of the existing evaluator: demo execution, input/output tests, hidden tests, simple unit-test functions and preview validation.

The current Reactor provides limited libraries like NumPy and SymPy. It does not currently include Pandas, SciPy, plotting, etc.

We may test the Reactor's behavior on wasiEvalPython in staging environment perhaps?

[bkmashiro]

I agree that this should not merge directly into main yet. I checked the repository and could not find an existing staging branch. Could you create a separate LF branch from the current main, for example feature/python-reactor-staging, and let me know the branch name? I can then retarget this PR to it.

[m-messer]

I've created the branch: feature/python-reactor-staging

Once you have merged into that branch, let me know, and I'll send you the staging endpoint. I will also add you as a collaborator so you can run the GitHub actions to deploy.

For the test evaluation function, is the existing evaluatePython function too complex, as it supports file uploading and libraries such as pandas and matplotlib? I would rather not duplicate functionality if we can help it, but if the features supported by the current version of evaluatePython are not currently implemented, I am happy to set up a simpler version for e2e testing all the way to the platform.

[bkmashiro]

Thanks, I have merged the PR into feature/python-reactor-staging.

Seems E2E consists of 2 parts

• test cases for evaluators
• e2e input and output from historical submissions DB

For the initial E2E, we can reuse the subset of evaluatePython test cases currently supported by wasiEvalPython: demo, I/O tests, hidden tests and simple unit tests.

Since this is a new evaluation-function name, it has no historical submissions yet, so historical replay will not block e2e.