feat(tunnel): embedded relay-tunnel receiver via PyO3 + Hypercorn loopback (WIP)

[slinkydeveloper]

WIP / draft. Python sibling of the Java (FFM) and TypeScript (napi) embedded relay-tunnel receivers. Reuses the same Rust engine as restatedev/sdk-shared-core#94 (branch relay, tunnel feature), consumed as a git dependency; swap to a crates.io release once that ships.

What

Serve a Restate Python deployment as a relay receiver using the embedded native engine (restate-sdk-shared-core, tunnel feature). The engine dials the relay on its own tokio runtime and bridges each forwarded request over a loopback socket into a local HTTP/2 (h2c) server hosting this SDK's ASGI app. The PyO3 boundary is control-plane only (start / status / stop); no per-request data crosses it, so dispatch — and therefore both the promise API and the codegen SDK — is untouched.

The SDK is already PyO3/maturin (restate._internal) and already ships Hypercorn (test/harness extras), so this is a small addition.

Rust (src/lib.rs, the existing cdylib)

• #[pyclass] RelayTunnel — start(config_json) / status() / stop() — over the engine's relay::{Config, Engine, Handle}, registered in the _internal module. stop() releases the GIL while the runtime drains (the engine never calls back into Python).
• Cargo.toml repoints restate-sdk-shared-core to the relay branch with the tunnel feature; adds serde_json. Note: this also moves the VM core 7.0.1 → 7.0.2 (same as the Java branch, where the full suite passed in CI).

Python (python/restate/tunnel.py, new tunnel = [hypercorn] extra)

• serve_tunnel(services, *, relay, identity_keys=None, protocol=None) builds the ASGI app via restate.app(...), boots Hypercorn (h2c) on 127.0.0.1:0 in a background thread (mirrors harness.py), waits until it is listening, then starts RelayTunnel at that port. Returns a TunnelHandle (status() / stop(), context manager). Accepts services from either programming model.

Prior-knowledge h2c into Hypercorn is exactly how Restate talks to this SDK in production, so the engine's loopback dial is low-risk here.

Verification

cargo build + fmt + clippy clean; ruff clean; maturin develop --extras tunnel builds. A lifecycle smoke test boots the tunnel and confirms the loopback Hypercorn serves the SDK ASGI app (GET /health → 200 {"status":"ok"}) with status running, then stops. The forwarded round-trip is the same engine proven in shared-core's tests/relay_loopback.rs. (Not yet run: the full pytest/testcontainers suite on the 7.0.2 bump — worth a CI run.)

Follow-up (captured in development/relay-receiver-plan.md)

The native next step: make the tunnel be the ASGI server — drive app(scope, receive, send) directly, no Hypercorn/socket — via pyo3-async-runtimes, honouring the SDK's full-duplex teardown (server_context.py:616). Higher-risk cross-runtime build, which is why the proven loopback ships first.

Deferred

• The ASGI-direct driver above.
• Restate Cloud /_/start-tunnel mode (engine is /whoami-only, same as Java/TS).
• Swap the shared-core git dep for a crates.io release once tunnel ships.
• Validate the tunnel feature's tokio/h2/rustls(ring) on the maturin manylinux/musllinux/macos release matrix before publishing.

🤖 Generated with Claude Code

[github-actions]

Test Results

  8 files  ±0    8 suites  ±0   3m 23s ⏱️ +5s
 60 tests ±0   60 ✅ ±0  0 💤 ±0  0 ❌ ±0 
267 runs  ±0  267 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit f45cbdf. ± Comparison against base commit 5dfca1a.