example: Shen at the edge — Envoy fronting the two OpenResty apps

[pyrex41]

What

A new examples/envoy/ — the third piece of the web trilogy. Envoy (the standard edge/mesh proxy) goes in front of the guestbook (examples/openresty) and the authz app (examples/openresty-authz), using each of Envoy's two integration seams for what it's good at:

1. ext_authz → the authz app. Envoy forwards every API request (method + path + authorization header, no body) to the authz service before routing it. The Prolog proof chain decides; denials return the discharge report to the client verbatim; and every edge decision lands in the same durable audit log as a direct API call. No authorization code lives in the proxy.
2. An Envoy Lua filter → rules.shen inside the proxy. Envoy's Lua filter embeds LuaJIT — shen-lua's primary host — so filter.lua boots the kernel once per worker thread and runs the guestbook's typed field rules in the proxy: malformed POSTs get their 400 at the edge, with the origin's exact typed error strings.

That makes it one rules.shen enforced on four hosts from one typed source: browser (ShenScript, shaken), edge (shen-lua on Envoy's LuaJIT), origin (shen-lua on OpenResty), and plain luajit in the selftests.

Changes to the authz example

• app.shen: a (route "CHECK" Path ...) route speaking ext_authz's http_service protocol — same authorize-* gate, same host-log, so edge decisions run the same proof chain and are durably audited. Unmapped paths fail closed (resource "" → denied "unknown resource").
• authz.shen: a new typed projection check-response — total over decision, and no shape of decision places document content in a check response, so the gateway structurally cannot become a data leak (render-doc's dual).
• app.lua / nginx.conf: the glue maps /authz/<path> + Authorization: Bearer <tok> to the CHECK dispatch; a /authz/ location.

New files

• envoy.yaml — ext_authz (fail-closed) + Lua filter + router; per-route disable for static content
• filter.lua — the edge validator (only the pure typed core belongs in-proxy: per-thread Lua states, no shared dicts/cosockets, httpCall only)
• authz.conf — the authz app as ext_authz backend on :8081, seeding a guestbook resource
• selftest.lua — the whole three-layer pipeline under plain luajit, in Envoy's exact filter order, asserting which layer answers each request
• README.md — architecture, run instructions, and verified deployment mechanics

Testing

Selftests (plain luajit):

• luajit examples/envoy/selftest.lua — passes (ext_authz + Lua filter + upstream; edge 400s match the origin's typed strings; all 13 edge decisions in the audit log)
• luajit examples/openresty-authz/selftest.lua — passes with new CHECK cases, decision-for-decision identical across file/lmdb/cosocket substrates (16 decisions)
• luajit examples/openresty/selftest.lua — passes (unchanged)

Live, end to end (Envoy 1.39.0 via Homebrew + OpenResty, arm64 macOS — the full three-process setup from the README):

• proof-chain denials at the edge with discharge reports: no token / tok-carol (not a member) / tok-bob POST (viewer needs editor) → 403 from ext_authz
• typed rules inside the proxy: tok-alice with a bad body → 400 {"errors":["name: is required"]} with x-shen-edge: rejected + server: envoy (proving the origin was never touched); blank-message and broken-JSON variants likewise
• the valid entry survives both gates → 201, stored in the real guestbook, read back
• revocation via the admin API flips the edge's answer to access to this resource was revoked
• static route bypass (per-route filter disable) serves the front end
• the durable audit log on :8081 contains every edge decision
• steady-state latency through the full chain: 3–6 ms per request; ~3 s to serving with a warm cache

Two operational findings from the live run, now documented in the README and set in its run command:

• SHEN_KERNEL_CACHE=<dedicated path>: the bytecode cache is keyed to the exact LuaJIT build, so Envoy's LuaJIT and the local luajit otherwise rewrite each other's shared default cache.
• SHEN_JIT=off on macOS: the hardened runtime denies Envoy's binary executable trace memory, so LuaJIT thrashes on failed trace compiles (measured ~550× on a probe loop; 40–66 s kernel boots). Interpreted: ~3 s boots, same single-digit-ms requests. Linux builds generally don't need it.

🤖 Generated with Claude Code

[pyrex41]

The two operational findings are now fixed in boot.lua rather than documented as env-var workarounds (commit 2ef93b7):

1. Per-build bytecode caches. The default cache path is now .shen-kernel-cache.<build-hash>.bin, hashed from jit.version .. jit.arch — the same fingerprint the content key already used. Local luajit, OpenResty's LuaJIT, and Envoy's LuaJIT each keep their own warm cache instead of invalidating each other's; writing a new default-path cache removes the obsolete shared-path file. Verified with three different LuaJIT builds coexisting in one cwd.

2. Auto-fallback when the JIT can't compile. Boot now probes the JIT directly — compile one throwaway hot loop under a jit.attach trace watcher — and turns the JIT off (with a stderr note) if no trace materializes. This catches hosts that report jit.status() == true but are denied executable memory (macOS hardened-runtime binaries embedding LuaJIT, like Envoy). SHEN_JIT=on skips the probe; healthy hosts compile the one probe trace and are unchanged.

Re-verified live with no environment variables: Envoy cold-boots to serving in 2 s (was 40–66 s), warm-restarts in 1 s, requests at 1.5–3 ms; the probe fired inside Envoy (4 stderr notes, one per Lua state) and stayed silent under OpenResty and local luajit, whose JITs are healthy. Kernel test suite ok, all three example selftests pass.