From ec540a6f7f75fc828a816c4e5328613a04f30712 Mon Sep 17 00:00:00 2001 From: Reuben Brooks Date: Tue, 11 Aug 2026 07:57:08 -0500 Subject: [PATCH 1/4] =?UTF-8?q?example:=20Shen=20at=20the=20edge=20?= =?UTF-8?q?=E2=80=94=20Envoy=20fronting=20the=20two=20OpenResty=20apps?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit examples/envoy/ puts Envoy in front of the guestbook and authz examples, using both of its integration seams: - ext_authz -> the authz app: every edge request runs the Prolog proof chain before routing; denials return the discharge report and every edge decision lands in the same durable audit log. New in the authz app: a (route "CHECK" Path ...) route speaking Envoy's ext_authz http_service protocol, and a typed check-response projection in authz.shen (total over `decision`, structurally unable to include document content). - an Envoy Lua filter -> rules.shen inside the proxy: filter.lua boots shen-lua once per worker thread on Envoy's LuaJIT and rejects malformed guestbook POSTs at the edge with the origin's exact typed error strings. One rules.shen, four hosts (browser / edge / origin / selftests). selftest.lua drives the whole pipeline under plain luajit in Envoy's exact filter order (a faithful fake of Envoy's request_handle drives the real filter.lua; the real authz app decides; only the guestbook upstream is a stand-in). The authz selftest gains CHECK cases, still decision-for-decision identical across file/lmdb/cosocket substrates. Co-Authored-By: Claude Fable 5 --- README.md | 1 + examples/README.md | 1 + examples/envoy/README.md | 175 +++++++++++++++++++++++++ examples/envoy/authz.conf | 76 +++++++++++ examples/envoy/envoy.yaml | 103 +++++++++++++++ examples/envoy/filter.lua | 109 ++++++++++++++++ examples/envoy/selftest.lua | 178 ++++++++++++++++++++++++++ examples/openresty-authz/README.md | 12 ++ examples/openresty-authz/app.lua | 13 ++ examples/openresty-authz/app.shen | 43 ++++++- examples/openresty-authz/authz.shen | 14 ++ examples/openresty-authz/nginx.conf | 8 ++ examples/openresty-authz/selftest.lua | 10 ++ 13 files changed, 737 insertions(+), 6 deletions(-) create mode 100644 examples/envoy/README.md create mode 100644 examples/envoy/authz.conf create mode 100644 examples/envoy/envoy.yaml create mode 100644 examples/envoy/filter.lua create mode 100644 examples/envoy/selftest.lua diff --git a/README.md b/README.md index 05ac1d3..50b567b 100644 --- a/README.md +++ b/README.md @@ -309,6 +309,7 @@ browser. One `rules.shen`, two runtimes, no client/server drift. See its | [`examples/pcr/`](examples/pcr/) | **proof-carrying requests over live facts**: the client carries a proof term, the OpenResty gate *checks* it — never searches — against a versioned fact store consulted at proof time, so revoking one fact makes the same proof bytes fail on the next request while delegation chains stay composable and every allow logs its justification ([README](examples/pcr/README.md)) | | [`examples/openresty/`](examples/openresty/) | a **complete web app in Shen on OpenResty** (nginx + LuaJIT): typed request validators + a Shen router behind a JSON API, with a front end that runs the **same** typed rules in the browser — Ratatoskr-shaken and ShenScript-compiled to a ~140 KB module. One `rules.shen`, validated client- and server-side. Runs standalone (`luajit examples/openresty/selftest.lua`) or under `openresty` ([README](examples/openresty/README.md)) | | [`examples/openresty-authz/`](examples/openresty-authz/) | durable multi-tenant **authorization**: the policy as a Prolog proof chain (`token → user → tenant → resource`), a typed `decision` witness that gates every response, and an event-sourced store (file + `lua-resty-lmdb`) whose append-only log makes decisions durable and auditable ([README](examples/openresty-authz/README.md)) | +| [`examples/envoy/`](examples/envoy/) | **Shen at the edge**: Envoy fronting both apps above — `ext_authz` runs every request through the authz proof chain (edge decisions durably audited), and an Envoy **Lua filter** runs the same typed `rules.shen` inside the proxy, so malformed requests get their typed 400 before costing an upstream hop ([README](examples/envoy/README.md)) | ## Certification / Testing diff --git a/examples/README.md b/examples/README.md index e0cead2..0629d20 100644 --- a/examples/README.md +++ b/examples/README.md @@ -15,6 +15,7 @@ has a `selftest.lua` that runs off-nginx): | [`pcr/`](pcr/) | **proof-carrying requests over live facts**: the client attaches a proof term, the OpenResty gate *checks* it — never searches — against a versioned fact store consulted at proof time, so revoking one fact makes the same proof bytes fail on the next request while delegation chains stay composable and every allow logs its full justification. `luajit examples/pcr/selftest.lua` | | [`openresty/`](openresty/) | a complete web app — typed Shen validators + a Shen router on OpenResty (nginx + LuaJIT), with a front end that runs the **same** rules in the browser (Ratatoskr-shaken, ShenScript-compiled). Runs standalone via `luajit examples/openresty/selftest.lua`; see [its README](openresty/README.md) to serve it. | | [`openresty-authz/`](openresty-authz/) | durable multi-tenant **authorization**: the policy as a Prolog proof chain (`token → user → tenant → resource`), a typed `decision` witness that gates every response, and an event-sourced store (file + `lua-resty-lmdb`) whose append-only log makes decisions durable and auditable. Runs standalone via `luajit examples/openresty-authz/selftest.lua`; see [its README](openresty-authz/README.md). | +| [`envoy/`](envoy/) | **Shen at the edge**: Envoy in front of both apps above — its `ext_authz` filter sends every request through the authz app's proof chain (edge decisions land in the same durable audit log), and an Envoy **Lua filter** runs the guestbook's typed `rules.shen` *inside the proxy* (LuaJIT), rejecting malformed bodies at the edge with the origin's exact error strings. One typed rule file, four hosts. Runs standalone via `luajit examples/envoy/selftest.lua`; see [its README](envoy/README.md). | The last three (`configc/`, `policy/`, `crdt/`) are a themed trio: each extracts a correctness-critical kernel into one typed, portable Shen file that runs diff --git a/examples/envoy/README.md b/examples/envoy/README.md new file mode 100644 index 0000000..27053f0 --- /dev/null +++ b/examples/envoy/README.md @@ -0,0 +1,175 @@ +# Shen at the edge: Envoy in front of two Shen services + +The third piece of the web trilogy. [`examples/openresty`](../openresty) is a +Shen web app; [`examples/openresty-authz`](../openresty-authz) is Shen +authorization as a proof chain. This example puts **[Envoy](https://www.envoyproxy.io)** +— the standard edge/mesh proxy — in front of both, and uses each of Envoy's two +integration seams for exactly what it is good at: + +1. **`ext_authz` → the authz app.** Envoy sends every API request (method + + path + `authorization` header, no body) to the authz service *before* + routing it. The Prolog proof chain decides; a denial returns the discharge + report to the client; and **every edge decision lands in the same durable + audit log** as a direct API call. No authorization code lives in the proxy. + +2. **A Lua filter → `rules.shen` inside the proxy.** Envoy's Lua filter embeds + LuaJIT — shen-lua's primary host — so the guestbook's typed field rules run + *in the proxy itself*: a malformed POST gets its 400 **at the edge**, with + the same typed error strings the browser and the origin produce, before it + costs an upstream hop. + +Which makes it one `rules.shen`, enforced on **four hosts** from one typed +source: the browser (ShenScript, Ratatoskr-shaken), the Envoy edge (shen-lua on +Envoy's LuaJIT), the origin (shen-lua on OpenResty), and plain `luajit` in the +selftests. Proved sound by the sequent-calculus typechecker wherever shen-lua +loads it. + +``` +examples/envoy/ + envoy.yaml the edge: ext_authz (→ authz app) + the Lua filter + routing + filter.lua Shen in the proxy — boots shen-lua once per worker thread, + loads ../openresty/rules.shen under (tc +), 400s bad bodies + authz.conf the authz app as ext_authz backend: same app as + examples/openresty-authz, on :8081, seeding a "guestbook" + resource (run with -p examples/envoy so its log lives here) + selftest.lua the whole three-layer pipeline under plain luajit — no Envoy, + no nginx (a faithful fake of Envoy's request_handle drives the + real filter.lua; the real authz app decides; only the + guestbook upstream is a stand-in) +``` + +## Try it without Envoy + +```sh +luajit examples/envoy/selftest.lua +``` + +It runs requests through the chain in Envoy's exact filter order — ext_authz, +then the Lua filter, then the upstream — and checks *which layer* answers: + +``` +== gate 1: the proof chain holds the edge (ext_authz) == + no token -> 403 @ ext_authz (unauthenticated: ...) + carol: not a member -> 403 @ ext_authz (not a member of tenant acme) + unmapped path fails closed -> 403 @ ext_authz (unknown resource) + bob (viewer) may read -> 200 @ upstream + bob (viewer) may NOT post -> 403 @ ext_authz (requires the editor role) +== gate 2: typed rules.shen at the edge (the Lua filter) == + alice posts, missing name -> 400 @ lua-filter (name: is required) + ... +== every edge decision is in the durable audit log == + #9 carol read guestbook deny not a member of tenant acme + ... +OK — all cases passed (ext_authz + Lua filter + upstream) +``` + +## Run it for real + +Three processes, all from the repo root (`brew install envoy openresty` or your +platform's equivalents): + +```sh +# 1. the guestbook origin, :8080 — the openresty example, unchanged +mkdir -p examples/openresty/logs +openresty -p "$PWD/examples/openresty" -c nginx.conf + +# 2. the authz app as ext_authz backend, :8081 +mkdir -p examples/envoy/logs +openresty -p "$PWD/examples/envoy" -c authz.conf + +# 3. Envoy at the edge, :10000 (run from the repo root — filter.lua and the +# bytecode cache resolve against the cwd) +envoy -c examples/envoy/envoy.yaml --concurrency 2 +``` + +Then drive the edge: + +```sh +# no token → the proof chain denies at the edge; the guestbook never sees it +curl -s localhost:10000/api/messages +# carol has no membership → denial WITH the discharge report, from ext_authz +curl -s localhost:10000/api/messages -H 'authorization: Bearer tok-carol' +# bob is a viewer: reads pass, posts need the editor role +curl -s localhost:10000/api/messages -H 'authorization: Bearer tok-bob' +curl -s localhost:10000/api/messages -H 'authorization: Bearer tok-bob' \ + -d '{"name":"bob","message":"hi"}' +# alice (editor) is authorized — but the EDGE now runs the typed rules: +# this 400 comes from Envoy's Lua filter, not the origin +curl -s localhost:10000/api/messages -H 'authorization: Bearer tok-alice' \ + -d '{"message":"no name"}' +# a valid entry survives both gates and reaches the guestbook +curl -s localhost:10000/api/messages -H 'authorization: Bearer tok-alice' \ + -d '{"name":"ada","message":"through the edge"}' +# every one of those edge decisions is durably audited in the authz app +curl -s localhost:8081/api/admin/audit -d '{"token":"tok-admin"}' +``` + +## How it fits together + +``` + ┌────────────── Envoy :10000 ──────────────┐ +client ─ request ──► │ 1 ext_authz ────────────────────────────┼──► authz app :8081 (OpenResty) + │ GET /authz/api/messages │ (route "CHECK" Path ...) + │ + authorization header, no body │ the Prolog proof chain over + │ 200 = allow; else the denial body │ durable facts; every decision + │ (the discharge report) goes to │ appended to the audit log + │ the client verbatim │ + │ 2 lua filter: filter.lua │ + │ rules.shen, typed, on Envoy's │ + │ LuaJIT → 400 at the edge │ + │ 3 router │ + └────────────────────┬─────────────────────┘ + ▼ + guestbook :8080 (OpenResty) + re-runs the SAME rules.shen as the + authoritative check (defense in depth) +``` + +## What Envoy changes, coming from OpenResty + +Both embed LuaJIT, but with opposite philosophies — and the split above falls +straight out of the differences: + +- **Lua's role.** In OpenResty, Lua is the *application platform*: cosockets, + `lua_shared_dict`, timers, `init_worker`. In Envoy, Lua is a *scripting + hook* (`envoy_on_request`/`envoy_on_response`) for inspecting and mutating + traffic; its only sanctioned I/O is `handle:httpCall()` to a configured + cluster. +- **No shared state.** Every Envoy worker *thread* has its own Lua state — + "there is no truly global data." No shared dict means no in-proxy store, no + in-proxy cache, no LMDB. +- **Real logic is externalized.** Envoy's own answer to "I have serious + request logic" is `ext_authz` / `ext_proc`: call a service. That is exactly + where the authz app slots in — unchanged except for one new route. + +So the proxy gets **only the pure typed core** (validation — per-request, no +state, no I/O), and everything stateful (the proof chain's facts, the durable +log) stays in OpenResty, where cosockets and LMDB live. The one rule from the +OpenResty examples carries over verbatim: never Shen's blocking file I/O on the +request path. + +## The mechanics worth knowing + +- **Boot is per worker thread.** `filter.lua`'s top level is Envoy's analogue + of `init_worker_by_lua`: it runs once per worker thread at config load, and + each thread boots its own kernel (~1 s the very first run, tens of ms once + the bytecode cache exists; the cache is written to the cwd, so run Envoy + from a writable directory — the repo root). `--concurrency 2` keeps the + demo's boot cost and memory footprint small. +- **The check endpoint fails closed, twice.** An unmapped path maps to + resource `""` → owner tenant `""` → `denied "unknown resource"` (the app); + and `failure_mode_allow: false` means an unreachable authz app is a 403, + never an allow (Envoy). The check response body is a *typed* projection — + `check-response` in `authz.shen` is total over `decision` and structurally + cannot include document content, so the gateway cannot become a data leak. +- **Path resolution.** `filter.lua` finds the repo via its own file path (or + `$SHEN_LUA_ROOT`, or the cwd as a last resort). Envoy loads it with + `default_source_code: { filename: ... }`, resolved against the cwd. +- **FFI.** The native soa32 Prolog/typecheck engine wants LuaJIT's FFI, which + Envoy's bundled LuaJIT ships; in a build without it, shen-lua falls back to + the (slower, still correct) legacy engine. On arm64 with an older LuaJIT, + the same `SHEN_JIT=off` mitigation from the main README applies — it is + read from the environment, which Envoy passes through. +- **`x-shen-edge`.** The filter stamps `x-shen-edge: validated` on requests it + lets through (and `rejected` on its 400s), so the origin — and your access + logs — can see the edge ran the rules. diff --git a/examples/envoy/authz.conf b/examples/envoy/authz.conf new file mode 100644 index 0000000..439d01c --- /dev/null +++ b/examples/envoy/authz.conf @@ -0,0 +1,76 @@ +# examples/envoy/authz.conf — the authz app as Envoy's ext_authz backend. +# +# The SAME app as examples/openresty-authz (loaded from there via package.path) +# with two demo-topology differences: it listens on 8081 (Envoy's authz +# cluster; the guestbook keeps 8080), and its seed includes a "guestbook" +# resource under tenant acme — the resource the CHECK route maps /api/messages +# to. Run from the repo root with THIS example dir as the prefix, so its log +# lives here: +# +# mkdir -p examples/envoy/logs +# openresty -p "$PWD/examples/envoy" -c authz.conf + +daemon off; +worker_processes 1; # one worker: one kernel boot, one replay +pid logs/nginx-authz.pid; +error_log logs/error-authz.log info; + +events { worker_connections 256; } + +http { + access_log logs/access-authz.log; + + types { application/json json; text/plain txt; } + default_type application/json; + + # Repo root and the authz example dir on package.path: the prefix is + # .../examples/envoy/, so ../../ is the repo root and ../openresty-authz/ + # is where app.lua / store.lua / authz.shen / app.shen live. + init_by_lua_block { + local prefix = ngx.config.prefix() + package.path = prefix .. "../../?.lua;" + .. prefix .. "../openresty-authz/?.lua;" .. package.path + } + + # Boot Shen ONCE per worker, open the durable store, replay its log, and + # seed the demo world on first boot only. Identical to the openresty-authz + # config except for the extra "guestbook" resource + memberships. + init_worker_by_lua_block { + local app = require("app") + local Store = require("store") + local store = Store.new{ + codec = app.json, + backend = "file", + path = ngx.config.prefix() .. "logs/authz-log.jsonl", + } + if store.seq() == 0 then + store.seed_token("tok-admin", "admin", true) + store.seed_token("tok-alice", "alice", false) + store.seed_token("tok-bob", "bob", false) + store.seed_token("tok-carol", "carol", false) + -- the resource the edge CHECK maps /api/messages to + store.create("acme", "guestbook", "guestbook access marker") + store.grant("alice", "acme", "editor") -- may read AND post + store.grant("bob", "acme", "viewer") -- may read, not post + -- carol: no membership — denied with a discharge report + end + app.use_store(store) + } + + server { + listen 8081; + + # Envoy's ext_authz check requests: " /authz" + # + the authorization header -> (route "CHECK" Path ...) in Shen. + location /authz/ { + content_by_lua_block { require("app").handle() } + } + + # The app's own JSON API stays reachable (admin: grant/revoke/audit), + # so you can revoke access and read the durable decision log while + # Envoy is enforcing at the edge. + location /api/ { + content_by_lua_block { require("app").handle() } + } + } +} diff --git a/examples/envoy/envoy.yaml b/examples/envoy/envoy.yaml new file mode 100644 index 0000000..397711b --- /dev/null +++ b/examples/envoy/envoy.yaml @@ -0,0 +1,103 @@ +# examples/envoy/envoy.yaml — Envoy at the edge, in front of two Shen services. +# +# Run FROM THE REPO ROOT (the Lua filter resolves shen-lua relative to itself, +# but the filename below is cwd-relative): +# +# envoy -c examples/envoy/envoy.yaml --concurrency 2 +# +# with the two OpenResty upstreams running (see README.md): +# 127.0.0.1:8080 the guestbook (examples/openresty, unchanged) +# 127.0.0.1:8081 the authz app (examples/openresty-authz via authz.conf) +# +# The HTTP filter chain, in order: +# 1. ext_authz — every /api/* request is sent (method + path + authorization +# header, no body) to the authz app's /authz/* endpoint. The +# Shen Prolog proof chain decides; 200 allows, anything else +# is returned to the client verbatim (the discharge report). +# 2. lua — filter.lua boots shen-lua ONCE PER WORKER THREAD and runs +# the guestbook's typed rules.shen on POST bodies: malformed +# requests get their 400 at the edge, with the same typed +# error strings the browser and the origin produce. +# 3. router — what survives both gates reaches the guestbook. +# +# --concurrency 2 keeps the demo light: each Envoy worker thread has its own +# Lua state, so each boots its own Shen kernel (~1 s the very first run, tens +# of ms once shen-lua's bytecode cache exists in the cwd). + +static_resources: + listeners: + - name: edge + address: + socket_address: { address: 127.0.0.1, port_value: 10000 } + filter_chains: + - filters: + - name: envoy.filters.network.http_connection_manager + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager + stat_prefix: shen_edge + route_config: + name: local_routes + virtual_hosts: + - name: app + domains: ["*"] + routes: + # The API: both gates on, then the guestbook. + - match: { prefix: "/api/" } + route: { cluster: guestbook } + # Everything else (the front-end HTML + the shaken browser + # validator) is static content: both gates off for this route. + - match: { prefix: "/" } + route: { cluster: guestbook } + typed_per_filter_config: + envoy.filters.http.ext_authz: + "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute + disabled: true + envoy.filters.http.lua: + "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.LuaPerRoute + disabled: true + http_filters: + - name: envoy.filters.http.ext_authz + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz + transport_api_version: V3 + # Fail closed: if the authz app is down, deny (403), never allow. + failure_mode_allow: false + http_service: + server_uri: + uri: http://127.0.0.1:8081 + cluster: authz + timeout: 1s + # The check request is " /authz" + # + the authorization header (forwarded by default, along with + # Host/Method/Path/Content-Length). No request body is sent. + path_prefix: "/authz" + - name: envoy.filters.http.lua + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua + default_source_code: + filename: examples/envoy/filter.lua + - name: envoy.filters.http.router + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router + + clusters: + - name: guestbook + type: STATIC + connect_timeout: 1s + load_assignment: + cluster_name: guestbook + endpoints: + - lb_endpoints: + - endpoint: + address: + socket_address: { address: 127.0.0.1, port_value: 8080 } + - name: authz + type: STATIC + connect_timeout: 1s + load_assignment: + cluster_name: authz + endpoints: + - lb_endpoints: + - endpoint: + address: + socket_address: { address: 127.0.0.1, port_value: 8081 } diff --git a/examples/envoy/filter.lua b/examples/envoy/filter.lua new file mode 100644 index 0000000..025980f --- /dev/null +++ b/examples/envoy/filter.lua @@ -0,0 +1,109 @@ +-- examples/envoy/filter.lua — Shen at the edge: an Envoy Lua HTTP filter. +-- +-- Envoy's Lua filter embeds LuaJIT — the same host tier as OpenResty — so the +-- typed core of the guestbook example (../openresty/rules.shen) can run INSIDE +-- the proxy: a malformed POST is rejected at the edge, with the SAME typed +-- error strings the browser (ShenScript) and the origin (shen-lua on +-- OpenResty) produce, before it ever costs an upstream hop. One rules.shen, +-- now enforced on a third host. +-- +-- What belongs here and what doesn't: Envoy's Lua environment is per worker +-- THREAD, has no shared dicts, no cosockets, and no timers — its only +-- sanctioned I/O is handle:httpCall() to a configured cluster. So the edge +-- gets ONLY the pure typed core (validation); everything stateful — the authz +-- proof chain, the durable store, the audit log — stays in the OpenResty +-- services this proxy fronts (see envoy.yaml + README.md). +-- +-- This top-level chunk runs ONCE per Envoy worker thread, when the filter +-- loads the script — Envoy's analogue of init_worker_by_lua. The kernel boot +-- (~1 s cold, tens of ms from the bytecode cache) happens here, never per +-- request. envoy_on_request below is the per-request hook. + +-- Resolve the repo root: an explicit SHEN_LUA_ROOT wins; otherwise derive it +-- from this file's own path (works under `luajit selftest.lua` and under an +-- Envoy that names the chunk after the source_code filename); last resort is +-- the cwd, which is right when Envoy is run from the repo root as the README +-- says. +local SRC = debug.getinfo(1, "S").source +local FILTER_DIR = SRC:match("^@(.*)[/\\][^/\\]+$") +local ROOT = os.getenv("SHEN_LUA_ROOT") + or (FILTER_DIR and FILTER_DIR .. "/../..") + or "." +package.path = ROOT .. "/?.lua;" .. package.path + +local shen = require("shen") +local IO = require("lua_interop") +local P = shen.prims + +-- Envoy bundles no JSON codec, so always use the repo's self-contained shim +-- (the same one the selftests use; cjson-compatible surface). +local cjson = assert(loadfile(ROOT .. "/examples/openresty/json_shim.lua"))() + +shen.boot{ quiet = true } + +-- The typed core, loaded under (tc +): the SAME file the origin server loads +-- and the browser build is shaken from. A type error in a rule aborts the +-- proxy's script load — the edge never runs unproved rules. +shen.eval("(tc +)") +P.F["load"](ROOT .. "/examples/openresty/rules.shen") +shen.eval("(tc -)") + +local validate = IO.fn("validate-message") -- val -> list of error strings + +-- Lua (decoded JSON) -> the tagged `val` shape rules.shen pattern-matches. +-- Identical to the marshaling in the two app.lua glues. +local sym = IO.sym +local function to_val(v) + local t = type(v) + if t == "string" then return { sym("s"), v } end + if t == "number" then return { sym("n"), v } end + if t == "boolean" then return { sym("b"), v } end + if t == "table" then + if v[1] ~= nil or next(v) == nil then + local a = {} + for i, e in ipairs(v) do a[i] = to_val(e) end + return { sym("arr"), a } + end + local es, i = {}, 0 + for k, val in pairs(v) do + if type(k) == "string" then i = i + 1; es[i] = { k, to_val(val) } end + end + return { sym("obj"), es } + end + return { sym("s"), tostring(v) } +end + +-- Validate a raw request body; nil means "clean, let it through", otherwise +-- an array of error strings — the same strings the origin would produce, +-- because it is the same rules.shen producing them. +local function edge_errors(raw) + local decoded, err = cjson.decode(raw) + if decoded == nil then return { "invalid JSON: " .. tostring(err) } end + local errs = validate(to_val(decoded)) -- an empty Shen list marshals to nil + if errs == nil or #errs == 0 then return nil end + return errs +end + +-- ---- the per-request hook ---------------------------------------------------- +-- Only guestbook creations carry a body worth ruling on; everything else +-- passes through untouched (authorization already happened — the ext_authz +-- filter runs before this one in envoy.yaml's filter chain). +function envoy_on_request(handle) + local headers = handle:headers() + local method = headers:get(":method") + local path = (headers:get(":path") or ""):match("^[^?]*") + if method ~= "POST" or path ~= "/api/messages" then return end + + local body = handle:body() -- buffers the full body (yields) + local raw = body and body:getBytes(0, body:length()) or "" + local errs = edge_errors(raw) + if errs then + handle:respond( + { [":status"] = "400", + ["content-type"] = "application/json", + ["x-shen-edge"] = "rejected" }, + cjson.encode({ errors = errs })) + return -- never reached: respond() ends the coroutine + end + headers:add("x-shen-edge", "validated") -- visible upstream: the edge ran the rules +end diff --git a/examples/envoy/selftest.lua b/examples/envoy/selftest.lua new file mode 100644 index 0000000..c37bbbd --- /dev/null +++ b/examples/envoy/selftest.lua @@ -0,0 +1,178 @@ +-- examples/envoy/selftest.lua — the three-layer pipeline, off-Envoy/off-nginx. +-- +-- luajit examples/envoy/selftest.lua (from the repo root) +-- +-- One luajit process plays Envoy's part and chains the REAL pieces in Envoy's +-- exact filter order: +-- +-- 1. ext_authz -> the real authz app (examples/openresty-authz): the +-- Prolog proof chain decides on method + path + token, +-- appending every decision to the durable audit log. +-- 2. lua filter -> the real filter.lua, driven through a faithful fake of +-- Envoy's request_handle (headers/body/respond): the typed +-- rules.shen rejects malformed bodies at the "edge". +-- 3. upstream -> a 10-line stand-in for the guestbook (the real one is +-- exercised by examples/openresty/selftest.lua; only +-- requests that survived BOTH gates ever reach it). +-- +-- Both Shen loads happen in this one process — filter.lua boots the kernel +-- and loads rules.shen (tc +), then the authz app loads authz.shen (tc +) and +-- app.shen — which is exactly what makes the shared-environment story +-- testable without Envoy or nginx. + +local root = arg[0]:match("^(.*)/examples/envoy/[^/]+$") or "." +package.path = root .. "/?.lua;" + .. root .. "/examples/openresty-authz/?.lua;" .. package.path + +-- ---- layer 2: the edge — filter.lua defines envoy_on_request ---------------- +dofile(root .. "/examples/envoy/filter.lua") +assert(type(envoy_on_request) == "function", "filter.lua did not define envoy_on_request") + +-- ---- layer 1: the authz service (real app + durable store) ------------------ +local app = require("app") -- examples/openresty-authz/app.lua +local Store = require("store") +local cjson = app.json + +local log_path = os.tmpname(); os.remove(log_path) +local store = Store.new{ codec = cjson, backend = "file", path = log_path } +-- the same world authz.conf seeds +store.seed_token("tok-admin", "admin", true) +store.seed_token("tok-alice", "alice", false) +store.seed_token("tok-bob", "bob", false) +store.seed_token("tok-carol", "carol", false) +store.create("acme", "guestbook", "guestbook access marker") +store.grant("alice", "acme", "editor") +store.grant("bob", "acme", "viewer") +app.use_store(store) + +-- ---- layer 3: the guestbook upstream (stand-in) ------------------------------ +local rows = {} +local function upstream(method, path, body) + if method == "GET" and path == "/api/messages" then + return 200, { messages = rows } + end + if method == "POST" and path == "/api/messages" then + rows[#rows + 1] = { name = body.name, message = body.message } + return 201, { ok = true } + end + return 404, { error = "not found" } +end + +-- ---- a faithful fake of Envoy's request_handle ------------------------------ +-- Speaks exactly the surface filter.lua uses: headers():get/add, body(): +-- length/getBytes, respond(headers, body). +local function fake_handle(method, path, raw_body) + local hdrs, added, responded = { [":method"] = method, [":path"] = path }, {}, nil + local h = {} + function h:headers() + return { + get = function(_, k) return hdrs[k] end, + add = function(_, k, v) added[k] = v end, + } + end + function h:body() + if not raw_body or raw_body == "" then return nil end + return { + length = function() return #raw_body end, + getBytes = function(_, off, len) return raw_body:sub(off + 1, off + len) end, + } + end + function h:respond(headers, body) responded = { headers = headers, body = body } end + function h:logInfo() end + return h, function() return responded end, added +end + +-- ---- "Envoy": the filter chain, in envoy.yaml's order ------------------------ +-- Returns status, decoded body, the stage that produced the response, and the +-- headers the Lua filter added to a request it let through. +local function through_envoy(method, path, token, body_tbl, raw_override) + -- 1. ext_authz: Envoy forwards " /authz" + the authorization + -- header; app.lua's glue turns that into (route "CHECK" Path ...). + local astatus, aresp = app.dispatch("CHECK", path:match("^[^?]*"), + { token = token or "", method = method }) + if astatus ~= 200 then return astatus, aresp, "ext_authz", {} end + -- 2. the Lua filter (filter.lua, for real) + local raw = raw_override or (body_tbl and cjson.encode(body_tbl)) or nil + local h, responded, added = fake_handle(method, path, raw) + envoy_on_request(h) + local r = responded() + if r then return tonumber(r.headers[":status"]), cjson.decode(r.body), "lua-filter", added end + -- 3. the upstream + local ustatus, ubody = upstream(method, path:match("^[^?]*"), body_tbl) + return ustatus, ubody, "upstream", added +end + +-- ---- the scenario ------------------------------------------------------------- +local fail = 0 +local last_resp, last_added +local function expect(label, want_status, want_stage, method, path, token, body, raw) + local status, resp, stage, added = through_envoy(method, path, token, body, raw) + last_resp, last_added = resp, added + local note = resp and (resp.error or (resp.errors and table.concat(resp.errors, "; "))) or "" + print((" %-40s -> %d @ %-9s %s"):format(label, status, stage, + note ~= "" and ("(" .. note .. ")") or "")) + if status ~= want_status or stage ~= want_stage then + fail = fail + 1 + print((" FAIL: expected %d @ %s"):format(want_status, want_stage)) + end + return resp +end +local function check(label, cond) + print(" " .. (cond and "ok " or "FAIL") .. " " .. label) + if not cond then fail = fail + 1 end +end + +print("== gate 1: the proof chain holds the edge (ext_authz) ==") +expect("no token", 403, "ext_authz", "GET", "/api/messages") +expect("carol: not a member", 403, "ext_authz", "GET", "/api/messages", "tok-carol") +expect("bad token", 403, "ext_authz", "GET", "/api/messages", "tok-nope") +expect("unmapped path fails closed", 403, "ext_authz", "GET", "/api/nope", "tok-alice") +expect("bob (viewer) may read", 200, "upstream", "GET", "/api/messages", "tok-bob") +expect("bob (viewer) may NOT post", 403, "ext_authz", "POST", "/api/messages", "tok-bob", + { name = "bob", message = "hi" }) + +print("\n== gate 2: typed rules.shen at the edge (the Lua filter) ==") +expect("alice posts, missing name", 400, "lua-filter", "POST", "/api/messages", "tok-alice", + { message = "anon" }) +check("edge error == the origin's typed string", + last_resp.errors and last_resp.errors[1] == "name: is required") +expect("alice posts, blank message", 400, "lua-filter", "POST", "/api/messages", "tok-alice", + { name = "ada", message = "" }) +check("edge error == the origin's typed string", + last_resp.errors and last_resp.errors[1] == "message: must be 1..280 characters") +expect("alice posts a non-object", 400, "lua-filter", "POST", "/api/messages", "tok-alice", + { "not", "an", "object" }) +check("edge error == the origin's typed string", + last_resp.errors and last_resp.errors[1] == "body: must be a JSON object") +expect("alice posts broken JSON", 400, "lua-filter", "POST", "/api/messages", "tok-alice", + nil, "{ definitely not json") +check("edge names the parse failure", + last_resp.errors and last_resp.errors[1]:find("^invalid JSON") ~= nil) + +print("\n== what survives both gates reaches the guestbook ==") +expect("alice posts a valid entry", 201, "upstream", "POST", "/api/messages", "tok-alice", + { name = "ada", message = "through the edge" }) +check("the Lua filter stamped x-shen-edge: validated", last_added["x-shen-edge"] == "validated") +expect("alice reads it back", 200, "upstream", "GET", "/api/messages", "tok-alice") +check("one row stored upstream", #rows == 1 and rows[1].name == "ada") + +print("\n== revocation: durable state changes the edge's answer ==") +local rstatus = app.dispatch("POST", "/api/admin/revoke", + { token = "tok-admin", user = "alice", resource = "guestbook" }) +check("admin revokes alice (direct API)", rstatus == 200) +expect("alice now denied at the edge",403, "ext_authz", "GET", "/api/messages", "tok-alice") + +print("\n== every edge decision is in the durable audit log ==") +local _, audit = app.dispatch("POST", "/api/admin/audit", { token = "tok-admin" }) +local edge_rows = {} +for _, row in ipairs((audit and audit.log) or {}) do + if row.resource == "guestbook" or row.resource == "" then + edge_rows[#edge_rows + 1] = row + print((" #%-2d %-6s %-6s %-10s %-6s %s"):format( + row.seq, row.user, row.action, row.resource, row.decision, row.reason)) + end +end +check("the log holds the whole session (13 edge decisions)", #edge_rows == 13) + +if fail == 0 then print("\nOK — all cases passed (ext_authz + Lua filter + upstream)") +else print(("\n%d case(s) FAILED"):format(fail)); os.exit(1) end diff --git a/examples/openresty-authz/README.md b/examples/openresty-authz/README.md index 33fb9df..2df3dba 100644 --- a/examples/openresty-authz/README.md +++ b/examples/openresty-authz/README.md @@ -244,3 +244,15 @@ curl -s localhost:8080/api/admin/audit -d '{"token":"tok-admin"}' - **Never use Shen's blocking file I/O under nginx.** The file backend is for the off-nginx demo; under OpenResty use the lmdb backend (or reach a DB via the non-blocking cosocket libraries), exactly as the guestbook README warns. + +## This app as a fleet-wide gate: Envoy `ext_authz` + +The app also speaks Envoy's [`ext_authz`](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ext_authz_filter) +protocol: Envoy forwards each edge request as ` /authz` plus its +`authorization` header, the glue dispatches `(route "CHECK" Path ...)`, and the +**same** `authorize-*` gate decides — so edge decisions run the same proof +chain and land in the same durable audit log as direct API calls, and every +service behind Envoy gets this authorization without embedding anything. The +response body is `check-response` in `authz.shen`: total over `decision` and +structurally unable to include document content, so the gateway cannot leak. +See [`examples/envoy`](../envoy) for the full three-layer setup. diff --git a/examples/openresty-authz/app.lua b/examples/openresty-authz/app.lua index 96e3859..6cb781e 100644 --- a/examples/openresty-authz/app.lua +++ b/examples/openresty-authz/app.lua @@ -118,6 +118,19 @@ local M = { dispatch = dispatch, to_val = to_val, from_val = from_val, function M.handle() local method = ngx.req.get_method() local path = ngx.var.uri + -- ext_authz gateway checks (see examples/envoy/): Envoy forwards the edge + -- request as " /authz" plus its authorization + -- header. No body is read — the check decides on method + path + token alone, + -- so Envoy never has to buffer request bodies for authorization. + if path:sub(1, 7) == "/authz/" then + local auth_hdr = ngx.req.get_headers()["authorization"] or "" + local token = auth_hdr:match("^[Bb]earer%s+(.*)$") or auth_hdr + local status, body = dispatch("CHECK", path:sub(7), { token = token, method = method }) + ngx.status = status + ngx.header.content_type = "application/json" + ngx.say(cjson.encode(body)) + return + end local decoded if method == "POST" or method == "PUT" then ngx.req.read_body() diff --git a/examples/openresty-authz/app.shen b/examples/openresty-authz/app.shen index cc70b32..1da2d5e 100644 --- a/examples/openresty-authz/app.shen +++ b/examples/openresty-authz/app.shen @@ -188,14 +188,45 @@ (freeze [200 [obj [["log" [arr (map (function logrow->val) (host-audit))]]]]])) _ -> (bad-body)) +\\ -- ext_authz gateway check (see examples/envoy/) ----------------------------- +\\ Envoy's ext_authz filter forwards each edge request — original method, path +\\ (prefixed /authz), and authorization header — to this app BEFORE routing it +\\ upstream. The glue (app.lua) turns that into +\\ (route "CHECK" Path [obj [["token" ...] ["method" ...]]]) +\\ and the SAME authorize-* gate decides, so edge decisions run the same proof +\\ chain and land in the same durable audit log as direct API calls. An +\\ unmapped path gets resource "" -> owner tenant "" -> denied "unknown +\\ resource": the gateway fails closed. + +\\ which protected resource does an upstream path correspond to? +(define resource-for + "/api/messages" -> "guestbook" + _ -> "") + +(define read-method? + M -> (element? M ["GET" "HEAD" "OPTIONS"])) + +(define do-check + Path [obj Es] -> (respond-check + (if (read-method? (sfield "method" Es)) + (authorize-read (sfield "token" Es) (resource-for Path)) + (authorize-write (sfield "token" Es) (resource-for Path)))) + _ _ -> (bad-body)) + +\\ status and body both come from the typed core, each total over `decision`; +\\ check-response structurally cannot include document content. +(define respond-check + D -> [(decision-status D) (check-response D)]) + \\ -- the router --------------------------------------------------------------- \\ Each handler validates its own body shape (an [obj ...] or a 400), so a \\ bodyless POST to a real route is a 400, not a 404. (define route - "POST" "/api/read" B -> (do-read B) - "POST" "/api/write" B -> (do-write B) - "POST" "/api/admin/grant" B -> (do-grant B) - "POST" "/api/admin/revoke" B -> (do-revoke B) - "POST" "/api/admin/create" B -> (do-create B) - "POST" "/api/admin/audit" B -> (do-audit B) + "POST" "/api/read" B -> (do-read B) + "POST" "/api/write" B -> (do-write B) + "POST" "/api/admin/grant" B -> (do-grant B) + "POST" "/api/admin/revoke" B -> (do-revoke B) + "POST" "/api/admin/create" B -> (do-create B) + "POST" "/api/admin/audit" B -> (do-audit B) + "CHECK" Path B -> (do-check Path B) _ _ _ -> [404 [obj [["error" [s "not found"]]]]]) diff --git a/examples/openresty-authz/authz.shen b/examples/openresty-authz/authz.shen index 683fd3f..11b1ae7 100644 --- a/examples/openresty-authz/authz.shen +++ b/examples/openresty-authz/authz.shen @@ -76,3 +76,17 @@ {decision --> number} [granted _ _ _] -> 200 [denied _] -> 403) + +\\ -- check-response: the gateway projection ----------------------------------- +\\ The body of an ext_authz CHECK answer (see examples/envoy/): a gateway asks +\\ "may this request proceed?" and must never receive the document itself. This +\\ is render-doc's dual, with the same load-time guarantee: total over +\\ `decision`, and NO shape of `decision` places content in a check response — +\\ the authorization gateway structurally cannot become a data leak. +(define check-response + {decision --> val} + [granted U T R] -> [obj [["ok" [b true]] + ["user" [s U]] + ["tenant" [s T]] + ["resource" [s R]]]] + [denied Why] -> [obj [["ok" [b false]] ["error" [s Why]]]]) diff --git a/examples/openresty-authz/nginx.conf b/examples/openresty-authz/nginx.conf index e0ca8b8..c13c3fc 100644 --- a/examples/openresty-authz/nginx.conf +++ b/examples/openresty-authz/nginx.conf @@ -87,5 +87,13 @@ http { location /api/ { content_by_lua_block { require("app").handle() } } + + # ext_authz gateway checks: Envoy (see examples/envoy/) forwards each + # edge request here as " /authz" + authorization header; + # the glue maps it to (route "CHECK" Path ...) — same proof chain, same + # durable audit log. 200 = allow, anything else = deny. + location /authz/ { + content_by_lua_block { require("app").handle() } + } } } diff --git a/examples/openresty-authz/selftest.lua b/examples/openresty-authz/selftest.lua index ee9b858..8bc1475 100644 --- a/examples/openresty-authz/selftest.lua +++ b/examples/openresty-authz/selftest.lua @@ -131,6 +131,16 @@ local function run_scenario(title, open_store, make_auth) expect("bodyless POST -> 400", 400, "POST", "/api/read") expect("audit needs admin -> 403", 403, "POST", "/api/admin/audit", { token = "tok-alice" }) + print("\n== ext_authz gateway checks (the Envoy edge — see examples/envoy) ==") + -- (route "CHECK" Path Body) is what app.lua's glue dispatches when Envoy's + -- ext_authz filter forwards an edge request as " /authz". + expect("create acme/guestbook", 200, "POST", "/api/admin/create", + { token = "tok-admin", tenant = "acme", resource = "guestbook", content = "guestbook access marker" }) + expect("edge GET by alice granted", 200, "CHECK", "/api/messages", { token = "tok-alice", method = "GET" }) + expect("edge POST by bob needs editor",403,"CHECK", "/api/messages", { token = "tok-bob", method = "POST" }) + expect("edge unmapped path fails closed",403,"CHECK","/api/nope", { token = "tok-alice", method = "GET" }) + expect("bodyless CHECK -> 400", 400, "CHECK", "/api/messages") + print("\n== durable execution: simulate a restart (reopen + replay the log) ==") local seq_before = s.seq() local s2 = open_store() -- brand-new store object, same durable log From 3b17fe62867d6ccea946c1234be90d1b0b14a97f Mon Sep 17 00:00:00 2001 From: Reuben Brooks Date: Tue, 11 Aug 2026 08:44:16 -0500 Subject: [PATCH 2/4] envoy example: document verified deployment mechanics Stood the stack up against Homebrew Envoy 1.39 on an arm64 Mac and folded the findings into the docs: - SHEN_KERNEL_CACHE: the bytecode cache is keyed to the exact LuaJIT build, so Envoy's LuaJIT and the local luajit rewrite each other's cache under the shared default path; a dedicated path fixes it. - SHEN_JIT=off on macOS: the hardened runtime denies Envoy's binary executable trace memory, so LuaJIT thrashes on failed trace compiles (measured ~550x slowdown, 40-66 s kernel boots). Interpreted, the edge boots in ~3 s warm and serves in 3-6 ms. - FFI confirmed present in Envoy's LuaJIT (soa32 engine runs as-is). Co-Authored-By: Claude Fable 5 --- examples/envoy/README.md | 41 ++++++++++++++++++++++++++------------- examples/envoy/envoy.yaml | 8 ++++++-- 2 files changed, 34 insertions(+), 15 deletions(-) diff --git a/examples/envoy/README.md b/examples/envoy/README.md index 27053f0..efd674c 100644 --- a/examples/envoy/README.md +++ b/examples/envoy/README.md @@ -77,9 +77,11 @@ openresty -p "$PWD/examples/openresty" -c nginx.conf mkdir -p examples/envoy/logs openresty -p "$PWD/examples/envoy" -c authz.conf -# 3. Envoy at the edge, :10000 (run from the repo root — filter.lua and the -# bytecode cache resolve against the cwd) -envoy -c examples/envoy/envoy.yaml --concurrency 2 +# 3. Envoy at the edge, :10000 (run from the repo root — filter.lua resolves +# against the cwd; the two env vars are explained under "The mechanics +# worth knowing", and SHEN_JIT=off matters on macOS) +SHEN_KERNEL_CACHE=examples/envoy/logs/kernel-cache.bin SHEN_JIT=off \ + envoy -c examples/envoy/envoy.yaml --concurrency 2 ``` Then drive the edge: @@ -151,11 +153,24 @@ request path. ## The mechanics worth knowing - **Boot is per worker thread.** `filter.lua`'s top level is Envoy's analogue - of `init_worker_by_lua`: it runs once per worker thread at config load, and - each thread boots its own kernel (~1 s the very first run, tens of ms once - the bytecode cache exists; the cache is written to the cwd, so run Envoy - from a writable directory — the repo root). `--concurrency 2` keeps the - demo's boot cost and memory footprint small. + of `init_worker_by_lua`: it runs once per worker thread at config load — + never per request. `--concurrency 2` keeps the demo's boot cost and memory + footprint small. +- **Two environment variables worth setting** (both findings from running this + against Homebrew Envoy 1.39 on an arm64 Mac): + - `SHEN_KERNEL_CACHE=examples/envoy/logs/kernel-cache.bin` — the bytecode + cache is keyed to the exact LuaJIT build, and Envoy's LuaJIT is not your + local `luajit`: with the shared default path (`.shen-kernel-cache.bin` in + the cwd) the two hosts invalidate and rewrite each other's cache every + time you alternate. A dedicated path gives Envoy its own warm cache. + - `SHEN_JIT=off` — macOS's hardened runtime denies Envoy's binary executable + trace memory, so with the JIT nominally on, every hot loop attempts a + trace, hits `failed to allocate mcode memory`, and retries: measured, a + 20M-iteration loop ran ~550× slower than local `luajit`, and the kernel + boot took 40–66 s. `jit.off()` makes it a clean interpreter: **~3 s to + serving** (warm cache) and edge requests at **3–6 ms**. Boot is the only + JIT-hungry phase — per-request validation is tiny either way. Linux Envoy + builds can generally allocate mcode; try without it there first. - **The check endpoint fails closed, twice.** An unmapped path maps to resource `""` → owner tenant `""` → `denied "unknown resource"` (the app); and `failure_mode_allow: false` means an unreachable authz app is a 403, @@ -165,11 +180,11 @@ request path. - **Path resolution.** `filter.lua` finds the repo via its own file path (or `$SHEN_LUA_ROOT`, or the cwd as a last resort). Envoy loads it with `default_source_code: { filename: ... }`, resolved against the cwd. -- **FFI.** The native soa32 Prolog/typecheck engine wants LuaJIT's FFI, which - Envoy's bundled LuaJIT ships; in a build without it, shen-lua falls back to - the (slower, still correct) legacy engine. On arm64 with an older LuaJIT, - the same `SHEN_JIT=off` mitigation from the main README applies — it is - read from the environment, which Envoy passes through. +- **FFI.** The native soa32 Prolog/typecheck engine wants LuaJIT's FFI, and + Envoy's bundled LuaJIT ships it (verified: `require("ffi")` loads and the + typed `rules.shen` load — which runs the typechecker — works in-proxy). In a + build without it, shen-lua falls back to the (slower, still correct) legacy + engine. - **`x-shen-edge`.** The filter stamps `x-shen-edge: validated` on requests it lets through (and `rejected` on its 400s), so the origin — and your access logs — can see the edge ran the rules. diff --git a/examples/envoy/envoy.yaml b/examples/envoy/envoy.yaml index 397711b..4f5619f 100644 --- a/examples/envoy/envoy.yaml +++ b/examples/envoy/envoy.yaml @@ -21,8 +21,12 @@ # 3. router — what survives both gates reaches the guestbook. # # --concurrency 2 keeps the demo light: each Envoy worker thread has its own -# Lua state, so each boots its own Shen kernel (~1 s the very first run, tens -# of ms once shen-lua's bytecode cache exists in the cwd). +# Lua state, so each boots its own Shen kernel. Two environment variables are +# worth setting when launching Envoy — SHEN_KERNEL_CACHE (a dedicated +# bytecode-cache path, so Envoy's LuaJIT and your local luajit stop rewriting +# each other's cache) and, on macOS, SHEN_JIT=off (the hardened runtime denies +# LuaJIT trace memory; the interpreter boots in ~3 s and serves in single-digit +# ms). See README.md, "The mechanics worth knowing". static_resources: listeners: From 2ef93b7f4176e4fbefa017feb7f167e3723e50f8 Mon Sep 17 00:00:00 2001 From: Reuben Brooks Date: Tue, 11 Aug 2026 09:38:10 -0500 Subject: [PATCH 3/4] boot: per-build bytecode caches + auto-fallback when the JIT can't compile MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes the two issues surfaced by running shen-lua inside Envoy's Lua filter (examples/envoy), both previously documented as manual env-var workarounds: 1. The default kernel cache path is now per Lua build (.shen-kernel-cache..bin). Bytecode is only portable within the exact build, so two hosts sharing the old single path (local luajit vs OpenResty's or Envoy's embedded LuaJIT) invalidated and rewrote each other's cache on every alternation, recompiling the kernel every time. Each build now keeps its own warm cache; writing a default-path cache removes the obsolete shared-path file; SHEN_KERNEL_CACHE still relocates or disables. Verified: three different LuaJIT builds coexisting, each warm-booting from its own file. 2. boot.lua now probes whether the JIT can actually materialize a trace (compile one throwaway hot loop under a jit.attach trace watcher). Hosts that report jit.status() == true but are denied executable memory — macOS hardened-runtime binaries embedding LuaJIT, e.g. Envoy — otherwise thrash on "failed to allocate mcode memory" re-recording aborts (measured ~550x on a probe loop; 40-66 s kernel boots). On a failed probe the JIT is turned off with a stderr note: measured 2 s cold / 1 s warm to serving, 1.5-3 ms edge requests. Healthy hosts (local luajit, OpenResty nginx) compile the one probe trace and are unchanged. SHEN_JIT=on skips the probe; SHEN_JIT=off still forces the interpreter. Kernel test suite: ok. All three example selftests pass. Envoy example re-verified live with NO env vars. Docs updated to match (the envoy README's env-var guidance now describes the automatic behavior). Co-Authored-By: Claude Fable 5 --- .gitignore | 4 +-- README.md | 7 ++++-- boot.lua | 47 +++++++++++++++++++++++++++++++++++- examples/envoy/README.md | 46 ++++++++++++++++++++--------------- examples/envoy/envoy.yaml | 10 +++----- examples/openresty/README.md | 5 ++-- 6 files changed, 86 insertions(+), 33 deletions(-) diff --git a/.gitignore b/.gitignore index 44e2624..fa036c9 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,6 @@ .claude/ -.shen-kernel-cache.bin -.shen-kernel-cache.bin.tmp +.shen-kernel-cache*.bin +.shen-kernel-cache*.bin.tmp build/shen-bundle.lua .fasl-test/ luacov.stats.out diff --git a/README.md b/README.md index 50b567b..db0430b 100644 --- a/README.md +++ b/README.md @@ -123,8 +123,11 @@ Two caches make warm starts near-instant (both content-keyed, both safe to delete at any time): * **Kernel bytecode cache** — the compiled kernel is `string.dump`ed after the - first boot (`.shen-kernel-cache.bin`); warm boots load it in **~30 ms** - instead of recompiling (~1 s). + first boot (`.shen-kernel-cache..bin`, one file per exact Lua build, + since bytecode is not portable across builds — so e.g. your `luajit` and an + embedded OpenResty/Envoy LuaJIT each keep their own warm cache instead of + invalidating each other's); warm boots load it in **~30 ms** instead of + recompiling (~1 s). * **User fasl cache** — `(load "prog.shen")` records its compiled chunks and replays them on later runs, skipping the reader, macroexpansion *and typechecking* (SBCL-fasl semantics: it typechecked when it compiled). diff --git a/boot.lua b/boot.lua index 62d72e0..7df3a42 100644 --- a/boot.lua +++ b/boot.lua @@ -53,6 +53,35 @@ do pcall(jit.opt.start, "sizemcode=2048", "maxmcode=131072", "maxtrace=8000", "maxside=400") end + -- Some hosts leave the JIT nominally ON but deny the process executable + -- trace memory: macOS hardened-runtime binaries embedding LuaJIT (e.g. + -- Envoy's Lua filter — no JIT entitlement) report jit.status() == true, + -- yet every trace attempt aborts with "failed to allocate mcode memory" + -- and is re-attempted on the next hot path. Measured in Envoy 1.39 on + -- an arm64 Mac: a hot loop ran ~550x slower than plain interpretation + -- and kernel boot took 40-66 s (vs ~3 s interpreted). Detect it + -- directly — compile one throwaway hot loop and watch for a trace + -- "stop" event — and fall back to the interpreter. SHEN_JIT=on skips + -- the probe (explicit opt-in for hosts you have verified). + if os.getenv("SHEN_JIT") ~= "on" and jit.status and jit.status() + and jit.attach then + local compiled = false + local watcher = function(what) + if what == "stop" then compiled = true end + end + if pcall(jit.attach, watcher, "trace") then + local mk = loadstring or load + local probe = mk("local s = 0 for i = 1, 400 do s = s + i end return s") + if probe then for _ = 1, 3 do probe() end end + pcall(jit.attach, watcher) -- detach + if not compiled then + disable_jit() + io.stderr:write("shen-lua: the JIT reports enabled but compiled no" + .. " trace (executable memory denied? hardened host?); running" + .. " interpreted. Set SHEN_JIT=on to skip this probe.\n") + end + end + end end end -- GC tuning. Compiled-KL workloads are cons-churn-heavy (jit.p on urdr's @@ -211,12 +240,21 @@ local function fnv1a(s, h) return h end +-- Each Lua build gets its own default cache FILE, not just its own key: +-- bytecode is only portable within the exact build, and two hosts sharing one +-- path — your `luajit` and an embedded LuaJIT with a different jit.version +-- (OpenResty's, Envoy's) — would see a key mismatch on every alternation and +-- invalidate + rewrite each other's cache, recompiling the kernel every time. +-- The filename suffix is a hash of the same version/arch fingerprint that +-- cache_key() folds into the content key. local function cache_path() if not bit then return nil end -- PUC Lua: no `bit` -> no cache keys local p = os.getenv("SHEN_KERNEL_CACHE") if p == "off" or p == "0" then return nil end if p and p ~= "" then return p end - return ".shen-kernel-cache.bin" + return ".shen-kernel-cache." + .. bit.tohex(fnv1a(jit and (jit.version .. jit.arch) or _VERSION)) + .. ".bin" end local function read_file(path) @@ -316,6 +354,13 @@ local function write_cache(path, key, chunks, arity) fh:write(table.concat(parts)); fh:close() os.remove(path) os.rename(tmp, path) + -- Writing a per-build default cache obsoletes the old shared-path file from + -- pre-per-build versions (it would sit stale forever otherwise). Only the + -- default path triggers this — an explicit SHEN_KERNEL_CACHE never does. + if path:match("^%.shen%-kernel%-cache%.%x+%.bin$") then + os.remove(".shen-kernel-cache.bin") + os.remove(".shen-kernel-cache.bin.tmp") + end end -- Parse a write_cache blob. key == nil skips the key check (used for the diff --git a/examples/envoy/README.md b/examples/envoy/README.md index efd674c..1f47645 100644 --- a/examples/envoy/README.md +++ b/examples/envoy/README.md @@ -77,11 +77,9 @@ openresty -p "$PWD/examples/openresty" -c nginx.conf mkdir -p examples/envoy/logs openresty -p "$PWD/examples/envoy" -c authz.conf -# 3. Envoy at the edge, :10000 (run from the repo root — filter.lua resolves -# against the cwd; the two env vars are explained under "The mechanics -# worth knowing", and SHEN_JIT=off matters on macOS) -SHEN_KERNEL_CACHE=examples/envoy/logs/kernel-cache.bin SHEN_JIT=off \ - envoy -c examples/envoy/envoy.yaml --concurrency 2 +# 3. Envoy at the edge, :10000 (run from the repo root — filter.lua and the +# bytecode cache resolve against the cwd) +envoy -c examples/envoy/envoy.yaml --concurrency 2 ``` Then drive the edge: @@ -156,21 +154,29 @@ request path. of `init_worker_by_lua`: it runs once per worker thread at config load — never per request. `--concurrency 2` keeps the demo's boot cost and memory footprint small. -- **Two environment variables worth setting** (both findings from running this - against Homebrew Envoy 1.39 on an arm64 Mac): - - `SHEN_KERNEL_CACHE=examples/envoy/logs/kernel-cache.bin` — the bytecode - cache is keyed to the exact LuaJIT build, and Envoy's LuaJIT is not your - local `luajit`: with the shared default path (`.shen-kernel-cache.bin` in - the cwd) the two hosts invalidate and rewrite each other's cache every - time you alternate. A dedicated path gives Envoy its own warm cache. - - `SHEN_JIT=off` — macOS's hardened runtime denies Envoy's binary executable - trace memory, so with the JIT nominally on, every hot loop attempts a - trace, hits `failed to allocate mcode memory`, and retries: measured, a - 20M-iteration loop ran ~550× slower than local `luajit`, and the kernel - boot took 40–66 s. `jit.off()` makes it a clean interpreter: **~3 s to - serving** (warm cache) and edge requests at **3–6 ms**. Boot is the only - JIT-hungry phase — per-request validation is tiny either way. Linux Envoy - builds can generally allocate mcode; try without it there first. +- **Two sharp edges this example surfaced — both now handled by `boot.lua` + automatically** (found by running against Homebrew Envoy 1.39 on an arm64 + Mac): + - **Per-build bytecode caches.** Bytecode is only portable within the exact + LuaJIT build, and Envoy's LuaJIT is not your local `luajit` — with the old + single shared cache file the two hosts invalidated and rewrote each + other's cache on every alternation, recompiling the kernel every time. + The default cache path is now per build + (`.shen-kernel-cache..bin`), so Envoy and your shell each keep + their own warm cache in the cwd. `SHEN_KERNEL_CACHE` still relocates or + disables it. + - **JIT-denied hosts run interpreted.** macOS's hardened runtime denies + Envoy's binary executable trace memory: `jit.status()` reports true, but + every trace attempt aborts with `failed to allocate mcode memory` and is + endlessly re-attempted — measured, a hot loop ran ~550× slower than local + `luajit` and kernel boots took 40–66 s. `boot.lua` now probes for exactly + this (compile one throwaway trace, watch for its stop event) and falls + back to the interpreter with a note on stderr: **~2 s to serving cold, + ~1 s warm**, edge requests at **1.5–3 ms**. Boot is the only JIT-hungry + phase — per-request validation is tiny either way. On hosts where the JIT + works (OpenResty's nginx, your shell's `luajit`, typical Linux Envoy + builds) the probe compiles one trace and changes nothing. `SHEN_JIT=on` + skips the probe; `SHEN_JIT=off` skips straight to the interpreter. - **The check endpoint fails closed, twice.** An unmapped path maps to resource `""` → owner tenant `""` → `denied "unknown resource"` (the app); and `failure_mode_allow: false` means an unreachable authz app is a 403, diff --git a/examples/envoy/envoy.yaml b/examples/envoy/envoy.yaml index 4f5619f..1cfe439 100644 --- a/examples/envoy/envoy.yaml +++ b/examples/envoy/envoy.yaml @@ -21,12 +21,10 @@ # 3. router — what survives both gates reaches the guestbook. # # --concurrency 2 keeps the demo light: each Envoy worker thread has its own -# Lua state, so each boots its own Shen kernel. Two environment variables are -# worth setting when launching Envoy — SHEN_KERNEL_CACHE (a dedicated -# bytecode-cache path, so Envoy's LuaJIT and your local luajit stop rewriting -# each other's cache) and, on macOS, SHEN_JIT=off (the hardened runtime denies -# LuaJIT trace memory; the interpreter boots in ~3 s and serves in single-digit -# ms). See README.md, "The mechanics worth knowing". +# Lua state, so each boots its own Shen kernel (~1-2 s from Envoy's own +# per-build bytecode cache; boot.lua auto-detects hosts that deny LuaJIT +# executable trace memory — like this one on macOS — and runs interpreted). +# See README.md, "The mechanics worth knowing". static_resources: listeners: diff --git a/examples/openresty/README.md b/examples/openresty/README.md index 5cc0a73..427fcbe 100644 --- a/examples/openresty/README.md +++ b/examples/openresty/README.md @@ -137,8 +137,9 @@ marshaling rules. (The browser does the same marshaling in JS — see - **Kernel state is per worker.** Globals don't cross workers — use `lua_shared_dict`, Redis, or a DB for shared state (storage here is a shared dict, so it's visible to every worker). -- **The bytecode cache** (`.shen-kernel-cache.bin`) is written in the worker's - cwd (the nginx prefix) on first boot; it's gitignored. +- **The bytecode cache** (`.shen-kernel-cache..bin`, one per exact + LuaJIT build) is written in the worker's cwd (the nginx prefix) on first + boot; it's gitignored. ## The front end: Shen in the browser, tree-shaken From a1d04ba88607008f4df1e6986e730dc77fb40f0f Mon Sep 17 00:00:00 2001 From: Reuben Brooks Date: Tue, 11 Aug 2026 10:02:39 -0500 Subject: [PATCH 4/4] boot: cite issue #55 in the mcode-probe comment Co-Authored-By: Claude Fable 5 --- boot.lua | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/boot.lua b/boot.lua index 7df3a42..d37a0ae 100644 --- a/boot.lua +++ b/boot.lua @@ -54,15 +54,15 @@ do "sizemcode=2048", "maxmcode=131072", "maxtrace=8000", "maxside=400") end -- Some hosts leave the JIT nominally ON but deny the process executable - -- trace memory: macOS hardened-runtime binaries embedding LuaJIT (e.g. - -- Envoy's Lua filter — no JIT entitlement) report jit.status() == true, - -- yet every trace attempt aborts with "failed to allocate mcode memory" - -- and is re-attempted on the next hot path. Measured in Envoy 1.39 on - -- an arm64 Mac: a hot loop ran ~550x slower than plain interpretation - -- and kernel boot took 40-66 s (vs ~3 s interpreted). Detect it - -- directly — compile one throwaway hot loop and watch for a trace - -- "stop" event — and fall back to the interpreter. SHEN_JIT=on skips - -- the probe (explicit opt-in for hosts you have verified). + -- trace memory (issue #55): macOS hardened-runtime binaries embedding + -- LuaJIT (e.g. Envoy's Lua filter — no JIT entitlement) report + -- jit.status() == true, yet every trace attempt aborts with "failed to + -- allocate mcode memory" and is re-attempted on the next hot path. + -- Measured in Envoy 1.39 on an arm64 Mac: a hot loop ran ~550x slower + -- than plain interpretation and kernel boot took 40-66 s (vs ~3 s + -- interpreted). Detect it directly — compile one throwaway hot loop and + -- watch for a trace "stop" event — and fall back to the interpreter. + -- SHEN_JIT=on skips the probe (explicit opt-in for verified hosts). if os.getenv("SHEN_JIT") ~= "on" and jit.status and jit.status() and jit.attach then local compiled = false