Skip to content

Make the golden-runner quiet mode discoverable + measured status for #46 - #56

Merged
pyrex41 merged 2 commits into
mainfrom
perf/hush-load-docs-46
Aug 11, 2026
Merged

Make the golden-runner quiet mode discoverable + measured status for #46#56
pyrex41 merged 2 commits into
mainfrom
perf/hush-load-docs-46

Conversation

@pyrex41

@pyrex41 pyrex41 commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Why

Issue #46 has three complaints. Re-measuring all three on main @ 12fab4b:

  1. Cold start ~1.5s — fixed by fasl: record shen.*lambdatable* deltas by name instead of serializing live lambdas #45/perf: urdr SHA workload series (fasl, hush-load, cons/fn/append, GC) #47. Warm trivial script is now 0.16 s (was 1.5 s).
  2. urdr SHA suites ~12× CL — improved to 1.6–3.5× by perf: urdr SHA workload series (fasl, hush-load, cons/fn/append, GC) #47; the residual is structural.
  3. -q unusable for golden runners — the fix shipped in perf: urdr SHA workload series (fasl, hush-load, cons/fn/append, GC) #47 as --hush-load, but it was documented only in source comments and bin/shen --help, had no regression test, and the README still showed -q as the only quiet mode. A runner author following the README lands straight back on the reported blocker (an empty run: on 41.2 *hush* gates pr itself).

This PR closes that last gap. No behaviour change to any existing mode.

Changes

  • README--hush-load in the launcher block, plus a Batch and golden-suite runners subsection: why -q structurally cannot work for this, a worked before/after, SHEN_HUSH_LOAD=1 for runners with fixed argv, and the cold/warm cache-independence guarantee.
  • shen.boot{hush_load = true} — the embedder spelling, alongside jit = false. Applied before the idempotency guard: it is a pure output-mode flag with no boot work behind it, so an embedder handed an already-booted kernel can still ask for it.
  • test/cli_spec.lua — 13 new assertions locking in all three modes (default / -q / --hush-load), that SHEN_HUSH_LOAD=1 is byte-identical to the flag, that --hush-load output is identical cold and on a warm fasl hit (so a golden file is cache-state independent), and the shen.boot{hush_load=true} path.
  • doc/PERF-URDR-RESULTS.md — an independent re-run of issue Performance: cold start ~1.5s for a trivial script (~100x shen-cl); urdr SHA suites ~12x CL; -q unusable for golden runners #46's own repro, and an honest account of where the residual boot cost lives.

Measured (arm64 macOS, LuaJIT 2.1.1774638290, shen-cl 8df94be, urdr ba85b09)

Interleaved CL/Lua pairs, min-of-N, shen-lua warm + --hush-load.

Startup — (output "hi~%")

Configuration wall
shen-lua warm (both caches) 0.16 s (median 0.18)
shen-lua first ever run (empty caches) 0.86 s
shen-lua SHEN_KERNEL_CACHE=off SHEN_FASL=off 0.71 s
shen-cl script 0.01 s

Warm boot CPU (min of 8): require 0.003 + load_kernel 0.051 + load_stdlib 0.100 = 0.154 s. The issue's initialise 1.77 s → 0.10 s.

load_stdlib is now the entire remaining cost, and it is 21 warm fasl hits (SHEN_FASL_DEBUG=1: 21/21 hit), not compilation. jit.p over it is flat — EQ 8%, append 6%, kdata_de 5%, is_cons / shen.assoc-> 5% each, fasl_read 3%. The cost is the replay rebuilding the environment, spread across the kernel's own list/assoc primitives; there is no hot spot left to cut. Closing the remaining ~16× to shen-cl means not rebuilding it at all (an image of the booted state — shen-cl's save-lisp-and-die equivalent), which is why this PR ships no boot micro-optimisation.

urdr suites (ALL PASS on both ports)

Suite shen-cl shen-lua ratio at issue open
shen/tests/prng 0.25 s 0.41 s 1.6× ~12×
shen/tests/search 0.80 s 2.52 s 3.1× ~12×
shen/tests/world 0.45 s 1.56 s 3.5×

Output modes, urdr prng

Invocation stdout lines
shen-cl script run-tests.shen 232
bin/shen run-tests.shen (default) 239
bin/shen --hush-load run-tests.shen 71
bin/shen -q run-tests.shen 0

--hush-load yields exactly the 71 lines the suite itself prints; shen-cl's 232 are those same 71 plus its own 161-line load echo.

Testing

  • luajit run-kernel-tests.lua134 passed / 0 failed, ok
  • make test877 pass / 0 fail across 18 specs (was 864 before the new cases)
  • luajit examples/openresty/selftest.lua — OK
  • luajit examples/openresty-authz/selftest.lua — OK (file + lmdb + cosocket)
  • luajit examples/envoy/selftest.lua — OK (ext_authz + Lua filter + upstream)
  • urdr prng / search / world under bin/shen --hush-load — ALL PASS

Addresses item 3 of #46; #46 stays open for the structural suite-compute residual.

🤖 Generated with Claude Code

Reuben Brooks and others added 2 commits August 11, 2026 12:37
`--hush-load` / `SHEN_HUSH_LOAD=1` — the mode issue #46 asked for (silence
`load`'s own echo, keep the program's `(output ...)`) — shipped in #47 but
was documented only in source comments and `bin/shen --help`, and had no
regression coverage. A runner author reading the README still found only
`-q`, which on the 41.2 kernel gates `pr` itself and so returns an empty
run: exactly the reported blocker.

- README: document `--hush-load` in the launcher block and add a
  "Batch and golden-suite runners" subsection explaining why `-q` cannot
  work here, with a worked before/after and the cache-independence
  guarantee.
- `shen.boot{hush_load = true}`: the embedder spelling of the same switch,
  alongside `jit = false`. Applied before the idempotency guard, since it
  is a pure output-mode flag with no boot work behind it — an embedder
  handed an already-booted kernel can still ask for it.
- `test/cli_spec.lua`: lock in all three modes (default / `-q` /
  `--hush-load`), the `SHEN_HUSH_LOAD=1` spelling being byte-identical to
  the flag, `--hush-load` output being identical cold and on a warm fasl
  hit, and the `shen.boot{hush_load=true}` path.
- `doc/PERF-URDR-RESULTS.md`: an independent re-measurement of issue #46's
  own repro at 12fab4b (startup, the three urdr suites vs shen-cl, and the
  per-mode stdout line counts), plus where the residual boot cost actually
  lives.

No behaviour change to existing modes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-only diff)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pyrex41
pyrex41 merged commit 67e2f43 into main Aug 11, 2026
2 checks passed
@pyrex41
pyrex41 deleted the perf/hush-load-docs-46 branch August 11, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant