Make the golden-runner quiet mode discoverable + measured status for #46 - #56
Merged
Conversation
`--hush-load` / `SHEN_HUSH_LOAD=1` — the mode issue #46 asked for (silence `load`'s own echo, keep the program's `(output ...)`) — shipped in #47 but was documented only in source comments and `bin/shen --help`, and had no regression coverage. A runner author reading the README still found only `-q`, which on the 41.2 kernel gates `pr` itself and so returns an empty run: exactly the reported blocker. - README: document `--hush-load` in the launcher block and add a "Batch and golden-suite runners" subsection explaining why `-q` cannot work here, with a worked before/after and the cache-independence guarantee. - `shen.boot{hush_load = true}`: the embedder spelling of the same switch, alongside `jit = false`. Applied before the idempotency guard, since it is a pure output-mode flag with no boot work behind it — an embedder handed an already-booted kernel can still ask for it. - `test/cli_spec.lua`: lock in all three modes (default / `-q` / `--hush-load`), the `SHEN_HUSH_LOAD=1` spelling being byte-identical to the flag, `--hush-load` output being identical cold and on a warm fasl hit, and the `shen.boot{hush_load=true}` path. - `doc/PERF-URDR-RESULTS.md`: an independent re-measurement of issue #46's own repro at 12fab4b (startup, the three urdr suites vs shen-cl, and the per-mode stdout line counts), plus where the residual boot cost actually lives. No behaviour change to existing modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-only diff) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Issue #46 has three complaints. Re-measuring all three on
main@12fab4b:-qunusable for golden runners — the fix shipped in perf: urdr SHA workload series (fasl, hush-load, cons/fn/append, GC) #47 as--hush-load, but it was documented only in source comments andbin/shen --help, had no regression test, and the README still showed-qas the only quiet mode. A runner author following the README lands straight back on the reported blocker (an empty run: on 41.2*hush*gatespritself).This PR closes that last gap. No behaviour change to any existing mode.
Changes
--hush-loadin the launcher block, plus a Batch and golden-suite runners subsection: why-qstructurally cannot work for this, a worked before/after,SHEN_HUSH_LOAD=1for runners with fixed argv, and the cold/warm cache-independence guarantee.shen.boot{hush_load = true}— the embedder spelling, alongsidejit = false. Applied before the idempotency guard: it is a pure output-mode flag with no boot work behind it, so an embedder handed an already-booted kernel can still ask for it.test/cli_spec.lua— 13 new assertions locking in all three modes (default /-q/--hush-load), thatSHEN_HUSH_LOAD=1is byte-identical to the flag, that--hush-loadoutput is identical cold and on a warm fasl hit (so a golden file is cache-state independent), and theshen.boot{hush_load=true}path.doc/PERF-URDR-RESULTS.md— an independent re-run of issue Performance: cold start ~1.5s for a trivial script (~100x shen-cl); urdr SHA suites ~12x CL; -q unusable for golden runners #46's own repro, and an honest account of where the residual boot cost lives.Measured (arm64 macOS, LuaJIT 2.1.1774638290, shen-cl
8df94be, urdrba85b09)Interleaved CL/Lua pairs, min-of-N, shen-lua warm +
--hush-load.Startup —
(output "hi~%")SHEN_KERNEL_CACHE=off SHEN_FASL=offscriptWarm boot CPU (min of 8):
require0.003 +load_kernel0.051 +load_stdlib0.100 = 0.154 s. The issue'sinitialise1.77 s → 0.10 s.load_stdlibis now the entire remaining cost, and it is 21 warm fasl hits (SHEN_FASL_DEBUG=1: 21/21 hit), not compilation.jit.pover it is flat —EQ8%,append6%,kdata_de5%,is_cons/shen.assoc->5% each,fasl_read3%. The cost is the replay rebuilding the environment, spread across the kernel's own list/assoc primitives; there is no hot spot left to cut. Closing the remaining ~16× to shen-cl means not rebuilding it at all (an image of the booted state — shen-cl'ssave-lisp-and-dieequivalent), which is why this PR ships no boot micro-optimisation.urdr suites (ALL PASS on both ports)
shen/tests/prngshen/tests/searchshen/tests/worldOutput modes, urdr prng
shen-cl script run-tests.shenbin/shen run-tests.shen(default)bin/shen --hush-load run-tests.shenbin/shen -q run-tests.shen--hush-loadyields exactly the 71 lines the suite itself prints; shen-cl's 232 are those same 71 plus its own 161-line load echo.Testing
luajit run-kernel-tests.lua— 134 passed / 0 failed, okmake test— 877 pass / 0 fail across 18 specs (was 864 before the new cases)luajit examples/openresty/selftest.lua— OKluajit examples/openresty-authz/selftest.lua— OK (file + lmdb + cosocket)luajit examples/envoy/selftest.lua— OK (ext_authz + Lua filter + upstream)prng/search/worldunderbin/shen --hush-load— ALL PASSAddresses item 3 of #46; #46 stays open for the structural suite-compute residual.
🤖 Generated with Claude Code