From 662745eb597549d5657069d0296f8e560cbe7f47 Mon Sep 17 00:00:00 2001 From: InauguralPhysicist Date: Wed, 28 Oct 2026 13:35:03 -0500 Subject: [PATCH] Update README measured status block (#60) Built by a Codex Cloud task: https://chatgpt.com/codex/tasks/task_e_6ae23cbc3644832d91b95ec72ce1cac1 --- LEDGER.md | 27 +++++++++---------- README.md | 9 ++++--- changes/changed/60-refresh-measured-status.md | 1 + 3 files changed, 20 insertions(+), 17 deletions(-) create mode 100644 changes/changed/60-refresh-measured-status.md diff --git a/LEDGER.md b/LEDGER.md index bb2b075..1ee5f23 100644 --- a/LEDGER.md +++ b/LEDGER.md @@ -1,12 +1,14 @@ -# EigenLua front end: divergence ledger (round 13) +# EigenLua front end: divergence ledger (round 15) Every row of `tools/grade.sh --candidate bin/eigenlua --tier all` that is not PASS/MATCH, classified. The detailed -failure tables below preserve the round-6 measurements; EigenScript v0.45.1 closes those upstream gaps. +failure tables below preserve the round-6 measurements; the current measurement uses EigenScript v0.47.0. -- **Runtime:** EigenScript v0.45.1 (round 7); v0.45.0 was used in round 6 (rounds 3-5 used `main` at 686d3b7d, whose `--version` printed - `0.44.0`). v0.45.0 has a heap use-after-free (EigenScript#1661) on an under-arity call of a 2+ parameter function - whose missing parameter is read about 1000 calls deep; the emitted code makes no under-arity call of such a function. +- **Runtime:** EigenScript v0.47.0 at commit `3d8a14b49e88a59a37e1d3501891afe2d430ffcb` (round 15). - **Reference:** PUC-Rio Lua 5.4.9 from `tools/build_ref.sh`. +- **Grader verdict, round 15 (Tier D, measured 2026-10-28):** + `SUMMARY S=0/0 D=145/157 fail=0 diverge=12 hang=0 crash=0 unsupported=0 examined=157 assert=n/a`. +- **Grader verdict, round 15 (Tier S, measured 2026-10-28):** + `SUMMARY S=2/23 D=0/0 fail=17 diverge=0 hang=1 crash=0 unsupported=3 examined=23 assert=trusted`. - **Grader verdict, end of round 10 (Tier D):** `SUMMARY S=0/0 D=113/125 fail=0 diverge=12 hang=0 crash=0 unsupported=0 examined=125 assert=n/a`. - **Grader verdict, end of round 11 (Tier D):** @@ -17,17 +19,14 @@ failure tables below preserve the round-6 measurements; EigenScript v0.45.1 clos `SUMMARY S=0/0 D=125/137 fail=0 diverge=12 hang=0 crash=0 unsupported=0 examined=137 assert=n/a`. - **Before round 7** (same checkout before EigenLua#35's changes, on v0.45.1): `SUMMARY S=23/23 D=108/108 fail=0 diverge=0 hang=0 crash=0 unsupported=0 examined=131 assert=trusted`. -- **Before this round** (round 5, 79e29669, re-measured on v0.45.0): `SUMMARY S=2/23 D=87/99 fail=15 diverge=12 +- **Before round 6** (round 5, 79e29669, re-measured on v0.45.0): `SUMMARY S=2/23 D=87/99 fail=15 diverge=12 hang=2 crash=0 unsupported=4 examined=122 assert=trusted`. Round 5's own line (on 686d3b7d) had fail=16 hang=1: cstack.lua is a HANG on v0.45.0 (below), with its first failure unchanged. -Tier D has 136 rows. Round 3 added 23 programs, round 4 added 8, round 5 added 16, round 6 added 9 (7 failing at -79e29669 for the bug they fix, 1 failing there only on a probe-found bug, 1 coverage row; section at the end). The 12 rows that do not match are the same 12 as in round 2, -all EigenScript gaps on v0.45.0. Round 7 added four programs for EigenLua#35. Round 8 added nine edge-case programs and fixed three -implementation defects. Round 9 adds two close-time debug rows and fixes the remaining coroutine-local cutoff defect. -Round 10 adds two deferred-close stack-restoration rows, round 11 adds nine table-traversal rows, and round 12 adds two more. -The 12 current divergences are the longstanding EigenScript -gaps listed below. +Tier D has 157 rows: 145 match and the 12 current divergences are the longstanding EigenScript gaps listed below. +Rounds 3 through 13 and their additions are described in the historical material below; rounds 14 and 15 added +20 matching rows without changing the divergence classes. Tier S has 23 rows. Across both tiers, the first-failure +classes remain 27 EigenScript gaps, 6 out-of-scope rows, no open front-end bugs, and no unspecified rows. ## Classes @@ -48,7 +47,7 @@ A row's class is the class of its **first failure**. Its first ten failures were | out of scope | 6 | 6 | 0 | | front-end bug (open) | 0 | 0 | 0 | | unspecified | 0 | 0 | 0 | -| PASS / MATCH | 98 | 2 | 96 | +| PASS / MATCH | 147 | 2 | 145 | **Scope of the zero in the "front-end bug (open)" row: graded rows only.** No graded row's first failure is a known front-end bug. That says nothing about code the corpus does not reach, and probing beyond it keeps finding diff --git a/README.md b/README.md index 966e737..efc82e4 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ inherits EigenScript's observer, trace tape and sandbox. It is graded against reference Lua 5.4.9 and the official Lua test suite. The bar and the gap ledger are in [GAPS.md](GAPS.md). -Status: front end round 9. `bin/eigenlua prog.lua` compiles with `src/compile.eigs` (written in EigenScript) +Status: front end round 15. `bin/eigenlua prog.lua` compiles with `src/compile.eigs` (written in EigenScript) and runs the result on the EigenScript VM: `$EIGENSCRIPT`, else `eigenscript` on `PATH`. It needs the `bool` type, which arrived in EigenScript v0.45.0 (EigenScript#1647). @@ -23,8 +23,11 @@ The runtime library has these parts: `goto`, ``, `load`/`require`, `_ENV` and the in-scope `debug` functions are compiled. -Measured 2026-10-08 (round 10, v0.45.1) with `tools/grade.sh --candidate bin/eigenlua --tier D`: -`SUMMARY S=0/0 D=113/125 fail=0 diverge=12 hang=0 crash=0 unsupported=0 examined=125 assert=n/a`. +Measured 2026-10-28 (round 15) on EigenScript v0.47.0 at commit +`3d8a14b49e88a59a37e1d3501891afe2d430ffcb` with `tools/grade.sh --candidate bin/eigenlua --tier D`: +`SUMMARY S=0/0 D=145/157 fail=0 diverge=12 hang=0 crash=0 unsupported=0 examined=157 assert=n/a`. +The Tier S run with `tools/grade.sh --candidate bin/eigenlua --tier S` produced: +`SUMMARY S=2/23 D=0/0 fail=17 diverge=0 hang=1 crash=0 unsupported=3 examined=23 assert=trusted`. Every non-passing row is classified in [LEDGER.md](LEDGER.md) by its first failure: - 27 are EigenScript gaps, each with a reproducer in `gaps/` (cstack.lua is now a HANG on v0.45.0: it runs past its time budget before its unchanged first failure); diff --git a/changes/changed/60-refresh-measured-status.md b/changes/changed/60-refresh-measured-status.md new file mode 100644 index 0000000..2d650f6 --- /dev/null +++ b/changes/changed/60-refresh-measured-status.md @@ -0,0 +1 @@ +- The README measured status now reports the current pinned EigenScript grade results (#60).