From 81679a6d6bf8b9c27d532a1dd095e5a571241ab7 Mon Sep 17 00:00:00 2001 From: tannevaled Date: Mon, 16 Nov 2026 02:13:57 +0100 Subject: [PATCH] README: the divergence section named its oracle twice and the two disagreed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "### What does not work yet" opens by saying what every entry under it was measured against: Every entry below was re-checked on `fc8ca1a` (darwin/arm64, 2026-10-08) by running it against **MRI 4.0.7 on the same host**. and forty-six lines later its table said: | | how it differs from MRI 4.0.5 | 4.0.5 was an EARLIER campaign's oracle. The section was re-measured against 4.0.7 and the sentence was updated; the header was not. rbgo itself now reports 4.0.9, so the header named a version matching neither the measurement nor the runtime. ⛔ And it was not harmless. Two rows added under that header yesterday (#948) inherited the wrong oracle, and I repeated "MRI 4.0.5" from it in that PR's text without noticing. A header is read as the column's definition. ⛔ WHY NO GATE SAW IT. TestReadmeRubyVersionMatchesTheRuntime checks the version the runtime REPORTS, against kernel_introspect.go, in both directions, with an exemption for historical lines. This header is a DIFFERENT claim — the version rbgo was COMPARED WITH — and nothing read it. A document can hold two version claims that are each individually plausible and still contradict each other; only reading both and comparing them catches that, and both facts are in the file, so the gate needs no runtime and no network. TestTheDivergenceTableNamesTheSectionsOwnOracle does that. It finds the section, takes the version from its "re-checked … against **MRI x.y.z" sentence and the version from each "differs from MRI x.y.z" table header, and requires them equal. A table header is not prose and gets no historical exemption: the rows under it are claims about the code as it is now. ⛔⛔ THE FIRST VERSION OF THAT GATE READ LINE BY LINE AND FOUND NOTHING. The sentence is wrapped at 80 columns, so "re-checked" is on one line and "against **MRI 4.0.7" on the next. It reported the absence and failed ONLY because "neither side found" is a failure in this test rather than nothing to report. Written the ordinary way it would have gone green on the very defect it exists for. The match is over the joined section text now, with the line number computed from the offset. Also: the row citing #799 now says that issue is CLOSED (2026-11-02). It is cited for "the list of what each of the 27 needs" — the move is done, the 27 are not, and a reader following a closed link should not expect otherwise. A closed issue is not finished work. The section's other 4.0.5 mentions are left alone on purpose: README.md lines 156-159 state that rows are "left as they were measured rather than rewritten", and those are other campaigns' oracles, not this table's. Measured: - the gate was run BEFORE the fix and named the defect at README.md:872 against README.md:826; it passes after, logging "section README.md:823..885 measured against MRI 4.0.7; 1 table header(s) checked"; - the two existing README gates still pass, with their denominators: 2 present-tense version claims against 4.0.9 plus 2 exempt, and 7 cross-references, 0 dangling; - gofmt clean; buildscan --vet: "go vet passed under all 19". Refs #799, #948 Co-Authored-By: Claude Opus 5 --- README.md | 4 +- readme_divergence_oracle_test.go | 134 +++++++++++++++++++++++++++++++ 2 files changed, 136 insertions(+), 2 deletions(-) create mode 100644 readme_divergence_oracle_test.go diff --git a/README.md b/README.md index d763dd7c..b25a893c 100644 --- a/README.md +++ b/README.md @@ -869,9 +869,9 @@ hang, so code relying on it for that is unprotected under rbgo — and unprotect Otherwise: -| | how it differs from MRI 4.0.5 | +| | how it differs from MRI 4.0.7 | | --- | --- | -| `RUBY_VERSION` | rbgo reports `"4.0.9"`, the latest MRI release (2026-10-23). The Ruby 4.0 **`Ruby` namespace** *is* implemented — all nine of `Ruby::{VERSION,RELEASE_DATE,PLATFORM,PATCHLEVEL,REVISION,COPYRIGHT,ENGINE,ENGINE_VERSION,DESCRIPTION}` are the **same objects** as their `RUBY_`-prefixed globals, as `version.c` defines them, so `Ruby::VERSION.equal?(RUBY_VERSION)` is `true`. The move from `3.4.1` was measured before it was made, with both engines built from one commit minus the version string: **+144 passing** over `core` and `language`, and **27 examples that went red**, which no pass total shows — see [#799](https://github.com/go-embedded-ruby/ruby/issues/799) for the list of what each of the 27 needs | +| `RUBY_VERSION` | rbgo reports `"4.0.9"`, the latest MRI release (2026-10-23). The Ruby 4.0 **`Ruby` namespace** *is* implemented — all nine of `Ruby::{VERSION,RELEASE_DATE,PLATFORM,PATCHLEVEL,REVISION,COPYRIGHT,ENGINE,ENGINE_VERSION,DESCRIPTION}` are the **same objects** as their `RUBY_`-prefixed globals, as `version.c` defines them, so `Ruby::VERSION.equal?(RUBY_VERSION)` is `true`. The move from `3.4.1` was measured before it was made, with both engines built from one commit minus the version string: **+144 passing** over `core` and `language`, and **27 examples that went red**, which no pass total shows — see [#799](https://github.com/go-embedded-ruby/ruby/issues/799), **closed 2026-11-02**, for the list of what each of the 27 needs: the move is done, the 27 are not, and a closed issue is not finished work | | `Numeric#to_int` | not defined — `Complex(2.9, 0).to_int` raises `NoMethodError`; MRI returns `2` | | `File::Stat#==` | `a == b` is `false` for two stats of the same file (MRI: `true`); `a.==(b)` answers `true` | | `File#stat` on a file unlinked while open | raises `Errno::ENOENT`; MRI answers from the open descriptor. rbgo's streams are buffered by path and hold no descriptor | diff --git a/readme_divergence_oracle_test.go b/readme_divergence_oracle_test.go new file mode 100644 index 00000000..1130eb15 --- /dev/null +++ b/readme_divergence_oracle_test.go @@ -0,0 +1,134 @@ +// Copyright (c) the go-embedded-ruby/ruby authors +// +// SPDX-License-Identifier: BSD-3-Clause + +package ruby_test + +import ( + "os" + "regexp" + "strings" + "testing" +) + +// ⛔ THE SECTION NAMED ITS ORACLE TWICE AND THE TWO DISAGREED. +// +// "### What does not work yet" opens by saying what every entry under it was +// measured against: +// +// Every entry below was re-checked on `fc8ca1a` (darwin/arm64, 2026-10-08) by +// running it against **MRI 4.0.7 on the same host**. +// +// and forty-six lines later its table said: +// +// | | how it differs from MRI 4.0.5 | +// +// 4.0.5 was the oracle of an EARLIER campaign. The section was re-measured +// against 4.0.7, the sentence was updated, the table header was not, and rbgo +// itself now reports 4.0.9 — so the header named a version that matched neither +// the measurement nor the runtime. Two rows added under it on 2026-11-15 (#948) +// inherited the wrong oracle, and its PR text repeated "MRI 4.0.5" from the +// header without anyone noticing. +// +// ⛔ WHY NO EXISTING GATE SAW IT. TestReadmeRubyVersionMatchesTheRuntime checks +// the version the runtime REPORTS, against kernel_introspect.go. This header is +// a different claim — the version rbgo was COMPARED WITH — and nothing read it. +// A document can hold two version claims that are each individually plausible +// and still contradict each other; the only thing that catches that is reading +// both and comparing them, which needs no runtime and no network because both +// facts are in the file. +// +// The exemptions in the version test do not apply here either: a table header is +// not prose and cannot be "historical", because the rows under it are claims +// about the code as it is now. +func TestTheDivergenceTableNamesTheSectionsOwnOracle(t *testing.T) { + readme, err := os.ReadFile("README.md") + if err != nil { + t.Fatalf("read README.md: %v", err) + } + lines := strings.Split(string(readme), "\n") + + const heading = "### What does not work yet" + start := -1 + for i, line := range lines { + if strings.TrimSpace(line) == heading { + start = i + break + } + } + if start < 0 { + t.Fatalf("README.md has no %q section: this test measured nothing, which is not a pass. "+ + "If the section was renamed, rename it here too rather than deleting the gate.", heading) + } + // The section ends at the next heading of the same or higher level. + end := len(lines) + for i := start + 1; i < len(lines); i++ { + if strings.HasPrefix(lines[i], "## ") || strings.HasPrefix(lines[i], "### ") { + end = i + break + } + } + section := lines[start:end] + + // Side one: what the section SAYS it was measured against. + // + // ⛔ MATCHED OVER THE JOINED TEXT, NOT LINE BY LINE. The sentence is wrapped + // at 80 columns, so "re-checked" sits on one line and "against **MRI 4.0.7" + // on the next. The first version of this test read line by line, found + // nothing, and said so — only because "neither side found" is a FAILURE here. + // Written the usual way, with an absence treated as nothing to report, it + // would have gone green on the very defect it was written for. + measured := regexp.MustCompile(`(?s)re-checked.{0,200}?against \*\*MRI (\d+\.\d+\.\d+)`) + // Side two: what the table header CLAIMS to differ from. Table rows are never + // wrapped, so this one is per line. + header := regexp.MustCompile(`^\|.*differs from MRI (\d+\.\d+\.\d+)`) + + var measuredAt, measuredLine string + var measuredAtLine int + joined := strings.Join(section, "\n") + if loc := measured.FindStringSubmatchIndex(joined); loc != nil { + measuredAt = joined[loc[2]:loc[3]] + // The line the VERSION sits on, which is the one a reader must edit. + measuredAtLine = start + strings.Count(joined[:loc[2]], "\n") + 1 + measuredLine = strings.TrimSpace(lines[measuredAtLine-1]) + } + + var headers []string + var headerLines []int + for i, line := range section { + if m := header.FindStringSubmatch(line); m != nil { + headers = append(headers, m[1]) + headerLines = append(headerLines, start+i+1) + } + } + + // ⛔ Either side missing means this gate read nothing. A regexp that matches + // nothing reports a clean document, which is the failure this whole file + // exists to prevent — so each absence is named, not skipped. + if measuredAt == "" { + t.Fatalf("README.md:%d..%d: no %q sentence naming the MRI version the entries were measured against. "+ + "The section's oracle is the thing the table is compared to; without it this test cannot judge anything.", + start+1, end, "re-checked … against **MRI x.y.z") + } + if len(headers) == 0 { + t.Fatalf("README.md:%d..%d: no table header of the form %q. "+ + "The section says it was measured against MRI %s and no table claims to differ from anything.", + start+1, end, "| | how it differs from MRI x.y.z |", measuredAt) + } + + for i, h := range headers { + if h != measuredAt { + t.Errorf(`README.md:%d says the table differs from MRI %s, but README.md:%d says every entry in this section was measured against MRI %s: + + %s + +A table header is not prose and cannot be historical: the rows under it are +claims about the code as it is NOW, so it must name the version they were +actually judged against. Change whichever of the two is wrong — and if the +section really was re-measured, the opening sentence is the one to update first, +because it carries the commit and the date.`, + headerLines[i], h, measuredAtLine, measuredAt, measuredLine) + } + } + t.Logf("section README.md:%d..%d measured against MRI %s; %d table header(s) checked", start+1, end, measuredAt, len(headers)) +}