Skip to content

README: the divergence section named its oracle twice and the two disagreed - #953

Merged
tannevaled merged 1 commit into
mainfrom
readme-divergence-oracle
Nov 16, 2026
Merged

tannevaled merged 1 commit into
mainfrom
readme-divergence-oracle

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

The section named its oracle twice and the two disagreed

### What does not work yet opens by saying what every entry under it was measured against:

Every entry below was re-checked on fc8ca1a (darwin/arm64, 2026-10-08) by running it against MRI 4.0.7 on the same host.

and forty-six lines later its table said:

| | how it differs from MRI 4.0.5 |

4.0.5 was an earlier campaign's oracle. The section was re-measured against 4.0.7 and the opening sentence was updated; the header was not. rbgo itself now reports "4.0.9", so the header named a version matching neither the measurement nor the runtime.

⛔ It was not harmless. Two rows added under that header yesterday (#948) inherited the wrong oracle, and I repeated "MRI 4.0.5" from it in that PR's own text without noticing. A header is read as the column's definition.

Why no existing gate saw it

TestReadmeRubyVersionMatchesTheRuntime is thorough about the version the runtime reports — both directions, with a documented exemption for historical lines. This header is a different claim: the version rbgo was compared with. Nothing read it.

A document can hold two version claims that are each individually plausible and still contradict each other. The only thing that catches that is reading both and comparing them — and since both facts are in the file, the gate needs no runtime and no network:

TestTheDivergenceTableNamesTheSectionsOwnOracle finds the section, takes the version from its re-checked … against **MRI x.y.z sentence and from each differs from MRI x.y.z table header, and requires them equal. A table header gets no historical exemption, because the rows under it are claims about the code as it is now.

⛔⛔ The first version of that gate found nothing, and said so

The sentence is wrapped at 80 columns: re-checked is on one line, against **MRI 4.0.7 on the next. A line-by-line match found neither side.

It failed only because "neither side found" is a failure in this test rather than nothing to report:

README.md:823..885: no "re-checked … against **MRI x.y.z" sentence naming the MRI
version the entries were measured against. The section's oracle is the thing the
table is compared to; without it this test cannot judge anything.

Written the ordinary way — an absence treated as nothing to report — it would have gone green on the very defect it exists for. The match is over the joined section text now, with the line number computed from the byte offset.

Also

The row citing #799 now says that issue is closed (2026-11-02). It is cited for "the list of what each of the 27 needs": the move is done, the 27 are not, and a reader following a closed link should not have to guess which. A closed issue is not finished work.

The section's other 4.0.5 mentions are left alone on purpose — README.md:156-159 states that rows are "left as they were measured rather than rewritten", and those name other campaigns' oracles, not this table's.

Checked, not assumed

  • The gate was run before the fix and named the defect at README.md:872 against README.md:826; after the fix it passes and logs section README.md:823..885 measured against MRI 4.0.7; 1 table header(s) checked.
  • Both existing README gates still pass with their denominators stated: 2 present-tense version claims against 4.0.9 plus 2 exempt lines named, and 7 cross-references with 0 dangling.
  • gofmt -l . prints nothing; buildscan --dir . --vet → "go vet passed under all 19"; go test . green.
  • Docs site needs nothing, and that was verified rather than assumed. content/en/conformance.md:358 already says "re-run on 4872e01 against MRI 4.0.7 on the same host", and its divergence table's header is | | | — no version claim, so no contradiction to fix. Its other 4.0.5 mentions are benchmark and campaign oracles, the same deliberate historical kind as the README's.
  • Landing: no MRI-version claim about this table at all.

Dispositions

Refs #799, #799, #948

Both stay as they are. #799 is already closed and this only makes the README say so; #948 is merged and is named because its two rows are what inherited the wrong header.

🤖 Generated with Claude Code

…agreed

"### What does not work yet" opens by saying what every entry under it was
measured against:

    Every entry below was re-checked on `fc8ca1a` (darwin/arm64, 2026-10-08)
    by running it against **MRI 4.0.7 on the same host**.

and forty-six lines later its table said:

    | | how it differs from MRI 4.0.5 |

4.0.5 was an EARLIER campaign's oracle. The section was re-measured against
4.0.7 and the sentence was updated; the header was not. rbgo itself now
reports 4.0.9, so the header named a version matching neither the
measurement nor the runtime.

⛔ And it was not harmless. Two rows added under that header yesterday
(#948) inherited the wrong oracle, and I repeated "MRI 4.0.5" from it in
that PR's text without noticing. A header is read as the column's
definition.

⛔ WHY NO GATE SAW IT. TestReadmeRubyVersionMatchesTheRuntime checks the
version the runtime REPORTS, against kernel_introspect.go, in both
directions, with an exemption for historical lines. This header is a
DIFFERENT claim — the version rbgo was COMPARED WITH — and nothing read
it. A document can hold two version claims that are each individually
plausible and still contradict each other; only reading both and
comparing them catches that, and both facts are in the file, so the gate
needs no runtime and no network.

TestTheDivergenceTableNamesTheSectionsOwnOracle does that. It finds the
section, takes the version from its "re-checked … against **MRI x.y.z"
sentence and the version from each "differs from MRI x.y.z" table header,
and requires them equal. A table header is not prose and gets no
historical exemption: the rows under it are claims about the code as it
is now.

⛔⛔ THE FIRST VERSION OF THAT GATE READ LINE BY LINE AND FOUND NOTHING.
The sentence is wrapped at 80 columns, so "re-checked" is on one line and
"against **MRI 4.0.7" on the next. It reported the absence and failed
ONLY because "neither side found" is a failure in this test rather than
nothing to report. Written the ordinary way it would have gone green on
the very defect it exists for. The match is over the joined section text
now, with the line number computed from the offset.

Also: the row citing #799 now says that issue is CLOSED (2026-11-02). It
is cited for "the list of what each of the 27 needs" — the move is done,
the 27 are not, and a reader following a closed link should not expect
otherwise. A closed issue is not finished work.

The section's other 4.0.5 mentions are left alone on purpose: README.md
lines 156-159 state that rows are "left as they were measured rather than
rewritten", and those are other campaigns' oracles, not this table's.

Measured:
  - the gate was run BEFORE the fix and named the defect at README.md:872
    against README.md:826; it passes after, logging "section
    README.md:823..885 measured against MRI 4.0.7; 1 table header(s)
    checked";
  - the two existing README gates still pass, with their denominators: 2
    present-tense version claims against 4.0.9 plus 2 exempt, and 7
    cross-references, 0 dangling;
  - gofmt clean; buildscan --vet: "go vet passed under all 19".

Refs #799, #948

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit d9c3fe8 into main Nov 16, 2026
26 checks passed
@tannevaled
tannevaled deleted the readme-divergence-oracle branch November 16, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant