Skip to content

README: the measured conformance figure, an embedding section, and the browser build that does not build - #678

Merged
tannevaled merged 1 commit into
mainfrom
wave34-docs-ruby
Sep 26, 2026
Merged

tannevaled merged 1 commit into
mainfrom
wave34-docs-ruby

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Brings the README in line with what the code does today. Every factual claim was re-measured on 3e8e3cc (darwin/arm64) against MRI 4.0.5 on the same host; anything that could not be reproduced was removed rather than reworded. FLOOR is untouched, and no VM source is changed.

Claims corrected, with the measurement

Conformance: 21,982 passing / floor 21,970 → 22,488 / floor 22,475. Three independent runs agreed exactly on 22,488 — run.sh twice (25 s and 60 s per-file timeouts, different parallelism) and once through a separate sweep that tallies all four counters. The full breakdown is 1,090 fail / 538 error / 477 skip, so the share of examples that ran and passed is 93.3 %, not the 91.4 % stated. (The old section also disagreed with itself: 91.4 % in the table, 91.1 % in the prose.)

Removed: "rbgo carries no line map, so __LINE__ is always 0" — which the README called "the single widest gap". It does not reproduce. rbgo and MRI now agree exactly:

$ rbgo probe.rb        # and: ruby probe.rb — identical output
LINE=1
BT=/tmp/probe.rb:2:in '<main>'
[]

__LINE__, backtrace line numbers, warn(uplevel:) prefixes and const_source_location all match MRI.

Removed: the intermittent autoload_spec.rb crash and the run-to-run spread built on it. Five consecutive runs of that file gave identical counts (pass=66 fail=11 error=2), and three full corpus sweeps never saw it fail. The README told readers to run the suite repeatedly and quote the low run; that advice was not supported by anything measurable here.

Added: GOOS=js GOARCH=wasm does not compile. The README called the browser a first-class, supported target.

$ CGO_ENABLED=0 GOOS=js GOARCH=wasm go build ./cmd/rbgo
internal/vm/errno.go:101:35: undefined: syscall.ENOTRECOVERABLE
internal/vm/errno.go:108:35: undefined: syscall.EOWNERDEAD
internal/vm/errno.go:120:35: undefined: syscall.ETXTBSY

errno.go carries no build tags, so it compiles for every target, and the js port lacks those three names. It is a regression, not a standing gap: the same build succeeds at 68cb53a, the commit before the Errno subsystem landed, producing a 95 MB module. Two things let it through — ci.yml gates only wasip1, and TestClosedWasmBuildIntegration, which does catch it, sits behind RBGO_BUILD_IT=1, which no workflow sets. Run with that variable set, it fails with exactly the error above. WASI is fine: wazero run rbgo.wasm -e 'puts (1..10).sum' prints 55.

Added: $stderr is not a separate stream. vm.New takes one writer and wires $stderr/STDERR to it, so warn and $stderr.puts arrive on stdout; only the CLI's uncaught-exception printer uses real stderr. rbgo -e '$stdout.print "O"; $stderr.print "E"' >out 2>err leaves err empty.

Added an "Embedding in a Go program" section. There was none, in the repository whose org is go-embedded-ruby. The contract documented was established by compiling and running a probe against the real API, not read off the source: one writer carrying both streams, a Ruby exception arriving as a Go error, a fresh VM per call, and require_relative resolving against the process working directory (proven by running the same binary from two directories — it succeeds in one and fails with LoadError: cannot load such file -- /tmp/lib in the other).

Added, each A/B'd against MRI: RUBY_VERSION reports "3.4.1"; magic encoding comments are not honoured (# encoding: ascii-8bit still reports UTF-8 where MRI reports ASCII-8BIT); pp does not exist in any form, though require "prettyprint" works.

Survived checking: the Errno claim. By set difference on darwin, 81 constant names are zero-valued under rbgo and 49 under MRI, every MRI zero is also zero under rbgo, and the difference is exactly the 32 the README claimed. The 178 modules across 176 orgs count reproduces under the README's own stated command. File::Stat#==, File#stat on an unlinked-open file, Numeric#to_int, Process.fork and BEGIN/END all still reproduce.

Other changes

  • Platforms now follows ci.yml: linux/macOS/windows under -race with the coverage gate on the POSIX lanes only, amd64+arm64 native, wasip1 built and run under wazero — and the four exotic arches marked as nightly, not per-PR, which the old text buried.
  • Every Quick start command was executed. rbgo build must run from a checkout (it shells out to go build against this module); elsewhere it fails with stat .../cmd/rbgo: directory not found, which the README did not say. Added a verified go run ...@latest one-command path, and noted there are no version tags.
  • The "Phase 6 + 8 in progress" badge is replaced by the conformance figure; the opening now states how complete this is, for a reader deciding whether to depend on it.

Flagged for you

BENCHMARKS.md line 98 claims rbgo+AOT is 18–24× faster than MRI+YJIT, and that does not follow from its own table (which gives AOT 10 ms vs YJIT 360 ms on loop and 20 ms vs 100 ms on fib — 36× and 5×). Re-running this repo's own bench/loop.rb and bench/fib.rb here: 8.7× and 2.0×. My MRI and MRI+YJIT timings match the published table closely (495/110 ms against its 480–500/100–110 ms), so the harnesses are comparable. I corrected the figure on the landing page but left BENCHMARKS.md alone — it is your measurement record, on your host. Your call.

🤖 Generated with Claude Code

…e browser build that does not build

Every factual claim here was re-measured on 3e8e3cc (darwin/arm64) against MRI
4.0.5 on the same host. What changed, and what condemned it:

- Conformance: 21,982 passing / floor 21,970 -> measured 22,488 / floor 22,475.
  Three independent runs (run.sh at two different per-file timeouts and job
  counts, plus a separate sweep tallying all four counters) each reported exactly
  22,488. fail/error/skip are 1,090/538/477, so the share of examples that ran and
  passed is 93.3%, not 91.4%. The old section also quoted 91.4% in its table and
  91.1% in its prose.

- Removed the "no line map, __LINE__ is always 0" section, which the README called
  its single widest gap. It no longer reproduces: rbgo and MRI now agree exactly on
  __LINE__, on backtrace line numbers, on warn(uplevel:) prefixes and on
  const_source_location.

- Removed the intermittent autoload_spec.rb crash. Five consecutive runs of that
  file gave identical results and three full corpus sweeps never saw it fail, so
  the run-to-run spread the README told readers to expect is not reproducible here.

- Added: GOOS=js GOARCH=wasm does not compile. internal/vm/errno.go references
  three syscall constants the js port lacks. It built at 68cb53a, the commit before
  the Errno subsystem landed, and no CI lane catches it because only wasip1 is
  gated. The README had called the browser a first-class/supported target.

- Added: $stderr is not a separate stream. The VM takes one writer and wires
  $stderr/STDERR to it, so warn and $stderr.puts arrive on stdout; only the CLI's
  uncaught-exception printer uses real stderr.

- Added an "Embedding in a Go program" section. There was none, in the repository
  whose org is called go-embedded-ruby. The public surface is one function,
  ruby.Run(src, out); the documented contract (merged streams, fresh VM per call,
  exceptions as Go errors, require_relative resolving against the process working
  directory) was established by compiling and running a probe against it.

- Added, each verified against MRI: RUBY_VERSION reports 3.4.1; magic encoding
  comments are not honoured; pp does not exist. Errno's "32 diverging constants"
  survived checking - by set difference, 81 names are zero-valued under rbgo and 49
  under MRI, with every MRI zero also zero under rbgo.

- Platforms now follow ci.yml: the four exotic arches are nightly, not per-PR.

- Quick start commands all executed. Noted that rbgo build must run from a checkout,
  which it does not say today and which fails confusingly elsewhere.

FLOOR is untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 2aa1c94 into main Sep 26, 2026
7 checks passed
@tannevaled
tannevaled deleted the wave34-docs-ruby branch September 26, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant