Repository navigation
README: the measured conformance figure, an embedding section, and the browser build that does not build - #678
Merged
Merged
Conversation
…e browser build that does not build Every factual claim here was re-measured on 3e8e3cc (darwin/arm64) against MRI 4.0.5 on the same host. What changed, and what condemned it: - Conformance: 21,982 passing / floor 21,970 -> measured 22,488 / floor 22,475. Three independent runs (run.sh at two different per-file timeouts and job counts, plus a separate sweep tallying all four counters) each reported exactly 22,488. fail/error/skip are 1,090/538/477, so the share of examples that ran and passed is 93.3%, not 91.4%. The old section also quoted 91.4% in its table and 91.1% in its prose. - Removed the "no line map, __LINE__ is always 0" section, which the README called its single widest gap. It no longer reproduces: rbgo and MRI now agree exactly on __LINE__, on backtrace line numbers, on warn(uplevel:) prefixes and on const_source_location. - Removed the intermittent autoload_spec.rb crash. Five consecutive runs of that file gave identical results and three full corpus sweeps never saw it fail, so the run-to-run spread the README told readers to expect is not reproducible here. - Added: GOOS=js GOARCH=wasm does not compile. internal/vm/errno.go references three syscall constants the js port lacks. It built at 68cb53a, the commit before the Errno subsystem landed, and no CI lane catches it because only wasip1 is gated. The README had called the browser a first-class/supported target. - Added: $stderr is not a separate stream. The VM takes one writer and wires $stderr/STDERR to it, so warn and $stderr.puts arrive on stdout; only the CLI's uncaught-exception printer uses real stderr. - Added an "Embedding in a Go program" section. There was none, in the repository whose org is called go-embedded-ruby. The public surface is one function, ruby.Run(src, out); the documented contract (merged streams, fresh VM per call, exceptions as Go errors, require_relative resolving against the process working directory) was established by compiling and running a probe against it. - Added, each verified against MRI: RUBY_VERSION reports 3.4.1; magic encoding comments are not honoured; pp does not exist. Errno's "32 diverging constants" survived checking - by set difference, 81 names are zero-valued under rbgo and 49 under MRI, with every MRI zero also zero under rbgo. - Platforms now follow ci.yml: the four exotic arches are nightly, not per-PR. - Quick start commands all executed. Noted that rbgo build must run from a checkout, which it does not say today and which fails confusingly elsewhere. FLOOR is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the README in line with what the code does today. Every factual claim was re-measured on
3e8e3cc(darwin/arm64) against MRI 4.0.5 on the same host; anything that could not be reproduced was removed rather than reworded.FLOORis untouched, and no VM source is changed.Claims corrected, with the measurement
Conformance: 21,982 passing / floor 21,970 → 22,488 / floor 22,475. Three independent runs agreed exactly on 22,488 —
run.shtwice (25 s and 60 s per-file timeouts, different parallelism) and once through a separate sweep that tallies all four counters. The full breakdown is 1,090 fail / 538 error / 477 skip, so the share of examples that ran and passed is 93.3 %, not the 91.4 % stated. (The old section also disagreed with itself: 91.4 % in the table, 91.1 % in the prose.)Removed: "rbgo carries no line map, so
__LINE__is always0" — which the README called "the single widest gap". It does not reproduce. rbgo and MRI now agree exactly:__LINE__, backtrace line numbers,warn(uplevel:)prefixes andconst_source_locationall match MRI.Removed: the intermittent
autoload_spec.rbcrash and the run-to-run spread built on it. Five consecutive runs of that file gave identical counts (pass=66 fail=11 error=2), and three full corpus sweeps never saw it fail. The README told readers to run the suite repeatedly and quote the low run; that advice was not supported by anything measurable here.Added:
GOOS=js GOARCH=wasmdoes not compile. The README called the browser a first-class, supported target.errno.gocarries no build tags, so it compiles for every target, and thejsport lacks those three names. It is a regression, not a standing gap: the same build succeeds at68cb53a, the commit before the Errno subsystem landed, producing a 95 MB module. Two things let it through —ci.ymlgates onlywasip1, andTestClosedWasmBuildIntegration, which does catch it, sits behindRBGO_BUILD_IT=1, which no workflow sets. Run with that variable set, it fails with exactly the error above. WASI is fine:wazero run rbgo.wasm -e 'puts (1..10).sum'prints55.Added:
$stderris not a separate stream.vm.Newtakes one writer and wires$stderr/STDERRto it, sowarnand$stderr.putsarrive on stdout; only the CLI's uncaught-exception printer uses real stderr.rbgo -e '$stdout.print "O"; $stderr.print "E"' >out 2>errleaveserrempty.Added an "Embedding in a Go program" section. There was none, in the repository whose org is
go-embedded-ruby. The contract documented was established by compiling and running a probe against the real API, not read off the source: one writer carrying both streams, a Ruby exception arriving as a Goerror, a fresh VM per call, andrequire_relativeresolving against the process working directory (proven by running the same binary from two directories — it succeeds in one and fails withLoadError: cannot load such file -- /tmp/libin the other).Added, each A/B'd against MRI:
RUBY_VERSIONreports"3.4.1"; magic encoding comments are not honoured (# encoding: ascii-8bitstill reportsUTF-8where MRI reportsASCII-8BIT);ppdoes not exist in any form, thoughrequire "prettyprint"works.Survived checking: the
Errnoclaim. By set difference on darwin, 81 constant names are zero-valued under rbgo and 49 under MRI, every MRI zero is also zero under rbgo, and the difference is exactly the 32 the README claimed. The178 modules across 176 orgscount reproduces under the README's own stated command.File::Stat#==,File#staton an unlinked-open file,Numeric#to_int,Process.forkandBEGIN/ENDall still reproduce.Other changes
ci.yml: linux/macOS/windows under-racewith the coverage gate on the POSIX lanes only, amd64+arm64 native,wasip1built and run under wazero — and the four exotic arches marked as nightly, not per-PR, which the old text buried.rbgo buildmust run from a checkout (it shells out togo buildagainst this module); elsewhere it fails withstat .../cmd/rbgo: directory not found, which the README did not say. Added a verifiedgo run ...@latestone-command path, and noted there are no version tags.Flagged for you
BENCHMARKS.mdline 98 claims rbgo+AOT is 18–24× faster than MRI+YJIT, and that does not follow from its own table (which gives AOT 10 ms vs YJIT 360 ms onloopand 20 ms vs 100 ms onfib— 36× and 5×). Re-running this repo's ownbench/loop.rbandbench/fib.rbhere: 8.7× and 2.0×. My MRI and MRI+YJIT timings match the published table closely (495/110 ms against its 480–500/100–110 ms), so the harnesses are comparable. I corrected the figure on the landing page but leftBENCHMARKS.mdalone — it is your measurement record, on your host. Your call.🤖 Generated with Claude Code