Repository navigation
docs: four published claims the last twelve pull requests falsified - #938
Merged
Merged
Conversation
Every figure here was already measured and merged; none of it needed a new run. What it needed was someone to ask which published sentences the merges had made false, which is a different question from "what did we fix". ## The four 1. `library/socket` **8 killed -> 7**, and the headline **75 -> 9** is **75 -> 8**. #935 measured it in the sentence "nine files killed by the clock across `library/socket` + `library/timeout` before, eight after". 2. A withdrawn figure. The README said the remaining hangs were "275 passing against MRI's 371". That control scored **7** on those 190 files, because `library/socket/spec_helper.rb` calls `MSpec.enable_feature` and the shim defined no `MSpec` (#936). A number counted against a broken control is not a smaller number, it is not a number; it is removed rather than adjusted. 3. The corpus map's "the rest is concentrated: matrix 233, bigdecimal 180, ..." list was compiled while `library/socket` contributed a flat **zero on both sides**. Repaired, socket is MRI 1790 / rbgo 224 -- about **-1560**, larger than every module on that list put together (#937). The two totals are deliberately **not summed**: different runs, different MRI patch releases (4.0.7 and 4.0.6), and a sum across two measurements is how two errors cancel. The published gap is marked a **floor** instead. 4. `docs/SHARED-ENGINES.md` still concluded, at the top of its `net-ftp` section, that **"#916 is not closed, because the measurement did not move ... still 39 killed"**, and told the reader `net-ftp` was not available for extraction. That was true on 2026-11-13 and is the opposite of the outcome: #927 released the GVL on the **data** connection and #928 added `Net::FTP.default_passive`, and net-ftp is **0 killed, -128**. The section now records why the inference was wrong -- "the fix did not move the measurement" was read as "the fix is incomplete" when the mechanism was wired at only some of its doors -- and lifts the extraction blocker while naming the -128 the new home inherits. ## Why a doc gate did not catch any of them Seven README drift gates run in this repository's root package and all seven are green both before and after this change: they check the conformance total against `BASELINE`, the Ruby version against the runtime, the Go floor, the numeric surface, the wasm lane, and the shim line numbers the prose cites. Not one of them looks at a figure in a dated prose table, because there is nothing to compare such a figure against -- the run that produced it is gone. That is the honest limit, and it is stated in the README rather than papered over: the dated map is explicitly "not a gate", and the only number CI enforces is per-file. The gates caught the thing they can catch -- when a 22-line insertion moved two cited line numbers, every lane went red. Refs #916, #933, #935, #936, #937 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Nov 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
docs: four published claims the last twelve pull requests falsified
Every figure here was already measured and merged. None of it needed a new run —
what it needed was someone to ask which published sentences the merges had made
false, which is a different question from "what did we fix", and nobody asks it
on the way out of a merge.
The four
1.
library/socket8 killed → 7, and the headline75 → 9is75 → 8.#935 measured it in so many words: "nine files killed by the clock across
library/socket+library/timeoutbefore, eight after". The README kept theold row for a day.
2. A figure withdrawn rather than adjusted. The README reported the remaining
hangs as "275 passing against MRI's 371". That control scored 7 on those
190 files, because
library/socket/spec_helper.rbcallsMSpec.enable_featureand the conformance shim defined no
MSpec(#936). A number counted against abroken control is not a smaller number — it is not a number. It is removed, with
the reason in its place.
3. The corpus map was compiled while its largest module read zero. The
"the rest is concentrated: matrix 233, bigdecimal 180, stringscanner 154…" list
was written while
library/socketcontributed a flat zero on both sides.Repaired, socket is MRI 1790 / rbgo 224 — about −1560, larger than every
module on that list put together (#937), and it is unimplemented surface rather
than hangs.
⛔ The two totals are deliberately not summed. They come from different runs
against different MRI patch releases (4.0.7 and 4.0.6), and a sum across two
measurements is how two errors cancel. The published gap is marked a floor
instead, which is a weaker claim that is actually true.
4.
docs/SHARED-ENGINES.mdtold readers the opposite of the outcome. Itsnet-ftpsection still concluded "#916 is not closed, because the measurementdid not move … still 39 killed", and that
net-ftpwas not available forextraction. True on 2026-11-13; the inverse of what happened. #927 released the
GVL on the data connection —
dialData,acceptData,sendData,retrlines,retrbinary— and #928 addedNet::FTP.default_passive, which waswhat had kept ruby/spec's active-mode fixture from ever reaching that path.
Re-measured: 0 killed, −128.
That section now also records why the inference was wrong, because the wrong
inference is the reusable part: "the fix did not move the measurement" was read
as "the fix is incomplete", when the mechanism was simply wired at only some of
its doors. The extraction blocker is lifted, and the −128 the new home inherits
is named — the same rule this document already applies to
digestandsecurerandom.Why no gate caught any of this
Seven README drift gates run in this repository's root package, and all seven are
green before and after this change: the conformance total against
BASELINE,the Ruby version against the runtime, the Go floor, the numeric surface, the wasm
lane, and the shim line numbers the prose cites.
Not one of them looks at a figure in a dated prose table, and that is not an
oversight — there is nothing to compare such a figure against, because the run
that produced it is gone. The honest response is the one already in the README:
the dated map says of itself that it is "one dated run, and not a gate", and the
only number CI enforces is per-file.
The gates did catch what they can catch. When a 22-line insertion moved two
README-cited
spec_helper.rbline numbers, every lane went red for a change thattouched one Ruby file.
Checks
gotest -run '<the seven README gates>' .— all seven pass, each alternativematched (
gotestfails and names any that match nothing).fixed here: one surface per pull request. Both are in flight separately —
layouts/_default/list.htmlon the landing states the inverted headline("the largest single debt is a hang rather than a missing feature"), and
content/en/conformance.mdon the docs site repeats the net-ftp figures.Refs #916, #933, #935, #936, #937, #927, #928
(#916 and #933 stay open: eight files still hang. #927, #928, #935 and #936 are
the merged pull requests whose outcomes these documents had not caught up with.
#937 stays open and is the one this change makes visible.)
🤖 Generated with Claude Code