Repository navigation
socket: open with a block must close, which is 27 of the hangs in #916 - #932
Merged
Merged
Conversation
TCPSocket.open, TCPServer.open and Socket.open were bare aliases for .new, so the block form closed nothing: the socket outlived the block and the peer waited for an EOF that never came. ⛔ THAT IS THE WHOLE net-http COLUMN OF #916. ruby/spec's library/net-http/http/fixtures/http_server.rb serves ONE client at a time and shuts down by opening a NEW connection to write an unterminated "CLOSE": def close TCPSocket.open(ip, port) { |socket| socket.write "CLOSE" } @thread.join end With that socket still open the server's `gets("\r\n\r\n")` never returns, the listen loop never comes back to accept, and @thread.join never completes. All 27 library/net-http files end on that join. Measured against the restored MRI oracle, the same ten-line reproduction: closed? after the block MRI true rbgo false the peer's gets MRI "CLOSE" rbgo blocks for ever join MRI :joined rbgo hangs and after this change rbgo prints the same seven lines as MRI, in the same order, and exits 0. ⛔ IT IS NOT THE GVL, which is what I expected to find. A goroutine dump of a hung get_spec.rb puts the server thread correctly INSIDE ioBlock with the GVL released (readUntil -> bufio.ReadByte -> net.(*conn).Read) and the main thread in threadJoin. The emulated GVL behaves; the socket was simply still open. Nor does Net::HTTP leak its own connection: the reproduction shows "peer closed" after the request under BOTH engines. HOW IT CLOSES, and why there is no fourth spelling of close: it sends the CLASS'S OWN `close` method, so TCPSocket, TCPServer and Socket each keep the one close they already had -- each guarded by `s.closed`, hence idempotent, so a block that closed the socket itself is not closed twice. The close runs in a defer, so an exception or a break leaves the socket closed as well: this is an ensure, not a tidy-up on the happy path. The test asserts the contract on the socket the block was handed rather than through a peer, so a regression FAILS it instead of HANGING it: the block's value comes back (not the socket), closed? is true after a normal return and after an exception, and `open` with no block still answers with an open socket exactly as .new does. Refs #931, #916 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ix exposed
The ruby/spec ratchet refused the previous commit with +27 examples overall
and ONE file down:
passing examples: 23699 (baseline 23672, +27)
-- REGRESSED (1) --
core/io/nonblock_spec.rb: 3 passing, baseline 4 (-1)
⛔ The gate was right to refuse, and nothing had regressed. The example is
TCPServer.open(0) { |socket| socket.nonblock?.should == true }
and while `open` ignored its block that body NEVER RAN: the example passed with
no assertion evaluated, and the baseline recorded that pass. Honouring the
block made it run, and it raised NoMethodError — TCPServer#nonblock? does not
exist. A test that had never run started running.
Measured both ways rather than assumed, because the +27 includes other work
merged into main today and the -1 could have been someone else's:
core/io/nonblock_spec.rb, main ba89ba6: pass=4 fail=0 error=0
core/io/nonblock_spec.rb, this branch: pass=3 fail=0 error=1
NoMethodError undefined method 'nonblock?' for an instance of TCPServer
So it was mine.
IO#nonblock? is installed on IO alone and reads an *IOObj field, which a socket
is not. This installs `nonblock?` on the socket classes FROM THE SAME
`io/nonblock` feature hook, so one place decides what that require adds and the
two cannot drift; the socket classes exist from startup, so they are there when
it fires.
The answer is `true` because it is true here: Go puts every socket descriptor
in non-blocking mode and polls it through the runtime, which is also why MRI
reports true for a socket by default.
⛔ `nonblock=` is deliberately absent. The blocking mode of these descriptors
belongs to the Go runtime, and a writer that accepted `false` and changed
nothing would be a more expensive lie than a missing method. MRI does allow it;
the divergence is stated here rather than papered over.
The file is back to 4 passing — but the fourth example now RUNS and asserts,
where on main it passes with nothing evaluated. Same number, different meaning.
Refs #931
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The coverage ratchet named installSocketNonblock at 83.3%, newly below 100%.
The uncovered statement was the `continue`:
cls, ok := vm.consts[name].(*RClass)
if !ok {
continue // ← all six classes exist, so nothing reaches this
}
All six socket classes are registered at startup, so that arm is unreachable on
this build and no test can honestly cover it. Writing the guard as
if cls, ok := vm.consts[name].(*RClass); ok { … }
keeps exactly the same protection — a name that is not a class is skipped
rather than type-asserted into a panic — and has no statement on the false
path to explain away.
⛔ The alternative was a test that deletes a class from vm.consts to drive the
`continue`. That buys the percentage by asserting a state the VM never has, and
leaves the next reader wondering which of the two is the lie. Removing the dead
statement is cheaper and truer.
Measured before pushing, with covratchet -require (#925), which is what that
flag exists for after two commits in one day failed to move this same number:
required 2 function(s), matched 2 record(s) in a profile of 7351
all required functions at 100%
The ruby/spec ratchet is already green on this head — the nonblock commit
settled core/io/nonblock_spec.rb — so this is the last red.
Refs #931
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tannevaled
added a commit
that referenced
this pull request
Nov 14, 2026
#932 merged while this was in review, so the table's net-http row said 27 as the CURRENT figure for a module that no longer hangs at all. Publishing that would have been the same defect this pull request exists to correct. Re-measured on main at 6b51deb, 321 files across library/net-http + library/socket + library/timeout, MRI 4.0.6 as the control, 30 s per file: 275 passing against MRI's 371, NINE killed under rbgo and none under MRI. So the tally is 75 -> 9, and the nine are the last two rows: Thread#kill cannot interrupt a thread blocked in a native call (#933). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tannevaled
added a commit
that referenced
this pull request
Nov 14, 2026
#934) * README: the Net::FTP hang figures describe code that has since changed The map of where the work is said: The largest single debt is a hang, not a missing feature. Net::FTP alone is 442 examples behind, and 39 of its 56 spec files have to be killed by the clock on rbgo and none on MRI (#916). Re-measured today against MRI 4.0.6, same corpus, 30 s per file, on main at ba89ba6: Net::FTP is 128 behind (459 -> 331) and NO file is killed at all. Only the figures this measurement actually covers are touched; the whole-corpus numbers above that bullet are left exactly as they were, because I have not re-swept the corpus. The replacement states the thing that took a day to establish and that a single number hides: ONE SYMPTOM, TWO UNRELATED CAUSES. Half the files hung because a blocking socket call held the emulated GVL (#924); the other half because TCPSocket.open with a block never closed the socket, so the peer waited for an EOF that never came (#932, open). Neither fix moves the other's files, which is why the first one looked like it had done nothing when it was measured against the second one's specs. The table also carries what is NOT fixed, with its cause named rather than left as a count: library/socket 8 and library/timeout 1, both because Thread#kill cannot interrupt a thread blocked in a native call (#933). A reader should be able to tell the fixed from the merely counted. ⛔ And the note that outlives all of these numbers: none of it was measurable until the reference interpreter was repaired. It had been scoring a clean 0 on every file because its bundled gems were off the load path, and a sweep reports that as a RESULT -- 56 files of "control 0 passing" reads as the implementation under test beating the reference. cmd/mriref (#930) is named so the next person measuring starts from a working oracle. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * README: net-http is now 0 too, measured on the merged main #932 merged while this was in review, so the table's net-http row said 27 as the CURRENT figure for a module that no longer hangs at all. Publishing that would have been the same defect this pull request exists to correct. Re-measured on main at 6b51deb, 321 files across library/net-http + library/socket + library/timeout, MRI 4.0.6 as the control, 30 s per file: 275 passing against MRI's 371, NINE killed under rbgo and none under MRI. So the tally is 75 -> 9, and the nine are the last two rows: Thread#kill cannot interrupt a thread blocked in a native call (#933). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
tannevaled
added a commit
that referenced
this pull request
Nov 15, 2026
#932 gave `open` MRI's block semantics on TCPSocket, TCPServer and Socket. #931's "shape of the fix" named the rest of the family in the same breath — UNIXSocket, UDPSocket, UNIXServer — and nothing carried it out. Those three went on aliasing `open` straight to `new`, so a block handed to them was DISCARDED WITHOUT A WORD: no error, no warning, a body that simply never ran. Three of six doors wired is this week's recurring shape, and the reason it keeps recurring is that the wrapper is shared while the WIRING is per class. | class | was | now | | --- | --- | --- | | `UDPSocket` (socket_udp.go) | `native: newFn` | `socketOpenWithBlock(udp, newFn)` | | `UNIXSocket` (socket_unix.go) | `native: newSock` | `socketOpenWithBlock(sock, newSock)` | | `UNIXServer` (socket_unix.go) | `native: newSrv` | `socketOpenWithBlock(srv, newSrv)` | ## Checked before writing: the wrapper could have been a no-op here `socketOpenWithBlock` closes through `cls.methods["close"]` — the class's OWN method table, not its ancestors. A class whose `close` is inherited would have taken the wrapper and closed nothing, and the fix would have read as done. All three define `close` directly: UNIXSocket through `installStreamIO` (socket.go:480), UNIXServer at socket_unix.go:144, UDPSocket at socket_udp.go:170. That is why the test asserts `closed?` rather than trusting the wrapper. ## ⛔ Each door proved by MUTATION, and each by ASSERTION A test that passes before and after proves nothing, so each wiring was reverted in turn: | reverted | the test said | | --- | --- | | `UDPSocket` | `[:udp, #<UDPSocket>, :block_never_ran]` + `[:udp_raise, false, :block_never_ran]` | | `UNIXSocket` | `[:unixsocket, #<UNIXSocket>, :block_never_ran]` | | `UNIXServer` | `[:unixserver, #<UNIXServer>, :block_never_ran, true]` | Three separate reds, each naming its own door and leaving the other two lines intact — so this is one case per door, not one case that happens to cover one of them. The first draft failed by `NoMethodError: undefined method 'closed?' for nil` instead. That is a panic standing in for an assertion: it detects the regression but does not say what happened. Each case now prints `:block_never_ran` for a variable the block never set, which makes a discarded block a readable diff. ⛔ And the UNIXSocket case deliberately has its server `close` AT ONCE without reading. A server waiting for EOF would, with the door unwired, leave the client open and HANG this test rather than fail it — the exact failure this whole issue is about, reproduced inside its own regression test. ## Two traps recorded in the code rather than worked around - **`t.TempDir()` puts the TEST'S NAME in the path, and `sun_path` is 104 bytes on darwin.** This test's name alone pushed the socket over, and `bind(2)` answers `invalid argument` — which reads as a bug in the code under test. The neighbouring tests fit only because their names are shorter, which is not a property anyone would think to preserve when renaming one. Short `os.MkdirTemp` dir, with the reason in a comment. - A **backtick inside the Ruby source** closes the Go raw string that holds it. The test's Ruby comments say `.new`, not a quoted `new`, for that reason. ## A fourth site, outside the socket family, filed rather than fixed Deriving the population instead of trusting #931's list found `internal/vm/resolv.go:199`: dns.smethods["open"] = dns.smethods["new"] `Resolv::DNS.open` with a block is the same defect. MRI 4.0.7 (`lib/ruby/4.0.0/resolv.rb:301-309`, read on this machine) yields, returns the block's value and closes in an `ensure`. It is NOT fixed here, because rbgo's `Resolv::DNS#close` currently raises `NotImplementedError` (resolv.go:207, in the "needs real DNS sockets" list, where closing a resolver that opened nothing does not belong) — so wiring the block form naively would trade a silent discard for a raise. Two steps, its own change, filed separately. Full `internal/vm` suite green (386 s under a 1-minute load average of 61); `gofmt -l .` silent. Refs #931, #932, #916 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
tannevaled
added a commit
that referenced
this pull request
Nov 17, 2026
…lter (#958) #950's second half asked whether Images::Image.open discarding its block is a gap, the way the socket doors' discarded blocks were (#931/#932: 27 hung spec files) and Augeas.open's was (#952). The row in testdata/open-doors.txt said "UNSPECIFIED, not a known defect" because nobody had read the reference. Read now. go-ruby-images' README calls itself "a MiniMagick / ruby-vips replacement", and MiniMagick's lib/mini_magick/image.rb declares: def self.open(path_or_url, ext = nil, **options) with NO block parameter — no `yield`, no `block_given?`, no `ensure`. The only blocks in its body are internal, handed to `uri.open` and `pathname.open`. ruby-vips has no `Image.open` at all; it spells this `Vips::Image.new_from_file`. A Ruby method that takes no block and is handed one ignores it, so rbgo is already right. Nothing to fix. ⛔⛔ AND THE "FIX" WOULD BE WORSE THAN POINTLESS. socketOpenWithBlock closes through the CLASS'S OWN `close`, and on Images::Image `close` is the MORPHOLOGICAL CLOSING operator — dilate then erode, taking a radius — exactly as `#open` is morphological opening. Wrapping the door would run an image filter on the way out of every block, believing it had released something. That is measured, not reasoned: wiring it makes the program die with ArgumentError: wrong number of arguments (given 0, expected 1) raised by that operator, called with no radius from the helper's defer. So three things land, none of them a behaviour change: - TestImagesImageOpenTakesNoBlock pins the contract. Without it the correct behaviour is INDISTINGUISHABLE from the defect it resembles: three other `open` doors that discarded a block were real bugs, so the next reader has every reason to assume a fourth. It asserts both halves a yield-only change would get wrong — the block does not run, and `open` answers the IMAGE, not the block's value. - the open-doors row is upgraded from UNSPECIFIED to reference-read, and says why the door STAYS listed: the row is what protects a correct door from being repaired. - socketOpenWithBlock's comment now states the assumption it was carrying silently — that `close` means "release" on the class being wrapped — with this counter-example. Nothing in the helper can tell the two meanings apart, since a native checks its own arguments and even arity is not introspectable; the door table is the check that can, so the instruction is to read what a new class's `close` does before wrapping it. ⛔ What I did NOT file, having looked: rbgo's `Image.open` takes one argument where MiniMagick takes `(path_or_url, ext = nil, **options)`, and `imagesArity` is a MINIMUM, so a second argument is silently ignored rather than refused. That is cosmetic for decoding — go-ruby-images' Open calls image.Decode, which SNIFFS the format from content via the registered PNG/JPEG/GIF/BMP/TIFF/WebP decoders, so the `ext` hint changes nothing either way. The URL half is a real gap (MiniMagick fetches http/https/ftp) and is #957 rather than smuggled in here, because implementing it gives the interpreter an outbound fetch driven by a path-shaped argument — a judgement for the maintainer, not a parity checkbox. Verified by mutation, not by reading: wiring the door red-lines BOTH gates independently — the behavioural test, and the ratchet, which reports the row as one that no longer needs one. Reverted, both green. gofmt clean, buildscan --vet under all 19. Closes #950 Refs #957, #931, #932, #952 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
socket:
openwith a block must close, which is 27 of the hangs in #916TCPSocket.open,TCPServer.openandSocket.openwere bare aliases for.new, so the block form closed nothing: the socket outlived the block and thepeer waited for an EOF that never came.
⛔ That is the whole
net-httpcolumn of issue 916library/net-http/http/fixtures/http_server.rbserves one client at a timeand shuts down by opening a new connection to write an unterminated
"CLOSE":With that socket still open the server's
gets("\r\n\r\n")never returns, thelistenloop never comes back round toaccept, and@thread.joinnevercompletes. Every one of the 27
library/net-httpfiles ends on that join.Measured, against the restored MRI oracle
The ten-line reproduction, before:
closed?after the blockgets"CLOSE"join:joinedAfter this change rbgo prints the same seven lines as MRI, in the same order,
and exits 0.
And over
library/net-http+library/socket+library/timeout, 321 files,30 s each:
27 hangs gone, +31 examples. The nine that remain are the other two columns
of that issue, untouched here:
library/socket×8 andlibrary/timeout×1.(
net-http/http/start_spec.rbis now killed on the control — MRI hangs onit and rbgo passes 12 — so the net-http column is 27 → 0.)
⛔ It is not the GVL, which is what I expected to find
A goroutine dump of a hung
get_spec.rbputs the server thread correctlyinside
ioBlockwith the GVL released (readUntil→bufio.ReadByte→net.(*conn).Read) and the main thread inthreadJoin. The emulated GVLbehaves; the socket was simply still open.
Nor does
Net::HTTPleak its own connection — my first guess, and wrong: thereproduction shows
peer closed (gets returned nil)after the request underboth engines.
How it closes, and why there is no fourth spelling of
closeIt sends the class's own
closemethod, soTCPSocket,TCPServerandSocketeach keep the single close they already had. Each is guarded bys.closedand therefore idempotent, so a block that closed the socket itselfis not closed twice. The close runs in a
defer, so an exception or abreakleaves it closed too: this is an ensure, not a tidy-up on the happy path.
The test fails rather than hangs
It asserts the contract on the socket the block was handed, not through a
peer: the block's value comes back (not the socket),
closed?is true after anormal return and after an exception, and
openwithout a block still givesan open socket exactly as
.newdoes. A regression therefore fails this testinstead of hanging it.
Refs #931, #916
(Both stay open: #931 also asks for an audit of the same block form on
UNIXSocketandUDPSocket, and forgets(sep)returning partial data at EOF,neither of which this change settles; #916 still has nine hangs left.)
🤖 Generated with Claude Code