Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -875,6 +875,8 @@ Otherwise:
| `Numeric#to_int` | not defined — `Complex(2.9, 0).to_int` raises `NoMethodError`; MRI returns `2` |
| `File::Stat#==` | `a == b` is `false` for two stats of the same file (MRI: `true`); `a.==(b)` answers `true` |
| `File#stat` on a file unlinked while open | raises `Errno::ENOENT`; MRI answers from the open descriptor. rbgo's streams are buffered by path and hold no descriptor |
| `IO#pwrite` / `IO#pread` | positional reads and writes go through the buffer, not a descriptor, so a `#pwrite`'s bytes are not on disk until a flush — MRI's `rb_io_pwrite` calls `pwrite(2)` and they are there the moment it returns. They are no longer *lost*: interpreter teardown flushes them like any buffered write ([#643](https://github.com/go-embedded-ruby/ruby/issues/643)), so a program that `pwrite`s and exits finds them. Only the *immediacy* differs, and it is the same root as the row above: no descriptor. [#949](https://github.com/go-embedded-ruby/ruby/issues/949) |
| an unflushed file whose directory is removed before the program ends | loses its bytes; MRI, holding a descriptor to an unlinked file, still writes them. Teardown re-opens the path to flush it, so it gets `ENOENT` — and reports nothing, as MRI's shutdown ignores a failing `fflush`. Same root again: streams are buffered by path, which is also [#635](https://github.com/go-embedded-ruby/ruby/issues/635) |
| `Errno` | all **158** of MRI's constant names are present, but **32** report errno `0` where MRI has a real number (`EAUTH`, `EBADRPC`, `EDEVERR`, …). Measured by set difference on darwin: 81 names are zero-valued under rbgo, 49 under MRI, and every MRI zero is also zero under rbgo. Go's portable `syscall` package does not expose the platform-only names |
| `pp` | neither `Kernel#pp` nor `require "pp"` exists (`require "prettyprint"` does work) |
| `Process.fork` | does not exist — Go's runtime cannot be forked safely |
Expand Down
88 changes: 88 additions & 0 deletions internal/vm/io.go
Original file line number Diff line number Diff line change
Expand Up @@ -899,6 +899,7 @@ func (vm *VM) openFileArgs(cls *RClass, args []object.Value) *IOObj {
}
o := openFileSpec(cls, path, &ms, perm, permGiven)
o.extEnc, o.intEnc, o.binmode, o.newline = ms.extEnc, ms.intEnc, ms.binmode, ms.newline
vm.trackOpenFile(o) // door 1 of 5: File.new / File.open / IO.new on a path
return o
}

Expand Down Expand Up @@ -1291,6 +1292,93 @@ func ioFlush(o *IOObj) {
}
}

// trackOpenFile records a writable, path-backed stream so interpreter teardown
// can flush it (flushOpenFiles). A stream with nothing to lose — no path, or not
// writable — is not recorded, which keeps IO.foreach and ARGF (both read-only,
// and both deliberately absent from the call sites) out of the slice and keeps a
// program that reads thousands of files from retaining their buffers.
//
// Call it wherever a writable path-backed stream becomes visible to Ruby. There
// are FIVE such sites: File.new/File.open (openFileArgs), IO.sysopen, IO#reopen,
// IO#pwrite, and ioAdoptDescriptor — the last reached by IO.new(fd), IO.for_fd
// and File.new(fd), which build a SECOND IOObj over a descriptor and are the
// door this mechanism was missing when it was first written with four.
// TestEveryWayToGetAWritableFileFlushesAtShutdown covers them one by one, and the
// differential in the pull request found the fifth, because a mechanism wired at
// only some of its doors is this repository's commonest defect shape.
func (vm *VM) trackOpenFile(o *IOObj) {
if o == nil || o.path == "" || !o.writable {
return
}
for _, p := range vm.openFiles {
if p == o { // idempotent: #pwrite and #reopen can land on a tracked stream
return
}
}
// Compact at the growth point: the filter is O(n) exactly when append would
// have copied O(n) anyway, so tracking stays amortised O(1) while the slice
// cannot outgrow the set of streams open at once by more than one step.
if len(vm.openFiles) == cap(vm.openFiles) {
live := vm.openFiles[:0]
for _, p := range vm.openFiles {
if !p.closed {
live = append(live, p)
}
}
vm.openFiles = live
}
vm.openFiles = append(vm.openFiles, o)
}

// flushOpenFiles flushes every tracked stream still open, which is what MRI's
// shutdown does by closing every IO (see openFiles). It runs at the end of
// runAtExit — AFTER the handlers, as MRI's rb_ec_teardown runs
// rb_ec_exec_end_proc before rb_objspace_call_finalizer, so a file an at_exit
// handler itself wrote reaches disk too — and NOT at all under Kernel#exit! /
// Process.exit!, whose _exit(2) skips the handlers and the flush alike.
//
// ⛔ A TEARDOWN FLUSH MUST NOT RAISE, and getting that wrong is how this
// function was first written. ioFlush re-opens the path (see its comment: a
// buffer-backed stream has no descriptor), so it can fail where MRI's fflush on
// a live fd cannot — most ordinarily when the directory has been removed under
// the stream:
//
// Dir.mktmpdir { |d| IO.for_fd(File.open(File.join(d, "f"), "w").fileno, "w").write("foo") }
//
// The block's exit deletes the directory, so the flush that follows gets ENOENT.
// Raising it turned that clean program into `Errno::ENOENT @ rb_sysopen`, which
// is what TestIOForFd and TestIOForFdModeCompat caught. MRI's shutdown ignores a
// failing fflush (rb_io_fptr_finalize keeps no error), and at teardown there is
// no longer anyone who could rescue one, so a Ruby error here is swallowed and
// the remaining streams are still attempted.
//
// What that COSTS is stated rather than hidden: a stream whose path can no
// longer be written loses its bytes, where MRI — holding a descriptor to an
// unlinked file — would still write them. That is the buffer-backed model's own
// limit (IOObj), not a decision taken here.
func (vm *VM) flushOpenFiles() {
files := vm.openFiles
vm.openFiles = nil
for _, o := range files {
if o.closed || !o.writable || !o.wbufDirty {
continue
}
vm.flushOneAtTeardown(o)
}
}

func (vm *VM) flushOneAtTeardown(o *IOObj) {
defer func() {
if r := recover(); r != nil {
if _, ok := r.(RubyError); ok {
return // see flushOpenFiles: teardown reports nothing, as MRI does
}
panic(r) // a broken VM invariant stays loud
}
}()
ioFlush(o)
}

// curStdout / curStderr / curStdin return the IO currently bound to the global,
// falling back to the raw VM writer when a host rebinds it to a non-IO value.
//
Expand Down
11 changes: 11 additions & 0 deletions internal/vm/io_class.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,10 @@ func (vm *VM) registerIOClassMethods(cIO, cFile *RClass) {
mode = vm.vmodeString(args[1]) // Integer flag set or fopen-style String
}
o := openFileIO(cFile, path, mode)
// Door 2 of 5: IO.sysopen's stream, which lives in fdTable. A program that
// writes through this very object (rather than through an IO.for_fd wrapper,
// which is door 5) needs it tracked here.
vm.trackOpenFile(o)
vm.nextFd++
o.fd = vm.nextFd
vm.fdTable[o.fd] = o
Expand Down Expand Up @@ -288,6 +292,13 @@ func (vm *VM) ioAdoptDescriptor(o, src *IOObj, pos []object.Value, opts *object.
}
}
}
// Door 5 of 5 (see trackOpenFile): the wrapper o is a DIFFERENT IOObj from the
// descriptor's src, sharing only its path and its buffer at this instant, and
// it is o that the program writes through. Tracking src at IO.sysopen is not
// enough — measured: IO.sysopen + IO.for_fd + write, unclosed, left the file
// empty under rbgo and held "payload" under MRI 4.0.7, with the sysopen site
// already tracked.
vm.trackOpenFile(o)
}

// splitIOOpts separates a trailing options Hash (IO.read/write keyword arguments)
Expand Down
14 changes: 14 additions & 0 deletions internal/vm/io_descriptors.go
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,14 @@ func defIOSeekable(cls *RClass) {
}
copy(o.buf[off:], data)
o.writable = true // a written File flushes its buffer back on flush/close
// Door 4 of 5. MRI's rb_io_pwrite calls pwrite(2) on the descriptor, so
// there the bytes are on disk the moment #pwrite returns; here they are in
// the buffer, which makes them losable exactly like a buffered #write.
// Marking them dirty and tracking the stream is what keeps them (#643);
// that they are not on disk UNTIL a flush is a separate divergence of the
// buffer-backed model, not one this records as fixed.
o.wbufDirty = true
vm.trackOpenFile(o)
return object.IntValue(int64(len(data)))
})
cls.define("sysseek", func(vm *VM, self object.Value, args []object.Value, _ *Proc) object.Value {
Expand Down Expand Up @@ -733,6 +741,12 @@ func ioReopenPath(vm *VM, o *IOObj, pos []object.Value, opts *object.Hash) objec
o.isStr, o.closed, o.lineno = true, false, 0
o.w, o.pipe, o.isWriteEnd, o.pipeSynced, o.reopened = nil, nil, false, 0, nil
o.closeOnExecOff = false
// Door 3 of 5: the stream now writes a DIFFERENT path, and the one that must
// be flushed at teardown is o — never fresh, which is a throwaway holding the
// same path and the file's pre-reopen bytes, and flushing it would write those
// stale bytes back over o's. That is why openFileIO does not track and its
// callers do.
vm.trackOpenFile(o)
if explicit {
ms := vm.ioResolveModeEnc(pos, opts)
o.extEnc, o.intEnc, o.binmode, o.newline = ms.extEnc, ms.intEnc, ms.binmode, ms.newline
Expand Down
20 changes: 17 additions & 3 deletions internal/vm/kernel_introspect.go
Original file line number Diff line number Diff line change
Expand Up @@ -784,22 +784,36 @@ func (vm *VM) frameFileLabel(i int) string {
}
}

// runAtExit runs the registered at_exit blocks in LIFO order. A raise inside one
// is swallowed so the remaining handlers still run (MRI reports it but keeps
// going); other control-flow signals propagate.
// runAtExit is this VM's interpreter teardown — MRI's rb_ec_teardown (eval.c),
// which has the same two jobs in this order: run the registered at_exit blocks
// in LIFO order (rb_ec_exec_end_proc), then let every open IO reach disk
// (rb_objspace_call_finalizer, whose fptr_finalize fflushes each one). A raise
// inside a handler is swallowed so the remaining handlers still run (MRI reports
// it but keeps going); other control-flow signals propagate.
//
// The order is MRI's and it matters: a file an at_exit handler itself writes is
// flushed, because the flush comes second.
func (vm *VM) runAtExit() {
// Kernel#exit! / Process.exit! runs none of them: process.c rb_f_exit_bang
// calls _exit(2), which is precisely "terminate without running the exit
// handlers". Measured against MRI 4.0.5: `at_exit { puts "x" }; exit!(4)`
// prints nothing and exits 4.
//
// _exit(2) skips the IO flush too — it is the libc exit handlers and the
// object-space finalizer that it bypasses, not just the Ruby ones — so an
// unflushed buffer is LOST under exit!, and this return is what reproduces
// that. Measured against MRI 4.0.7, writing 12 bytes to a file and never
// closing it: `exit` leaves 12 bytes on disk, `exit!` leaves 0.
if vm.exitImmediate {
vm.atExit = nil
vm.openFiles = nil
return
}
for i := len(vm.atExit) - 1; i >= 0; i-- {
vm.runAtExitOne(vm.atExit[i])
}
vm.atExit = nil
vm.flushOpenFiles()
}

func (vm *VM) runAtExitOne(blk *Proc) {
Expand Down
Loading
Loading