Repository navigation
Carry the ruby2_keywords flag through the splat binding - #728
Conversation
Module#ruby2_keywords and Proc#ruby2_keywords were accepted and checked but the flag they set was never written anywhere, so every delegation shape still lost its keywords. Two halves were missing, and either one alone is unobservable: the callee must FLAG the trailing keyword Hash it binds into its *rest, and a call site splatting that Hash must re-splat it AS keywords. bytecode.ISeq gains Ruby2Keywords, MRI's ISEQ_BODY(iseq)->param.flags.ruby2_keywords. It lives on the ISeq, not the Method or the Proc, because that is what makes it survive aliasing (vm_method.c rb_mod_ruby2_keywords:3114) and Proc#dup/#clone (proc.c proc_ruby2_keywords:4133), both at tag ruby_4_0. wave_c_ruby2_keywords.go holds the two halves. The marking copies the hash before flagging it, so the caller's hash is untouched; the re-splat hands the callee a fresh UNMARKED copy, so the flag travels exactly one delegation hop. A marked Hash passed as an ordinary positional argument keeps both its identity and its flag, which is why the re-splat sits in applyKWSplat -- reached only by the array-dispatch opcodes -- and not on the generic send path. core/module/ruby2_keywords_spec.rb 6 -> 14 of 14 (MRI parity) core/proc/ruby2_keywords_spec.rb 4 -> 6 of 6 (MRI parity) core/hash/ruby2_keywords_hash_spec.rb 9 -> 10 of 11 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MRI registers it beside top_public and top_private, on main's singleton class, with rb_mod_ruby2_keywords applied to main's default definee (vm_method.c:3560 and top_ruby2_keywords:3252 at tag ruby_4_0). main is not a Module, so Module's private instance method never reached it: def m(*args); args.last; end ruby2_keywords :m died with NoMethodError and now marks Object#m, as MRI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Marking without propagation passes Hash.ruby2_keywords_hash? and still fails every delegation, so the end-to-end cases assert the predicate and the delegated call in the same test. The unit arms cover the shapes no Ruby program reaches through the compiler, and a negative control — an (*a, **k) method, which MRI refuses to mark — so the flag cannot pass by being unconditional. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
They must not all answer alike, and a rule keyed off the runtime type of
the last argument would make them. What decides in MRI is the call
site's keyword flag -- VM_CALL_KWARG for a written keyword list,
VM_CALL_KW_SPLAT for `**h` -- read as kw_flag in
setup_parameters_complex and in vm_caller_setup_arg_splat. rbgo's
stand-in is bytecode.FlagSendNoKW / FlagSendKWSplat, which
ruby2KeywordsBindRest reads as noKW.
mid(k: 1) marked
mid({k: 1}) NOT marked
mid(**h) marked
mid(*arr) NOT marked
Three are asserted at MRI 4.0.5's answers. Shape 4 passes only because
the noKW gate is consulted: disabling that gate alone makes exactly that
sub-test fail, which is the known-bad control for this test.
Shape 2 is rbgo's one divergence and it is not this feature's.
go-ruby-parser v0.8.0 -- the latest release -- parses `f({k: 1})` and
`f(k: 1)` into the same *ast.HashLit, with no brace record and no
position to re-read the source at, and ast.Call carries only Args. On
origin/main an ordinary `def f(*a, **k)` already reads `f({x: 1})` as
[[], {x: 1}] where MRI reads [[{x: 1}], {}]. So the sub-test pins the
INVARIANT rather than the wrong value: a ruby2_keywords delegator must
read a braced literal exactly as a plain **kwargs method does. When the
parser grows a Braced bit both move together and the test still passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Retracting my block. I checked the mechanism myself and the author is right: I mis-aimed it, and the fix I asked for would have been wrong. Measured on def f(*a, **k) = [a, k]
p f({x: 1}) # MRI [[{x: 1}], {}] main [[], {x: 1}] <- already wrong
p f(x: 1) # MRI [[], {x: 1}] main [[], {x: 1}]
h = {y: 2}
p f(h) # MRI [[{y: 2}], {}] main [[{y: 2}], {}] <- a VARIABLE is read correctly
p f(**h) # MRI [[], {y: 2}] main [[], {y: 2}]So a braced literal at a call site is already read as keywords, and only a braced literal is — which matches the stated cause exactly: And the delegation shape without the flag is byte-identical on both builds: def target(*a, **kw) = [a, kw]
def plain(*a) = target(*a)
p plain({k: 1})
# MRI [[{k: 1}], {}] | main [[{k: 1}], {}] | this PR [[{k: 1}], {}]What this PR actually does to that case is subtler than "gets it wrong", and worth writing down: it removes an accidental cancellation. Without the flag, the parser's misreading at the call site is undone by the positional re-splat at the next hop, and two errors cancel into the right answer. Pinning the invariant in shape 2 — a The real fix is a |
What was wrong
Module#ruby2_keywordsandProc#ruby2_keywordswere already accepted, arity-checked and warned about correctly — but the flag they exist to set was never written anywhere.internal/vm/modulefunc.gosaid so in a comment:So every delegation through a
*restsilently lost its keywords.The two halves
ruby2_keywordsonly works if both of these happen, and either one alone is unobservable:*rest. MRI'ssetup_parameters_complex(vm_args.c, theISEQ_BODY(iseq)->param.flags.ruby2_keywords && kw_flagarm):rb_hash_dup+RHASH_SET_PASS_AS_KEYWORDS. The caller's hash is left alone.vm_caller_setup_arg_splatsetsVM_CALL_KW_SPLAT), and the callee receives a fresh unmarked copy. The flag travels exactly one delegation hop.Marking alone satisfies
Hash.ruby2_keywords_hash?and still fails every delegation, which is why the tests assert the predicate and the delegated call together.The one shape that must not be touched is a flagged Hash passed as an ordinary positional argument —
obj.single(marked)keeps both its identity and its flag. That is why the re-splat lives inapplyKWSplat, reached only by the array-dispatch opcodes, and not on the generic send path.Ablation — one cause or two?
Two halves of one cause, with a strict dependency. Each built alone against every witness file:
core/module/ruby2_keywords_spec.rbcore/proc/ruby2_keywords_spec.rbcore/hash/ruby2_keywords_hash_spec.rblanguage/keyword_arguments_spec.rbcore/string/chilled_string_spec.rbThe re-splat alone moves nothing at all — with nothing ever marked it has no input. Marking alone gets 11 of 14 module examples; the three it cannot reach are the delegation shapes. The chilled-string row is the control for independence: neither half moves it.
main.ruby2_keywordsMRI registers it as a private singleton method of
main, besidetop_publicandtop_private(vm_method.c:3560, bodytop_ruby2_keywords:3252→rb_mod_ruby2_keywordson main's default definee).mainis not a Module, soModule's private instance method never reached it:The four call shapes, and which MRI flag decides
A Hash can reach a
ruby2_keywords*restfour ways and they must not answer alike:mid(k: 1)mid({k: 1})mid(**h)mid(*arr),arr.lasta plain HashThe flag I read. MRI never looks at the runtime type of the last argument; it reads the call info's keyword bits —
VM_CALL_KWARGfor a written keyword list andVM_CALL_KW_SPLATfor**h— askw_flaginsetup_parameters_complex(vm_args.c, theISEQ_BODY(iseq)->param.flags.ruby2_keywords && kw_flagarm) and again invm_caller_setup_arg_splat. rbgo's stand-in for that pair isbytecode.FlagSendNoKW/FlagSendKWSplat, whichruby2KeywordsBindRestreceives asnoKW.That gate is load-bearing, not decorative: shape 4 passes only because of it. Disabling the
noKWtest alone — and nothing else — makes exactly the shape-4 sub-test fail (mid(*arr) delegated = [[], {k: 1}], want [[{k: 1}], {}]). That is the known-bad control for this test. A rule keyed off "is the last element a Hash" would get shapes 2 and 4 wrong; this one gets shape 4 right.Shape 2 is rbgo's one divergence, and it is not this feature's.
go-ruby-parser@v0.8.0— the latest published release, checked against the module proxy — parsesf({k: 1})andf(k: 1)into the same*ast.HashLit, with no brace record, no position to re-read the source at, andast.Callcarrying onlyArgs []Node. SonoKWis false for both spellings, everywhere in rbgo. Control, onorigin/main, with noruby2_keywordsinvolved at all:main already reads a braced literal as keywords; this branch inherits that verdict rather than introducing one. The test therefore pins the invariant instead of the wrong value — a
ruby2_keywordsdelegator must read a braced literal exactly as a plain**kwargsmethod does, so this feature can never become more wrong than the rest of the VM — and when the parser grows aBracedbit both sides move together and the test keeps passing.I could not close shape 2 from inside rbgo, and I did not want to trade it for shape 1: requiring positive proof of keywords (i.e. only
FlagSendKWSplat) would fixmid({k: 1})and breakmid(k: 1), which is the whole point ofruby2_keywords, and would give back 8 of the 12 examples below. The real fix is aBracedbit onast.HashLitupstream, which would also close 6 of the 9 remaininglanguage/keyword_arguments_spec.rbfailures.Conformance
Population: the 2206
language/+core/*_spec.rbfiles of the rubyspec corpus at SPEC_SHA 87b1631. Full sweep both sides, same corpus, same harness,rows==inputs==2206and identical structural verdicts on both sides (OK=2024 ZERO=178 TIMEOUT=3 DIED=1).Passing examples 23394 → 23407 (+13 raw, +12 attributable).
core/module/ruby2_keywords_spec.rbcore/proc/ruby2_keywords_spec.rbcore/hash/ruby2_keywords_hash_spec.rblanguage/keyword_arguments_spec.rbcore/random/bytes_spec.rbcore/random/bytes_spec.rbwas re-run three times on each binary and flips 8/9 on both, so it is excluded from the attributable total.What is still red, and why
Measured, not assumed:
language/keyword_arguments_spec.rb, 9 remaining, classified by reading the assertion each example dies on:go-ruby-parser@v0.8.0'sast.HashLithas noBracedfield, so the compiler cannot tellm({a: 1})fromm(a: 1)— theFlagSendNoKWdoc comment already says so, and I confirmed the field is still absent upstream. Advancing them needs a parser change.proc { |*args, **kwargs| },-> (*args, **kwargs) {}) die onm.(empty):Proc#callis a native re-dispatch and cannot carry the call site's keyword/positional verdict.zsuper) dies onobj.m(empty): the implicit-superpath does not carryFlagSendNoKW, so an empty positional Hash is peeled back into keywords.All three non-braced causes predate this change and are unchanged by it. What this PR did move here is three examples —
call(*ruby2_keyword_args),super(*ruby2_keyword_args)andyield(*ruby2_keyword_args)— past their previous failing assertion (them(a: 1)delegation, which now passes) onto the braced-literal one.core/hash/ruby2_keywords_hash_spec.rb, 1 remaining — "copies instance variables".Hash.ruby2_keywords_hashdoes not copy ivars. A separate defect ininternal/vm/builtins.go, outside this change's cluster.core/main/ruby2_keywords_spec.rb, 1 remaining. The registration above is correct and works, but the example assertsmain.private_methods(false), and rbgo'sprivate_methods(false)does not honour itsfalse: it returns 60 names where MRI returns 6 ([:define_method, :include, :private, :public, :ruby2_keywords, :using]), and omits:privatetoo. AreflectMethodNamesdefect ininternal/vm/builtins.go, unrelated to this change.Files outside the wave-C cluster
Two small hooks had to land in
internal/vm/vm.go, which wave A owns. Both are call-outs into the new file, so a conflict resolves in one line:applyKWSplat— 5 added lines, the re-splat.exec's splat binding — 4 added lines (3 comment, 1 call), the marking.internal/bytecode/bytecode.go,internal/vm/modulefunc.goandinternal/vm/proc.goare claimed by no wave.Byte-for-byte against MRI 4.0.5
A 14-line delegation program (
single/kwargs/splat/splat1/proc_call/send, direct positional,Proc#clone, zero-arg, and the identity/flag of every hash involved) run onruby 4.0.5 (2026-05-20 revision 64336ffd0e) +PRISM [arm64-darwin25]and on this branch: 13 of 14 lines identical. The one divergence iso.m({a: 9})— the braced-hash-literal parser limitation above, which predates this change.Checks
CGO_ENABLED=0 go test ./...green (Go 1.27.1, the CI toolchain), 0FAILlines.gofmt -l internal/ cmd/clean.-coverpkg=$(go list ./internal/...)runs, one in a cleanorigin/mainworktree and one here, eachcover -funcentry resolved in the tree that produced its profile. Totals were not compared:diffempty);main7391, branch 7397 — the six added are exactly the new file's, and none of them is in the below-100 list, so all six are at 100.0%. Nothing vanished.🤖 Generated with Claude Code