Skip to content

Carry the ruby2_keywords flag through the splat binding - #728

Merged
tannevaled merged 4 commits into
mainfrom
wave-c-ruby2-keywords-flag
Sep 28, 2026
Merged

tannevaled merged 4 commits into
mainfrom
wave-c-ruby2-keywords-flag

Conversation

@tannevaled

@tannevaled tannevaled commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

What was wrong

Module#ruby2_keywords and Proc#ruby2_keywords were already accepted, arity-checked and warned about correctly — but the flag they exist to set was never written anywhere. internal/vm/modulefunc.go said so in a comment:

The flag itself is not carried yet: MRI writes ISEQ_BODY(...)->param.flags.ruby2_keywords, which rbgo's bytecode.ISeq has no field for, and honouring it belongs to the splat binding.

So every delegation through a *rest silently lost its keywords.

The two halves

ruby2_keywords only works if both of these happen, and either one alone is unobservable:

  1. Marking — a method or proc whose ISeq carries the flag, called with keywords, binds a flagged copy of the trailing keyword Hash into its *rest. MRI's setup_parameters_complex (vm_args.c, the ISEQ_BODY(iseq)->param.flags.ruby2_keywords && kw_flag arm): rb_hash_dup + RHASH_SET_PASS_AS_KEYWORDS. The caller's hash is left alone.
  2. Re-splat — when an argument Array is splatted at a call site and its last element is a flagged Hash, the call becomes a keyword call (vm_caller_setup_arg_splat sets VM_CALL_KW_SPLAT), and the callee receives a fresh unmarked copy. The flag travels exactly one delegation hop.

Marking alone satisfies Hash.ruby2_keywords_hash? and still fails every delegation, which is why the tests assert the predicate and the delegated call together.

The one shape that must not be touched is a flagged Hash passed as an ordinary positional argument — obj.single(marked) keeps both its identity and its flag. That is why the re-splat lives in applyKWSplat, reached only by the array-dispatch opcodes, and not on the generic send path.

Ablation — one cause or two?

Two halves of one cause, with a strict dependency. Each built alone against every witness file:

spec file base marking only re-splat only both MRI 4.0.5
core/module/ruby2_keywords_spec.rb 6p/8f 11p/2f 6p/8f 14p/0f 14p/0f
core/proc/ruby2_keywords_spec.rb 4p/2f 6p/0f 4p/2f 6p/0f 6p/0f
core/hash/ruby2_keywords_hash_spec.rb 9p/2f 10p/1f 9p/2f 10p/1f 11p/0f
language/keyword_arguments_spec.rb 13p/9f 13p/9f 13p/9f 14p/9f 23p/0f
core/string/chilled_string_spec.rb 6p/11f 6p/11f 6p/11f 6p/11f 17p/0f

The re-splat alone moves nothing at all — with nothing ever marked it has no input. Marking alone gets 11 of 14 module examples; the three it cannot reach are the delegation shapes. The chilled-string row is the control for independence: neither half moves it.

main.ruby2_keywords

MRI registers it as a private singleton method of main, beside top_public and top_private (vm_method.c:3560, body top_ruby2_keywords:3252 → rb_mod_ruby2_keywords on main's default definee). main is not a Module, so Module's private instance method never reached it:

def m(*args) = args.last
ruby2_keywords :m        # NoMethodError before this PR
p Hash.ruby2_keywords_hash?(m(a: 1))   # => true, as MRI

The four call shapes, and which MRI flag decides

A Hash can reach a ruby2_keywords *rest four ways and they must not answer alike:

call MRI 4.0.5 this branch
mid(k: 1) marked marked
mid({k: 1}) NOT marked marked — see below
mid(**h) marked marked
mid(*arr), arr.last a plain Hash NOT marked NOT marked

The flag I read. MRI never looks at the runtime type of the last argument; it reads the call info's keyword bits — VM_CALL_KWARG for a written keyword list and VM_CALL_KW_SPLAT for **h — as kw_flag in setup_parameters_complex (vm_args.c, the ISEQ_BODY(iseq)->param.flags.ruby2_keywords && kw_flag arm) and again in vm_caller_setup_arg_splat. rbgo's stand-in for that pair is bytecode.FlagSendNoKW / FlagSendKWSplat, which ruby2KeywordsBindRest receives as noKW.

That gate is load-bearing, not decorative: shape 4 passes only because of it. Disabling the noKW test alone — and nothing else — makes exactly the shape-4 sub-test fail (mid(*arr) delegated = [[], {k: 1}], want [[{k: 1}], {}]). That is the known-bad control for this test. A rule keyed off "is the last element a Hash" would get shapes 2 and 4 wrong; this one gets shape 4 right.

Shape 2 is rbgo's one divergence, and it is not this feature's. go-ruby-parser@v0.8.0 — the latest published release, checked against the module proxy — parses f({k: 1}) and f(k: 1) into the same *ast.HashLit, with no brace record, no position to re-read the source at, and ast.Call carrying only Args []Node. So noKW is false for both spellings, everywhere in rbgo. Control, on origin/main, with no ruby2_keywords involved at all:

def f(*a, **k) = [a, k]
p f({x: 1})   # MRI [[{x: 1}], {}]   origin/main [[], {x: 1}]
p f(x: 1)     # MRI [[], {x: 1}]     origin/main [[], {x: 1}]

main already reads a braced literal as keywords; this branch inherits that verdict rather than introducing one. The test therefore pins the invariant instead of the wrong value — a ruby2_keywords delegator must read a braced literal exactly as a plain **kwargs method does, so this feature can never become more wrong than the rest of the VM — and when the parser grows a Braced bit both sides move together and the test keeps passing.

I could not close shape 2 from inside rbgo, and I did not want to trade it for shape 1: requiring positive proof of keywords (i.e. only FlagSendKWSplat) would fix mid({k: 1}) and break mid(k: 1), which is the whole point of ruby2_keywords, and would give back 8 of the 12 examples below. The real fix is a Braced bit on ast.HashLit upstream, which would also close 6 of the 9 remaining language/keyword_arguments_spec.rb failures.

Conformance

Population: the 2206 language/ + core/ *_spec.rb files of the rubyspec corpus at SPEC_SHA 87b1631. Full sweep both sides, same corpus, same harness, rows==inputs==2206 and identical structural verdicts on both sides (OK=2024 ZERO=178 TIMEOUT=3 DIED=1).

Passing examples 23394 → 23407 (+13 raw, +12 attributable).

spec file before after
core/module/ruby2_keywords_spec.rb 6 14 (MRI parity)
core/proc/ruby2_keywords_spec.rb 4 6 (MRI parity)
core/hash/ruby2_keywords_hash_spec.rb 9 10
language/keyword_arguments_spec.rb 13 14
core/random/bytes_spec.rb 8 9 — flake, not attributable

core/random/bytes_spec.rb was re-run three times on each binary and flips 8/9 on both, so it is excluded from the attributable total.

What is still red, and why

Measured, not assumed:

  • language/keyword_arguments_spec.rb, 9 remaining, classified by reading the assertion each example dies on:

    • 6 die on the documented braced-hash-literal limitation. go-ruby-parser@v0.8.0's ast.HashLit has no Braced field, so the compiler cannot tell m({a: 1}) from m(a: 1) — the FlagSendNoKW doc comment already says so, and I confirmed the field is still absent upstream. Advancing them needs a parser change.
    • 2 (proc { |*args, **kwargs| }, -> (*args, **kwargs) {}) die on m.(empty): Proc#call is a native re-dispatch and cannot carry the call site's keyword/positional verdict.
    • 1 (zsuper) dies on obj.m(empty): the implicit-super path does not carry FlagSendNoKW, so an empty positional Hash is peeled back into keywords.

    All three non-braced causes predate this change and are unchanged by it. What this PR did move here is three examples — call(*ruby2_keyword_args), super(*ruby2_keyword_args) and yield(*ruby2_keyword_args) — past their previous failing assertion (the m(a: 1) delegation, which now passes) onto the braced-literal one.

  • core/hash/ruby2_keywords_hash_spec.rb, 1 remaining — "copies instance variables". Hash.ruby2_keywords_hash does not copy ivars. A separate defect in internal/vm/builtins.go, outside this change's cluster.

  • core/main/ruby2_keywords_spec.rb, 1 remaining. The registration above is correct and works, but the example asserts main.private_methods(false), and rbgo's private_methods(false) does not honour its false: it returns 60 names where MRI returns 6 ([:define_method, :include, :private, :public, :ruby2_keywords, :using]), and omits :private too. A reflectMethodNames defect in internal/vm/builtins.go, unrelated to this change.

Files outside the wave-C cluster

Two small hooks had to land in internal/vm/vm.go, which wave A owns. Both are call-outs into the new file, so a conflict resolves in one line:

  • applyKWSplat — 5 added lines, the re-splat.
  • exec's splat binding — 4 added lines (3 comment, 1 call), the marking.

internal/bytecode/bytecode.go, internal/vm/modulefunc.go and internal/vm/proc.go are claimed by no wave.

Byte-for-byte against MRI 4.0.5

A 14-line delegation program (single/kwargs/splat/splat1/proc_call/send, direct positional, Proc#clone, zero-arg, and the identity/flag of every hash involved) run on ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +PRISM [arm64-darwin25] and on this branch: 13 of 14 lines identical. The one divergence is o.m({a: 9}) — the braced-hash-literal parser limitation above, which predates this change.

Checks

  • CGO_ENABLED=0 go test ./... green (Go 1.27.1, the CI toolchain), 0 FAIL lines.
  • gofmt -l internal/ cmd/ clean.
  • Coverage. Two full -coverpkg=$(go list ./internal/...) runs, one in a clean origin/main worktree and one here, each cover -func entry resolved in the tree that produced its profile. Totals were not compared:
    • below-100 function list: 16 entries on each side, identical (diff empty);
    • full function set: main 7391, branch 7397 — the six added are exactly the new file's, and none of them is in the below-100 list, so all six are at 100.0%. Nothing vanished.

🤖 Generated with Claude Code

tannevaled and others added 4 commits September 28, 2026 12:54
Module#ruby2_keywords and Proc#ruby2_keywords were accepted and checked
but the flag they set was never written anywhere, so every delegation
shape still lost its keywords. Two halves were missing, and either one
alone is unobservable: the callee must FLAG the trailing keyword Hash it
binds into its *rest, and a call site splatting that Hash must re-splat
it AS keywords.

bytecode.ISeq gains Ruby2Keywords, MRI's
ISEQ_BODY(iseq)->param.flags.ruby2_keywords. It lives on the ISeq, not
the Method or the Proc, because that is what makes it survive aliasing
(vm_method.c rb_mod_ruby2_keywords:3114) and Proc#dup/#clone (proc.c
proc_ruby2_keywords:4133), both at tag ruby_4_0.

wave_c_ruby2_keywords.go holds the two halves. The marking copies the
hash before flagging it, so the caller's hash is untouched; the re-splat
hands the callee a fresh UNMARKED copy, so the flag travels exactly one
delegation hop. A marked Hash passed as an ordinary positional argument
keeps both its identity and its flag, which is why the re-splat sits in
applyKWSplat -- reached only by the array-dispatch opcodes -- and not on
the generic send path.

core/module/ruby2_keywords_spec.rb 6 -> 14 of 14 (MRI parity)
core/proc/ruby2_keywords_spec.rb    4 -> 6  of 6  (MRI parity)
core/hash/ruby2_keywords_hash_spec.rb 9 -> 10 of 11

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MRI registers it beside top_public and top_private, on main's singleton
class, with rb_mod_ruby2_keywords applied to main's default definee
(vm_method.c:3560 and top_ruby2_keywords:3252 at tag ruby_4_0). main is
not a Module, so Module's private instance method never reached it:

  def m(*args); args.last; end
  ruby2_keywords :m

died with NoMethodError and now marks Object#m, as MRI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Marking without propagation passes Hash.ruby2_keywords_hash? and still
fails every delegation, so the end-to-end cases assert the predicate and
the delegated call in the same test. The unit arms cover the shapes no
Ruby program reaches through the compiler, and a negative control — an
(*a, **k) method, which MRI refuses to mark — so the flag cannot pass by
being unconditional.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
They must not all answer alike, and a rule keyed off the runtime type of
the last argument would make them. What decides in MRI is the call
site's keyword flag -- VM_CALL_KWARG for a written keyword list,
VM_CALL_KW_SPLAT for `**h` -- read as kw_flag in
setup_parameters_complex and in vm_caller_setup_arg_splat. rbgo's
stand-in is bytecode.FlagSendNoKW / FlagSendKWSplat, which
ruby2KeywordsBindRest reads as noKW.

  mid(k: 1)    marked
  mid({k: 1})  NOT marked
  mid(**h)     marked
  mid(*arr)    NOT marked

Three are asserted at MRI 4.0.5's answers. Shape 4 passes only because
the noKW gate is consulted: disabling that gate alone makes exactly that
sub-test fail, which is the known-bad control for this test.

Shape 2 is rbgo's one divergence and it is not this feature's.
go-ruby-parser v0.8.0 -- the latest release -- parses `f({k: 1})` and
`f(k: 1)` into the same *ast.HashLit, with no brace record and no
position to re-read the source at, and ast.Call carries only Args. On
origin/main an ordinary `def f(*a, **k)` already reads `f({x: 1})` as
[[], {x: 1}] where MRI reads [[{x: 1}], {}]. So the sub-test pins the
INVARIANT rather than the wrong value: a ruby2_keywords delegator must
read a braced literal exactly as a plain **kwargs method does. When the
parser grows a Braced bit both move together and the test still passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled

Copy link
Copy Markdown
Contributor Author

Retracting my block. I checked the mechanism myself and the author is right: I mis-aimed it, and the fix I asked for would have been wrong.

Measured on origin/main, with no ruby2_keywords anywhere in the program:

def f(*a, **k) = [a, k]
p f({x: 1})   # MRI [[{x: 1}], {}]   main [[], {x: 1}]   <- already wrong
p f(x: 1)     # MRI [[], {x: 1}]     main [[], {x: 1}]
h = {y: 2}
p f(h)        # MRI [[{y: 2}], {}]   main [[{y: 2}], {}] <- a VARIABLE is read correctly
p f(**h)      # MRI [[], {y: 2}]     main [[], {y: 2}]

So a braced literal at a call site is already read as keywords, and only a braced literal is — which matches the stated cause exactly: go-ruby-parser builds the same *ast.HashLit for f({k: 1}) and f(k: 1), so nothing downstream can tell them apart.

And the delegation shape without the flag is byte-identical on both builds:

def target(*a, **kw) = [a, kw]
def plain(*a) = target(*a)
p plain({k: 1})
# MRI [[{k: 1}], {}] | main [[{k: 1}], {}] | this PR [[{k: 1}], {}]

What this PR actually does to that case is subtler than "gets it wrong", and worth writing down: it removes an accidental cancellation. Without the flag, the parser's misreading at the call site is undone by the positional re-splat at the next hop, and two errors cancel into the right answer. ruby2_keywords propagates the first reading faithfully, so the cancellation stops and the underlying defect becomes visible. That is the feature behaving correctly on wrong input.

Pinning the invariant in shape 2 — a ruby2_keywords delegator must read a braced literal exactly as a plain **kwargs method does — rather than freezing the wrong value is the right call. A test that froze [[], {k: 1}] would entrench the confusion and would have to be edited when the parser is fixed; this one goes green on its own when both move together.

The real fix is a Braced bit on ast.HashLit upstream in go-ruby-parser, which is where I am filing it.

@tannevaled
tannevaled merged commit 1a79138 into main Sep 28, 2026
9 checks passed
@tannevaled
tannevaled deleted the wave-c-ruby2-keywords-flag branch September 28, 2026 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant