Skip to content

Emit shortest round-trip decimal for numbers (RFC 8785 §3.2.2.3) - #7

Open
dngr2 wants to merge 1 commit into
dryruby:developfrom
dngr2:fix-shortest-number-serialization
Open

Emit shortest round-trip decimal for numbers (RFC 8785 §3.2.2.3)#7
dngr2 wants to merge 1 commit into
dryruby:developfrom
dngr2:fix-shortest-number-serialization

Conversation

@dngr2

@dngr2 dngr2 commented Aug 16, 2026

Copy link
Copy Markdown

The bug

Numeric#to_json_c14n serializes numbers with "%.15E" % num, which produces 16 significant digits. IEEE-754 doubles need up to 17 significant digits to round-trip. Any value that needs the 17th digit is canonicalized to a string that parses back to a different double — a direct violation of RFC 8785 §3.2.2.3, which mandates the ECMAScript ("shortest") number form.

The canonical example:

(0.1 + 0.2).to_json_c14n   # => "0.3"   (parses back to 0.3, NOT 0.30000000000000004)

Because a canonical form that doesn't round-trip defeats the entire purpose of canonicalization (two parties hashing "the same" value get different bytes), this bites real doubles constantly. Testing the current code against Node's String(x) (the ES6 reference the RFC points to) over ~20k random doubles, 9,152 diverged. A few:

input current output correct (ES6 / RFC 8785)
0.1 + 0.2 0.3 0.30000000000000004
1.7976931348623157e308 (Float::MAX) 1.797693134862316e+308 1.7976931348623157e+308
5e-324 (Float::MIN subnormal) 4.940656458412465e-324 5e-324
123456789012345680.0 123456789012345700 123456789012345680

Large integers are affected too — they're coerced through the same 16-digit float path:

295147905179352830000.to_json_c14n   # => "295147905179352800000"  (should be ...830000)

This is exactly why spec/number_spec.rb had a block of cases commented out as "Outside Ruby Range". They aren't outside Ruby's range at all (5e-324, 1e+23, Float::MAX are all perfectly representable) — they were commented out because %.15E couldn't produce their correct canonical form.

The fix

Ruby's Float#to_s already emits the shortest decimal that round-trips. This PR takes the digit sequence from Float#to_s instead of %.15E, and normalizes it into the single-leading-digit form the existing exponent-placement logic already expects. That logic (positional vs. exponential thresholds, +/- exponent formatting, -00) is unchanged.

Verification

  • Conformance: patched gem vs. Node String(x) over the same ~20k-double battery (edge cases + subnormals + random bit patterns): 0 divergences (was 9,152). End-to-end, a mixed nested object canonicalizes byte-identically to the reference canonicalize.js.
  • No regression: every previously-passing number_spec case still passes; the RangeError cases still raise.
  • Restored coverage: the previously-commented "Outside Ruby Range" cases are un-commented and pass, plus new round-trip regressions (0.1 + 0.2, 123456789012345680.0). rspec spec/number_spec.rb: 24 examples, 0 failures.

The number serializer formatted values with "%.15E", which yields 16
significant digits. IEEE-754 doubles need up to 17 to round-trip, so any
value requiring the 17th digit was canonicalized to a string that parses
back to a *different* double -- a direct violation of RFC 8785 §3.2.2.3,
which requires the ECMAScript shortest form. For example 0.1 + 0.2 became
"0.3". Large integers were also coerced through the 16-digit float path
and lost precision (e.g. 295147905179352830000 -> ...800000).

Take the shortest digit sequence from Float#to_s (which already produces
the shortest round-tripping representation) instead, and normalize it into
the single-leading-digit form the existing exponent logic expects.

The number_spec cases previously commented out as "Outside Ruby Range"
were never outside Ruby's range -- they failed only because of the 16-digit
limit. They are restored and pass, along with new round-trip regressions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant