Skip to content

Preserve the spec exponent of exact integer powers (#161) - #162

Open
chiliec wants to merge 2 commits into
cockroachdb:masterfrom
chiliec:fix-pow-exact-integer-exponent
Open

chiliec wants to merge 2 commits into
cockroachdb:masterfrom
chiliec:fix-pow-exact-integer-exponent

Conversation

@chiliec

@chiliec chiliec commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

What

Context.Pow with an exact negative integer exponent no longer preserves the exponent that the General Decimal Arithmetic power operation specifies.

At precision 34, Pow(2, -1) returns 0.5000000000000000000000000000000000 (exponent -34) and reports Rounded; Pow(10, -2) returns 0.0100...0. apd v3.0.0 returned 0.5 and 0.01 with no flags. The GDA power operation says an exact integer-power result carries the exponent it would have under repeated multiplication (with the reciprocal of the base for a negative exponent), e.g. power('2','-3') ==> '0.125' (http://speleotrove.com/decimal/daops.html#refpower). Python's decimal returns 0.5 at the same precision.

The regression came in with 04d1483 ("apd: don't strip trailing zeros in Quo and Sqrt", v3.1.0): integerPower divides one by the computed power with Quo, which now keeps precision digits, so Pow inherited a division convention the power spec does not share. The GDA suite in gda_test.go compares with Cmp, which ignores the exponent, so powx101 power 2 -1 -> 0.5 still passed.

Reported in #161 (found via cue-lang/cue, which uses apd for arbitrary-precision numbers). This PR addresses the exact-exponent part only, not the separate precision-0 question in that issue.

Fix

In integerPower, after computing the reciprocal for a negative exponent, Reduce the quotient when it is exact (!Inexact). An exact reciprocal of an integer power has no genuine trailing zeros, so this strips only the precision padding and restores the spec exponent; the Rounded flag is no longer raised for exact results. Inexact reciprocals (e.g. 3**-1) are left untouched.

Tests

Added TestPowExactInteger in decimal_test.go asserting the text, exponent, and condition flags for exact positive and negative integer powers plus one inexact case.

Validation (go1.23.2, linux/amd64):

$ go test -run TestPowExactInteger -v ./
--- PASS: TestPowExactInteger (0.00s)
    --- PASS: TestPowExactInteger/2**-1
    --- PASS: TestPowExactInteger/2**-3
    --- PASS: TestPowExactInteger/10**-2
    --- PASS: TestPowExactInteger/5**-2
    --- PASS: TestPowExactInteger/2**3
    --- PASS: TestPowExactInteger/10**3
    --- PASS: TestPowExactInteger/3**-1
PASS

$ go test ./
ok  	github.com/cockroachdb/apd/v3	3.372s

The full suite (including the GDA gda_test.go power tests) passes on the branch.

Context.Pow computes negative integer powers as 1/x**|y| via Quo, which
since v3.1.0 keeps precision trailing zeros, so Pow(2, -1) at precision 34
returned 0.5000...0 (exponent -34, Rounded) instead of 0.5. The General
Decimal Arithmetic power operation specifies that an exact integer-power
result carries the exponent it would have under repeated multiplication,
e.g. power('2','-3') ==> '0.125'.

Reduce the quotient when the reciprocal is exact so the exact result keeps
its natural exponent and no longer reports Rounded. Inexact reciprocals are
left untouched.

@mw5h mw5h left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this at 40e2d30 against the GDA power text, Python's decimal, and the official power.decTest vectors. The fix is correct; one flag leak should be fixed before merge and the test could be broader. Details:

Why Reduce gives exactly the spec exponent. The reciprocal 1/x of x = X·10^e can never carry a trailing zero (c·X·10^(f+e) = 1 is impossible for integers with f > -e), and powers of a reduced coefficient never gain one, so the fully reduced form is the "repeated multiplication of the reciprocal" exponent the spec describes. This holds for bases with trailing zeros too, and I confirmed it against Python on 112 cases at precisions 34/20/10/3/2/1/0 and in decimal128, Emax-999 and system-limit contexts: 2.50**-1 = 0.4, 0.1**-1 = 1E+1, 1E+2**-2 = 0.0001, 1.0**-3 = 1, -2**-3 = -0.125, 2.5E+500**-2 -> 1.6E-1001 Subnormal (GDA powx424, exact text), positive powers unchanged. !ed.Flags.Inexact() reads the accumulated flags, so inexact positive powers (2**-120, 5**-120) are correctly left unreduced, and 2**-40 at prec 10 is inexact and rounds correctly. TestPowExactInteger fails on master (4/7) and passes here. With gda_test.go temporarily made exponent-sensitive for power (compare with CmpTotal), master fails 62 vectors, this branch 19, and none of the 19 is PR-only (all are pre-existing non-integer-exponent padding cases like 1 ** 12.3 -> 1.00). Side benefits: Pow(10,-99990) no longer errors "exponent out of range", and precision 0 gives 0.5 instead of 0.50000000000.

1. Rounded still leaks when x**|y| exceeded the internal precision by zeros only

BaseContext.WithPrecision(34).Pow(d, 10, -50) returns 1E-50 with rounded. Python gives 1E-50 with no flags, and GDA powx121 (10 ** -77), powx122, powx2317 (10 ** -383) list no conditions. Cause: in the squaring loop, Mul rounds 10^50 (51 digits) to the 44-digit internal precision, dropping only zeros, which sets Rounded without Inexact. The !Inexact gate correctly lets Reduce run, but the stale Rounded is returned. On master this was invisible because every exact reciprocal already reported Rounded from the final c.round. So the PR body's "the Rounded flag is no longer raised for exact results" is true for |y| below roughly the precision and false beyond it, and callers gating on res.Rounded() see a flag that depends on |y|.

Fix I verified:

if ed.Err() == nil && !ed.Flags.Inexact() {
    z.Reduce(z)
    ed.Flags &^= Rounded
}

Rounded && !Inexact from the loop means only zero digits were dropped and the value is exact; the later c.round in Pow still adds Rounded/Inexact legitimately if the reduced value exceeds c.Precision. With this, the strict GDA harness goes 26 → 23 failures (exactly powx121/122/2317 fixed, nothing new), the full suite passes, and all 112 battery cases match Python's flags including subnormal ones (d128 10**-6144 -> [subnormal] instead of [subnormal, rounded]). Please add {"10", "-50", "1E-50", -50, 0} to the test; it fails on the current branch.

2. Test coverage

TestPowExactInteger only has integer bases with exponent 0 and small |y|. The cases the fix actually depends on are missing: trailing-zero / non-zero-exponent bases (0.5**-2 = 4, 0.1**-1 = 1E+1, 2.50**-1 = 0.4, 1E+2**-2 = 0.0001), a negative base, an exact result exceeding the context precision but not the internal one (5**-60 = 1.152921504606846976E-42), precision 0, and a subnormal/Emax context. The more durable option is making gda_test.go compare power results with CmpTotal (the tc.Operation != "power" exemption at the Cmp call) and allow-listing the 19 pre-existing non-integer cases; that would have caught the v3.1.0 regression in the first place.

3. Notes, not blockers

  • Overflowing negative powers now report overflow, inexact without rounded (e.g. GDA powx421 2.5E-501 ** -2); master had rounded only by accident from the padding c.round dropped. apd's Mul/Quo overflow paths already omit Rounded, so this makes Pow consistent with the library and exposes a pre-existing apd-wide deviation from the spec.
  • Relationship to #165 ("Follow the ideal exponent in Quo"): master + #165 without this PR produces byte-identical output on my whole battery, including the Rounded leak above (so item 1 applies to #165's approach as well). For an exact 1/z the ideal exponent is always the fully reduced one, so if #165 lands this line becomes a harmless no-op, and if this lands first #165 needs no change. This is the narrower, lower-risk fix and stands on its own.
  • CockroachDB: power()/^ output for exact negative integer exponents loses its padding at precision 20 (0.50000000000000000000 → 0.5). PG prints 0.5000000000000000, matching neither. I found no negative-exponent expectations in logic_test/decimal; other logic test files were not checked. Worth grepping before bumping apd.

@chiliec

chiliec commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review. Pushed 8bdfe31:

  1. Applied your fix: ed.Flags &^= Rounded after Reduce, with a comment explaining why it's safe. Added {"10", "-50", "1E-50", -50, 0}; it fails without the line (got condition rounded).
  2. Broadened TestPowExactInteger with a per-case precision and added 0.5**-2, 0.1**-1, 2.50**-1, 1E+2**-2, 1.0**-3, -2**-3, 5**-60 at precision 34 (exact) and 10 (inexact), and precision 0. Expected values, exponents and conditions all come from Python's decimal.

I left the gda_test.go CmpTotal change for a follow-up, since it needs an allow-list of the 19 pre-existing cases and is a separate decision. Happy to do it here if you prefer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants