Skip to content

perf: float32 Exp in AVX2, correctly rounded and as fast as NumPy - #90

Merged
tannevaled merged 4 commits into
mainfrom
perf/float32-exp-simd
Nov 2, 2026
Merged

tannevaled merged 4 commits into
mainfrom
perf/float32-exp-simd

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

float32 Exp is now correctly rounded and at NumPy's speed or better on amd64. NumPy's own source puts its float32 exp at 2.52 ULP.

The algorithm: Arm optimized-routines' correctly rounded vector expf (math/aarch64/advsimd/cr_expf.c + cr_expf_utils.h, MIT).

  • expCR transcribes it one operation per line, every vfmaq as math.FMA.
  • The 2^(i/64) table is re-derived at 300 bits (TestExpCRTable).
  • Its float64 error is 1.006 ULP, so the exhaustive certifier uses a 4-ULP margin for it.

Proof (all 2³² float32 inputs): 7 decided by the 300-bit reference, 0 incorrectly rounded, with no exception table needed.

The kernel: exp32CRAVX2 (go-asmgen, AVX2+FMA) does the same operations on 8 floats at a time.

  • It widens with VCVTPS2PD, gathers the table with VPGATHERQQ, takes the exponent from z<<46, and rounds once with VCVTPD2PS.
  • Inputs are clamped to ±512, which gives the same float32 result, and NaN lanes are restored from the input.
  • It is bit-identical to expCR on 100 000 inputs, every special value, and every length 0–70, and it is fenced.
  • On Zen 3 the exhaustive test, which runs it, finds 0 incorrectly rounded of 4 294 967 296.

Every architecture now runs expCR (in Go where there is no kernel yet), so the bits are the same everywhere.

float32 Exp, one Zen 3 core, abbench 8 rounds v0.32.0 this PR NumPy 2.5.4
16 Ki 90 µs 26.9 µs 27.1 µs
1 Mi 5.8 ms 1.48 ms 1.66 ms

Throughput is +235% and +289% (p = 0.000). The host was at load 59 on 128 cores; NumPy ran in the same session on the same pinned core.

float32 Log behaves exactly as before, moved to its own log32; the old via64 exp path is gone. Still open: float32 Log at 3× NumPy, and Exp kernels for the other architectures. The audit entry is in SECURITY.md.

🤖 Generated with Claude Code

tannevaled and others added 4 commits November 2, 2026 11:44
…very float32

expCR transcribes the double-precision exp of Arm optimized-routines'
cr_expf (math/aarch64/advsimd/cr_expf.c, MIT) one operation per line,
every vfmaq as math.FMA, so a SIMD kernel can repeat it bit for bit:
x/ln2 rounded to 1/64 by a shift, a 64-entry table of 2^(i/64)
(re-derived at 300 bits by TestExpCRTable), and a degree-5 polynomial.

Its float64 error is up to 1.006 ULP, so the exhaustive certifier
takes a per-function margin (4 ULP here; 2 for the 0.504-ULP kernels).
Over all 2^32 float32 inputs: 7 decided by the 300-bit reference,
0 incorrectly rounded. No exception table needed. This is the target
for the SIMD float32 exp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every architecture's float32 Exp now runs expCR, the transcription of
Arm's correctly rounded expf (proven over all 2^32 inputs in the
previous commit). On amd64 with AVX2+FMA, exp32CRAVX2 (go-asmgen)
repeats its operations eight floats at a time: VCVTPS2PD, the FMA
sequence, VPGATHERQQ for the 2^(i/64) table, the exponent from z<<46,
then one VCVTPD2PS. Inputs are clamped to +-512 (same float32 result)
and NaN lanes restored. The kernel is bit-identical to expCR (100 000
inputs, specials, lengths 0-70) and fenced. On Zen 3 the exhaustive
test, which runs it, finds 0 incorrectly rounded of 2^32.

Zen 3, one core, abbench 8 rounds: 16 Ki 90 -> 26.9 us (NumPy 27.1),
1 Mi 5.8 -> 1.48 ms (NumPy 1.66). Parity, then 11% faster than NumPy,
which is at 2.52 ULP; we are correctly rounded.

Log moves to its own log32 (unchanged behaviour); the old via64 exp
path is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_test.go

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 2155b94 into main Nov 2, 2026
12 checks passed
@tannevaled
tannevaled deleted the perf/float32-exp-simd branch November 2, 2026 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant