Repository navigation
perf: SIMD inner product (AVX2/FMA, NEON) for Dot, MatVec and Dot1DP - #16
Merged
Merged
Conversation
dotRange, under Dot (1-D), MatVec and Dot1DP, was a four-chain scalar loop (gc does not vectorise it); on one Zen 3 core a 2^20 dot took 464 us, 2x NumPy's single thread. Two go-asmgen kernels replace it: - amd64 dotFMA: four YMM accumulators, b read straight from memory by VFMADD231PD; gated on the FMA probe like the GEMM kernel (the flag is renamed useFMA, since it now covers both). - arm64 dotNEON: eight D2 accumulators for the four FP pipes of an Apple core, folded with FMLA against 1.0. The other targets keep dotRange. Zen 3, interleaved with main: Dot 2^20 1 core 464 -> 229 us (NumPy 231) 16 cores 159 -> 99 us MatVec 1024^2 1 core 482 -> 134 us (NumPy 153) 16 cores 165 -> 89 us TestDotSIMD: exact on integer data for every length 0..70 (main loop, folds, tail), within n*eps*sum|a_i*b_i| on random data, NaN and Inf propagate; on amd64 it also runs with FMA forced off. Passes on a real Zen 3 (FMA path). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tannevaled
force-pushed
the
perf/simd-dot
branch
from
October 4, 2026 14:19
ec6b217 to
f975594
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
dotRange, under Dot (1-D), MatVec and Dot1DP, was a four-chain scalar loop
(gc does not vectorise it); on one Zen 3 core a 2^20 dot took 464 us, 2x
NumPy's single thread. Two go-asmgen kernels replace it:
VFMADD231PD; gated on the FMA probe like the GEMM kernel (the flag is
renamed useFMA, since it now covers both).
core, folded with FMLA against 1.0.
The other targets keep dotRange. Zen 3, interleaved with main:
Dot 2^20 1 core 464 -> 229 us (NumPy 231) 16 cores 159 -> 99 us
MatVec 1024^2 1 core 482 -> 134 us (NumPy 153) 16 cores 165 -> 89 us
TestDotSIMD: exact on integer data for every length 0..70 (main loop,
folds, tail), within nepssum|a_i*b_i| on random data, NaN and Inf
propagate; on amd64 it also runs with FMA forced off. Passes on a real
Zen 3 (FMA path).
🤖 Generated with Claude Code