Skip to content

perf(gemm): pack full B panels with whole-row array moves - #75

Merged
tannevaled merged 1 commit into
mainfrom
perf/gemm-pack
Oct 26, 2026
Merged

tannevaled merged 1 commit into
mainfrom
perf/gemm-pack

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

packB copied each row run of a full B panel element by element; a pprof of a 64² float32 product put it at 21% of the time. A full panel now moves each run with one fixed-size array assignment (8 or 16 elements). The compiler lowers it to vector moves, with no runtime call, and the conversions are bounds-checked.

MatMul vs v0.26.0, abbench 10 rounds, 1 core 64² 256² 1024²
Zen 3 float32 −11.6% −7.1% −1.7%
Zen 3 float64 −4.4% −2.3% ~
N1 float32 −16.4% −8.9% ~
N1 float64 −9.5% ~ ~

TestPackRowsWhole covers both widths, checks that nothing is written past the last run, and covers the declined width. No new memory access: the conversions are checked against the same slices the element loop indexed.

🤖 Generated with Claude Code

packB copied each NR-element row run of a full panel element by
element. A pprof of a 64² float32 product put it at 21% of the time.
A full panel (nr == NR, unit column stride) now moves each run with one
fixed-size array assignment, *(*[NR]T)(dst) = *(*[NR]T)(src), for the
two widths in use (8 and 16). The compiler lowers it to vector moves
with no runtime call. The conversions are bounds-checked like any
slice-to-array conversion.

abbench, 10 rounds, one core, against v0.26.0:
  Zen 3  float32 -11.6/-7.1/-1.7% (64²/256²/1024²), float64 -4.4/-2.3/~
  N1     float32 -16.4/-8.9/~,                     float64 -9.5/~/~

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 679c055 into main Oct 26, 2026
12 checks passed
@tannevaled
tannevaled deleted the perf/gemm-pack branch October 26, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant