Skip to content

perf: stream a repeated broadcast operand instead of materialising it - #14

Merged
tannevaled merged 1 commit into
mainfrom
perf/broadcast-rows
Oct 4, 2026
Merged

tannevaled merged 1 commit into
mainfrom
perf/broadcast-rows

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

M + row (1024x1024 plus 1x1024) lost to NumPy 0.26x on a Zen 3 even in a
Workspace, because the broadcast path copied the row out to a full 8 MiB
matrix before adding. When one operand has the full result shape and the
other, leading 1s dropped, is a suffix of it, the second just repeats in
blocks of its own length: binOp now hands the live slices to the SIMD
kernel block by block (kernels.RepeatP), split across cores. Blocks under
64 elements keep the old path.

Zen 3, 16 cores pinned, BroadcastAdd 1024^2 + row:
heap 3.1 ms -> 0.84 ms
Workspace 1.76 ms -> 114-196 us (NumPy 1 thread: 460 us)

The new test checks Sub and Div in both operand orders, matrix+row,
matrix+1-D, 3-D minus 2-D, 3-D plus a (1,1,n) row and a too-short row,
serially and with the block loop forced parallel, against an element by
element oracle; swapping the operands in RepeatP makes it fail.

🤖 Generated with Claude Code

M + row (1024x1024 plus 1x1024) lost to NumPy 0.26x on a Zen 3 even in a
Workspace, because the broadcast path copied the row out to a full 8 MiB
matrix before adding. When one operand has the full result shape and the
other, leading 1s dropped, is a suffix of it, the second just repeats in
blocks of its own length: binOp now hands the live slices to the SIMD
kernel block by block (kernels.RepeatP), split across cores. Blocks under
64 elements keep the old path.

Zen 3, 16 cores pinned, BroadcastAdd 1024^2 + row:
  heap       3.1 ms  -> 0.84 ms
  Workspace  1.76 ms -> 114-196 us   (NumPy 1 thread: 460 us)

The new test checks Sub and Div in both operand orders, matrix+row,
matrix+1-D, 3-D minus 2-D, 3-D plus a (1,1,n) row and a too-short row,
serially and with the block loop forced parallel, against an element by
element oracle; swapping the operands in RepeatP makes it fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tannevaled
tannevaled force-pushed the perf/broadcast-rows branch from dac2121 to d5fc1e3 Compare October 4, 2026 14:07
@tannevaled
tannevaled merged commit ae279b5 into main Oct 4, 2026
12 checks passed
@tannevaled
tannevaled deleted the perf/broadcast-rows branch October 4, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant