Skip to content

perf: stream scalar operands; fill by doubling copy (#118) - #121

Merged
tannevaled merged 3 commits into
mainfrom
perf/scalar-operand-fill
Nov 15, 2026
Merged

tannevaled merged 3 commits into
mainfrom
perf/scalar-operand-fill

Conversation

@tannevaled

@tannevaled tannevaled commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

First step on the result-allocation cause of #118. Two of the slow rows were not the allocation itself.

Scalar operands no longer get copied to the result's size.

  • AddScalar … DivScalar, x op Scalar(v), and a 1×1 array against a matrix all built an 800 kB copy of the scalar (on 10⁵ elements) before calling the kernel.
  • binOp and binOpInto (so AddInto … DivInto too) now recognise a one-element operand against a contiguous Float64 array of the result's shape (scalarOperand). A strided operand, such as a transpose, still takes the general path. kernels.ScalarP streams a block of 512 copies through the same SIMD kernel, as RepeatP already does for repeated rows.
  • Every element is computed by the same instruction, so the results are bit-identical.

Fills use doubling copy. Full, Ones and FullOf filled one element at a time; that includes every dtype through the generated fillStore. kernels.Fill, generic over the element type, writes one element and then doubles the filled prefix with copy. Only Full was timed. Ones calls Full. FullOf was not measured.

Measured on cfarm420 (EPYC 7773X, Zen 3): one pinned core, GOMAXPROCS=1, load 2.2–2.4, three interleaved rounds, 10⁵ float64.

before after
AddScalar 183–203 µs, 1.6 MB 95–110 µs, 0.8 MB
Full 131–135 µs 84–104 µs

What remains is Go's own cost. A CPU profile of a bare make([]float64, 1e5) shows about 60% garbage-collector marking (assists in the allocating goroutine) and about 23% zeroing. docs/perf.md gains a "Since the table" section that says so and points to *Into and Workspace. The published table stays the measured v0.41.0 run.

Tests

  • TestScalarPSameBits: same bits as the materialised broadcast, for both operand orders, at every block boundary and across ParThreshold.
  • TestScalarOperandPaths: AddScalar…DivScalar, scalar-first Sub and Div, a one-element view at offset 4, and 1×1 against 7×143.
  • TestScalarOperandPaths also covers scalar-first DivInto, and SubInto with out aliasing x.
  • TestScalarOperandAllocs: AddScalar allocates at most the result plus one block per call, and AddInto with a scalar at most one block.
  • Mutation: bypassing the *Into path fails the second bound (803 344 B).
  • TestFill.
  • Mutations checked: swapping the operand order fails TestScalarOperandPaths by assertion, and bypassing the path fails TestScalarOperandAllocs (1 606 316 B > 808 192).

alloc_bench_test.go keeps the benchmarks that take the result's cost apart.

Refs #118

🤖 Generated with Claude Code

tannevaled and others added 3 commits November 15, 2026 10:32
A binary operation with a one-element operand (AddScalar ... DivScalar,
x op Scalar(v), a 1x1 array against a matrix) materialised the scalar to
the result's size before calling the kernel: two 800 kB buffers per call
on 10^5 elements. binOp now recognises the case (scalarOperand) and
kernels.ScalarP streams a block of 512 copies through the same SIMD
kernel, like RepeatP does for repeated rows, so each element is computed
by the same instruction and the result is the same bits.

Full, Ones and FullOf (every dtype, through the generated fillStore)
filled one element at a time. kernels.Fill, generic over the element
type, writes one element and doubles the filled prefix with copy.

cfarm420 (EPYC 7773X, Zen 3), one pinned core, GOMAXPROCS=1, load
2.2-2.4, three interleaved rounds, 10^5 float64:
  AddScalar  183-203 us -> 95-110 us,  1.6 MB -> 0.8 MB per call
  Full       131-135 us -> 84-104 us

alloc_bench_test.go keeps the benchmarks that take the result's cost
apart. A CPU profile of a bare make([]float64, 1e5) shows ~60% GC
marking (assists) and ~23% zeroing; docs/perf.md says so and points to
the *Into forms and Workspace.

Refs #118

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… was measured (#118)

Review of #121: binOpInto still copied a one-element operand to full
size through operandFor; it now takes the scalarOperand path, out may
alias an input as before. docs/perf.md names the cases the path covers
(contiguous Float64 operand of the result's shape) and says FullOf was
not timed.

Refs #118

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #121: the aliasing case (out == x in AddInto) was tested only
below ParThreshold. TestScalarPSameBits now also runs every kernel and
order with dst == full, at every block boundary and at ParThreshold + 3,
where the blocks are split across workers.

Refs #118

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 8370f0f into main Nov 15, 2026
13 checks passed
@tannevaled
tannevaled deleted the perf/scalar-operand-fill branch November 15, 2026 09:45
tannevaled added a commit to go-ndarray/go-ndarray.github.io that referenced this pull request Nov 15, 2026
The library card names v0.41.3, and the playground builds against it
(scalar operands streamed instead of copied, fills by doubling copy:
go-ndarray/ndarray#121). The single-core claims on the page are those of
the measured v0.41.0 table and do not change.

Refs go-ndarray/ndarray#118

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
tannevaled added a commit to go-ndarray/docs that referenced this pull request Nov 15, 2026
Every mirrored page and hugo.yaml now point at v0.41.3. Performance mirrors
docs/perf.md's new "Since the table" section (go-ndarray/ndarray#121):
scalar operands are streamed instead of copied (AddScalar 183-203 us ->
95-110 us, half the memory), fills use doubling copy (Full 131-135 us ->
84-104 us), and the rest of the allocation cost is Go's GC and zeroing.
The other mirrored sources are unchanged between v0.41.1 and v0.41.3.

Refs go-ndarray/ndarray#118

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant