From 01c478b2c2fef8293ca8b2ecf877b68af2293c9c Mon Sep 17 00:00:00 2001 From: tannevaled Date: Sun, 1 Nov 2026 09:38:37 +0100 Subject: [PATCH] docs: sync the mirrored pages to v0.30.0 (scalar draws in random) Re-run scripts/sync-mirrors.sh at v0.30.0 and set params.ndarray.version. random.md gains the allocation-free scalar draws; the other mirrored pages pick up the source changes since the previous sync. The hextra pin is unchanged. Co-Authored-By: Claude Opus 5.5 --- content/en/_index.md | 2 +- content/en/dtypes.md | 2 +- content/en/linalg.md | 4 ++-- content/en/performance.md | 23 ++++++++++++++++++++++- content/en/random.md | 33 +++++++++++++++++++++++++++++---- content/en/roadmap.md | 4 ++-- hugo.yaml | 2 +- 7 files changed, 58 insertions(+), 12 deletions(-) diff --git a/content/en/_index.md b/content/en/_index.md index f6584be..badfd2c 100644 --- a/content/en/_index.md +++ b/content/en/_index.md @@ -46,7 +46,7 @@ WebAssembly, with nothing to install — build a pipeline step by step (creation | Manipulation | `Flatten`, `ExpandDims`, `Squeeze`, `Concatenate`, `Stack`, `VStack`, `HStack` | | Linear algebra | `MatMul`, `Dot`, `Inner`, `Outer`; package `linalg` ([page](linalg.md)): `Solve`, `Inv`, `Det`, `Slogdet`, `Cholesky`, `QR`, `SVD`, `Eigh`, `Eig`, `Lstsq`, `Pinv`, `MatrixRank`, `Norm`, `Cond`, `MatrixPower`, `MultiDot` | | Fourier transforms | package `fft`: `numpy.fft` on arrays (`FFT`/`RFFT`/`FFT2`/`FFTN`/… with `n`/`s`, `axes`, `norm`), over [go-fft](https://github.com/go-fft/fft) | -| Random numbers | package `random` ([page](random.md)): PCG64, PCG64DXSM, MT19937, Philox, SFC64; `Generator` (`DefaultRNG`, 40 samplers) and the legacy `RandomState` (`Seed`, `Rand`) | +| Random numbers | package `random` ([page](random.md)): PCG64, PCG64DXSM, MT19937, Philox, SFC64; `Generator` (`DefaultRNG`, 40 samplers) and the legacy `RandomState` (`Seed`, `Rand`); allocation-free scalar draws for solvers (`Float64`, `FillUniform`, `IntN`, `FillIntN`, `ShuffleInt64`) | | Notebooks & storage | `Repr` (character for character what NumPy 2.5.3's `repr` prints, checked against NumPy's own test strings), `Display` (text and HTML), `MarshalBinary`/`UnmarshalBinary` (the same bytes on every architecture) | | Memory reuse | `NewWorkspace`, `Use`, `Reset`, `Detach` | diff --git a/content/en/dtypes.md b/content/en/dtypes.md index 91c63f9..84c689d 100644 --- a/content/en/dtypes.md +++ b/content/en/dtypes.md @@ -8,7 +8,7 @@ description: "Why the dtype is a run-time property of one Array type, how arrays tags: [dtypes, numpy] --- -> Mirrored from [`docs/design-dtypes.md`](https://github.com/go-ndarray/ndarray/blob/v0.29.1/docs/design-dtypes.md) at v0.29.1. +> Mirrored from [`docs/design-dtypes.md`](https://github.com/go-ndarray/ndarray/blob/v0.30.0/docs/design-dtypes.md) at v0.30.0. > Status: implemented (Phase 2). Supersedes decision 4 of > [plan-ndarray.md](roadmap.md) ("`float64` first"). diff --git a/content/en/linalg.md b/content/en/linalg.md index 2190095..f0804f3 100644 --- a/content/en/linalg.md +++ b/content/en/linalg.md @@ -8,9 +8,9 @@ description: "Solve, Inv, Det, Cholesky, QR, SVD, Eig and the rest of numpy.lina tags: [linalg, numpy] --- -> Mirrored from [`docs/linalg.md`](https://github.com/go-ndarray/ndarray/blob/v0.29.1/docs/linalg.md) at v0.29.1. +> Mirrored from [`docs/linalg.md`](https://github.com/go-ndarray/ndarray/blob/v0.30.0/docs/linalg.md) at v0.30.0. -Package [`linalg`](https://pkg.go.dev/github.com/go-ndarray/ndarray@v0.29.1/linalg) gives ndarray arrays the functions of +Package [`linalg`](https://pkg.go.dev/github.com/go-ndarray/ndarray@v0.30.0/linalg) gives ndarray arrays the functions of `numpy.linalg`, written in Go without cgo or assembly of its own: the matrix products it relies on are the ndarray package's packed SIMD GEMM. diff --git a/content/en/performance.md b/content/en/performance.md index 91d7731..fc0cd5a 100644 --- a/content/en/performance.md +++ b/content/en/performance.md @@ -1383,7 +1383,28 @@ takes 8.02 µs against NumPy's 8.14: **parity**. float64 takes 16.3 against 15.8 µs (1.03×). **Open defect: products are still up to 1.2× slower than OpenBLAS on one -core** at 256² and 1024² on N1, and float32 1024² on Zen 3 is at 1.04×. The exception is float64 at 1024² on Zen 3, where we are 3% faster. +core** at 256² and 1024² on N1, and float32 1024² on Zen 3 is at 1.04×. + +**Open defect: on a busy N1 host we slow down and OpenBLAS does not** +(measured 2026-10-31). The host was at load 21 on 64 cores. Back to back on +the same pinned core, NumPy's times were the usual ones, but ours were about +a third slower than on a quiet host. So the ratio to NumPy grew to: + +| N1, load 21, ours / NumPy | 64² | 256² | 1024² | +|---|---:|---:|---:| +| float64 | 1.27 | 1.40 | 1.54 | +| float32 | 1.39 | 1.25 | 1.27 | + +Ruled out on that host, by A/B: + +- a regression: v0.26.1, v0.27.0 and v0.29.1 are equally slow there; +- reading A in place: v0.26.1, which packs A, is no faster; +- the block sizes: OpenBLAS's `param.h` values (KC 320, MC 240) change + nothing, and NC = 128 or 256 (a B block well inside the 1 MiB L2) moves + float64 by 4–9% but float32 1024² by +4%. + +The cause is not found. The quiet-host ratios above stand for a quiet host, +and this one for a contended one. The exception is float64 at 1024² on Zen 3, where we are 3% faster. The float32 kernels for riscv64 and loong64 are still to write. **Open: the allocating operations.** `Add` is still 1.5–3.2× slower than diff --git a/content/en/random.md b/content/en/random.md index e2737ad..73f4d53 100644 --- a/content/en/random.md +++ b/content/en/random.md @@ -8,9 +8,9 @@ description: "numpy.random with NumPy's numbers for the same seed: the bit gener tags: [random, numpy] --- -> Mirrored from [`docs/random.md`](https://github.com/go-ndarray/ndarray/blob/v0.29.1/docs/random.md) at v0.29.1. +> Mirrored from [`docs/random.md`](https://github.com/go-ndarray/ndarray/blob/v0.30.0/docs/random.md) at v0.30.0. -Package [`random`](https://pkg.go.dev/github.com/go-ndarray/ndarray@v0.29.1/random) is `numpy.random` over go-ndarray arrays. Its goal +Package [`random`](https://pkg.go.dev/github.com/go-ndarray/ndarray@v0.30.0/random) is `numpy.random` over go-ndarray arrays. Its goal is migration: a notebook moved from NumPy to Go keeps its numbers for the same seed. @@ -24,7 +24,7 @@ u, _ := random.Rand(3) // np.random.rand(3) ``` `Example`, `ExampleSeed` and `ExampleGenerator_ChoiceN` in -[`example_test.go`](https://github.com/go-ndarray/ndarray/blob/v0.29.1/random/example_test.go) print NumPy's own numbers. +[`example_test.go`](https://github.com/go-ndarray/ndarray/blob/v0.30.0/random/example_test.go) print NumPy's own numbers. ## What is implemented @@ -46,6 +46,30 @@ u, _ := random.Rand(3) // np.random.rand(3) | `rand`, `randn`, `randint`, `random_sample`, `normal`, `uniform`, `choice`, `shuffle`, `permutation`, `binomial`, `poisson`, `standard_normal`, `standard_exponential`, `exponential`, `standard_gamma`, `gamma`, `chisquare`, `lognormal`, `multinomial`, `bytes` | `RandomState` methods of the same names | | `np.random.seed`, `rand`, `randn`, `randint`, `random_sample`, `normal`, `uniform`, `choice`, `shuffle`, `permutation`, `binomial`, `poisson` | package functions over a shared `RandomState` behind a mutex | +### Allocation-free scalar draws + +Every sampler above returns a `*ndarray.Array`. A solver that needs a few draws +per candidate (differential evolution, basin hopping) would spend its time +allocating, so `Generator` and `RandomState` both have five methods that draw +into caller storage or return a Go scalar, allocate nothing, and consume +exactly the bit-generator output NumPy consumes for the same call: + +| NumPy | `Generator` / `RandomState` | +|---|---| +| `random()` / `random_sample()`, `size=None` | `Float64() float64` | +| `uniform(low, high, size=len(dst))` | `FillUniform(dst []float64, low, high float64) error` (Generator: error when `high-low` is negative, NaN or infinite; legacy: only NaN or infinite, `high < low` is accepted) | +| `integers(0, high)` / `randint(0, high)`, `size=None`, int64 | `IntN(high int64) int64` (Lemire for `Generator`, masked rejection for `RandomState`; panics when `high <= 0`, like `math/rand`) | +| `integers(0, high, size=len(dst))` / `randint(…)` | `FillIntN(dst []int64, high int64) error` (error when `high <= 0` and `dst` is not empty; nothing is drawn) | +| `shuffle(x)` of a 1-D int64 array | `ShuffleInt64(d []int64)` (Fisher-Yates from the end, `random_interval`) | + +A failing call draws nothing, as in NumPy. A mixed sequence of 32-bit and +64-bit draws keeps PCG64's buffered half-word exactly as NumPy does. +`testdata/scalar.py` and `TestScalarDrawsMatchNumPy` compare 7 296 calls on 192 +scripts (48 seeds × Generator over PCG64 and MT19937, `RandomState` over MT19937 +and PCG64) with NumPy 2.5.3, errors included; `TestScalarAllocations` asserts +zero allocations with `testing.AllocsPerRun`; `FuzzScalarDraws` throws NaN, +±Inf, negative and zero bounds, `MaxInt64` and `MinInt64` at them. + The size comes last; no size gives a 0-d array holding what NumPy returns as a scalar (`size=None`), which draws exactly the same values. `ChoiceOptions`' zero value is NumPy's defaults (`NoReplace`, `NoShuffle` are the negations). @@ -118,7 +142,7 @@ random arguments. Used as-is, that made 3% of `np.random.randn` values and 10% of `lognormal` values one ULP off. So `random` rounds `log`, `log1p`, `exp`, `expm1` and `pow` **correctly** -([`libm.go`](https://github.com/go-ndarray/ndarray/blob/v0.29.1/random/libm.go)): a fast evaluation to about 2⁻⁶⁸ with Ziv's +([`libm.go`](https://github.com/go-ndarray/ndarray/blob/v0.30.0/random/libm.go)): a fast evaluation to about 2⁻⁶⁸ with Ziv's rounding test, and a double-double fallback (about 2⁻¹⁰⁰) for the one argument in ~2¹⁵ that needs it. Checked against mpmath at 300 bits on 63,000 arguments (0 misrounded; the committed `libm.json` keeps 8,400), and the fast paths @@ -223,6 +247,7 @@ interface. The samplers that call `pow`/`exp` per draw cost about what uv run --python cpython-3.12-macos-x86_64 --with numpy==2.5.3 python random/testdata/cases.py > random/testdata/cases.json uv run --with mpmath python random/testdata/libm.py > random/testdata/libm.json uv run --with mpmath python random/testdata/libm_tables.py > random/zz_libm_tables.go +GONDARRAY_REGEN=1 go test ./random -run TestRegenerateScalarFixture # scalar.json: x86-64 NumPy in an amd64 container python3 random/testdata/zig2go.py /numpy/random/src/distributions/ziggurat_constants.h random/zz_ziggurat.go ``` diff --git a/content/en/roadmap.md b/content/en/roadmap.md index a128a41..a68a440 100644 --- a/content/en/roadmap.md +++ b/content/en/roadmap.md @@ -8,7 +8,7 @@ description: "The plan for go-ndarray and what ships today." tags: [roadmap] --- -> Mirrored from [`docs/plan-ndarray.md`](https://github.com/go-ndarray/ndarray/blob/v0.29.1/docs/plan-ndarray.md) at v0.29.1. +> Mirrored from [`docs/plan-ndarray.md`](https://github.com/go-ndarray/ndarray/blob/v0.30.0/docs/plan-ndarray.md) at v0.30.0. > Goal: a **pure-Go (CGO=0)** NumPy-style N-dimensional array library, as a > **standalone, reusable** module with correct scalar kernels now and a clean @@ -112,7 +112,7 @@ suite (plus the per-arch CI jobs already wired in `.github/workflows/ci.yml`). - **Performance — DONE (beats NumPy on its core ops; matmul at tuned-BLAS parity).** Several levers behind the kernel seam, measured honestly vs NumPy 2.2.4 (see [docs/perf.md](performance.md) and the repo's - [BENCHMARKS.md](https://github.com/go-ndarray/ndarray/blob/v0.29.1/BENCHMARKS.md)): + [BENCHMARKS.md](https://github.com/go-ndarray/ndarray/blob/v0.30.0/BENCHMARKS.md)): - **Multicore.** `Add/Sub/Mul/Div/Map`, `Sum/Max/Min`, the axis reductions, `Concatenate`/`Stack`, and the GEMM fan out across `GOMAXPROCS` above a size threshold (small arrays stay serial). Plus an allocation fix: the diff --git a/hugo.yaml b/hugo.yaml index 02834e7..3be1cf1 100644 --- a/hugo.yaml +++ b/hugo.yaml @@ -100,7 +100,7 @@ params: # The ndarray release these pages describe. Its minor version is the # documentation's version label (v0.9.1 -> 0.9): the CI reads it here. ndarray: - version: "v0.29.1" + version: "v0.30.0" navbar: displayTitle: true