Skip to content

Measure the 2.6.0 performance pair at the last kernel change, the 2662nd #2060

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Rafael-SOWNet merged 1 commit into from
Oct 22, 2026
Merged

Measure the 2.6.0 performance pair at the last kernel change, the 2662nd #2060

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 49 additions & 47 deletions Sources/AngouriMath/Docs/WhatsNew/version_performance_control.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,48 +180,48 @@ honest reason -- a compiled delegate over `Complex` has nothing to put on the he

---

## The 1957th and the 2549th, measured together on one machine — the pair for 2.6.0
## The 1957th and the 2662nd, measured together on one machine — the pair for 2.6.0

The pair the release checklist owes: `v2.5.0`
([`ab7a8bd6`](https://github.com/asc-community/AngouriMath/commit/ab7a8bd6b92d9ae99b3e23e51b10dca5c0e8fe11), the 1957th)
re-measured beside
[`001b1caa`](https://github.com/asc-community/AngouriMath/commit/001b1caa694f1dd8476c3a54d93d10d7a7fcfe4d), the
2549th, which is the last change to the kernel before the release. Both columns come from one run of
`Sources/Utils/benchmark_key_commits.sh`, on 2026-10-18, on one machine with nothing else on it. The period is the
whole of 2.6.0, 592 commits.
[`784f69d5`](https://github.com/asc-community/AngouriMath/commit/784f69d5b07d2f0c153c67791468b3754f2e3250), the 2662nd, which is the last change to
the kernel before the release. Both columns come from one run of `Sources/Utils/benchmark_key_commits.sh`, on
2026-10-22, on one machine with nothing else on it. The period is the whole of 2.6.0, 705 commits.

The `v2.5.0` column agrees with the 1954th column of the 2.5.0 pair, measured on 2026-09-09 three commits earlier.
Every row the gate measures agrees to within 0.02%, and five agree to the byte, `SimplifyEasy` among them.
`CompileEasy` and `CompileHard` differ by 0.30% and 0.19%. The gate leaves those two ungated because their
allocation includes building a delegate.
It replaces the pair measured on 2026-10-18 against the 2549th, which was then the last change to the kernel; 113
commits came after it. The `v2.5.0` column agrees with that run's on every row the gate measures, to within 0.02%, and
`Derivate` and `SimplifyEasy` agree to the byte. `CompileEasy` reads 11,003 here and 10,970 there, 0.30% apart. The
gate leaves it and `CompileHard` ungated because their allocation includes building a delegate.

### Allocation

| benchmark | v2.5.0 | 2549th | change |
|---|--:|--:|--:|
| `CompileEasy` | 10,970 | 11,003 | +0.30% |
| `CompileHard` | 20,434 | 20,434 | 0.00% |
| `Derivate` | 53,239 | 56,190 | **+5.54%** |
| `EvalPolynomialFresh` | 650,888 | 12,536 | **−98.07%** |
| `EvalPolynomialFresh15Digits` | 265,816 | 12,064 | **−95.46%** |
| `EvalTranscendentalFresh` | 5,668,395 | 34,760 | **−99.39%** |
| `EvalTrig` | 1,341,457 | 69,944 | **−94.79%** |
| `EvalTrigPrecise` | 12,742,285 | 336,552 | **−97.36%** |
| `ParseEasy` | 18,261 | 18,421 | +0.88% |
| `ParseHard` | 3,584,817 | 60,103 | **−98.32%** |
| `SimplifyEasy` | 79,282 | 78,978 | −0.38% |
| `SimplifyHard` | 331,710,136 | 176,205,784 | **−46.88%** |
| `SolveEasy` | 8,865,872 | 454,021 | **−94.88%** |
| `SolveEasyMedium` | 65,648 | 66,744 | **+1.67%** |
| `SolveHard` | 11,884,272 | 10,001,832 | **−15.84%** |
| `SolveMedium` | 455,893 | 459,134 | +0.71% |
| `SolveMediumHard` | 1,449,394 | 1,411,524 | −2.61% |
| benchmark | v2.5.0 | 2662nd | change | since the 2549th |
|---|--:|--:|--:|--:|
| `CompileEasy` | 11,003 | 11,003 | 0.00% | 0.00% |
| `CompileHard` | 20,434 | 20,737 | +1.48% | +1.48% |
| `Derivate` | 53,239 | 56,247 | **+5.65%** | +0.10% |
| `EvalPolynomialFresh` | 650,888 | 12,536 | **−98.07%** | 0.00% |
| `EvalPolynomialFresh15Digits` | 265,816 | 12,064 | **−95.46%** | 0.00% |
| `EvalTranscendentalFresh` | 5,668,395 | 34,760 | **−99.39%** | 0.00% |
| `EvalTrig` | 1,341,457 | 69,944 | **−94.79%** | 0.00% |
| `EvalTrigPrecise` | 12,742,285 | 336,552 | **−97.36%** | 0.00% |
| `ParseEasy` | 18,258 | 18,400 | +0.78% | −0.11% |
| `ParseHard` | 3,584,841 | 60,023 | **−98.33%** | −0.13% |
| `SimplifyEasy` | 79,282 | 79,682 | +0.50% | +0.89% |
| `SimplifyHard` | 331,710,008 | 157,898,664 | **−52.40%** | **−10.39%** |
| `SolveEasy` | 8,865,936 | 453,396 | **−94.89%** | −0.14% |
| `SolveEasyMedium` | 65,650 | 66,664 | **+1.54%** | −0.12% |
| `SolveHard` | 11,884,016 | 9,998,784 | **−15.86%** | −0.03% |
| `SolveMedium` | 455,925 | 458,910 | +0.65% | −0.05% |
| `SolveMediumHard` | 1,449,394 | 1,412,412 | −2.55% | +0.06% |

Bytes allocated per call. `EvalEasy`, `RunEasy`, `RunMedium` and `RunHard` allocate nothing in both columns and are
left out.
left out. *Since the 2549th* compares with the 2549th column of the run of 2026-10-18.

- **The evaluation rows** are the numerical-evaluation work of the 2050th to the 2077th, recorded below.
- **`ParseHard`** is the 2203rd, the parser predicting in SLL first.
- **`SimplifyHard`** fell a further 10.4% after the 2549th. That is not attributed to any section of this file.

### The steps compose, and that was checked

Expand All @@ -231,12 +231,12 @@ left out.
- −41.4% and −20.2% at the 2054th to the 2075th;
- −42.1% at the 2077th.

Composed, they predict −94.9%, and the end-to-end measurement is −94.88%.
Composed, they predict −94.9%, and the end-to-end measurement is −94.89%.

`SimplifyHard`'s recorded steps predict about −44.5%, and the measurement is −46.88%. The 2.4 points between them
are not attributed to any section of this file.
`SimplifyHard`'s recorded steps predict about −44.5%. It measured −46.88% at the 2549th and −52.40% here, and neither
gap is attributed.

### `Derivate` went up 5.5%, in steps none of which the gate could see
### `Derivate` went up 5.65%, in steps none of which the gate could see

`Derivate` differentiates a parsed constant, so no parser change reaches it. Its gate baseline moved five times
across the release, each move within the gate's 3%:
Expand All @@ -249,7 +249,8 @@ across the release, each move within the gate's 3%:
| 2117th | 54,567 | +1.9% |
| 2130th | 55,095 | +1.0% |
| 2203rd | 55,783 | +1.2% |
| 2549th, measured here | 56,190 | +0.7% |
| 2549th, the run of 2026-10-18 | 56,190 | +0.7% |
| 2662nd, measured here | 56,247 | +0.1% |

A baseline is re-recorded by the pull request that moves it, so each step covers every commit since the step before it,
not one change. The largest step, +1,039 bytes, is the 2117th,
Expand All @@ -258,27 +259,28 @@ a cached list of their operands, and so makes each of those nodes larger. That i
**no step is bisected**, so the increase is recorded here as unattributed. This file has one other case of steps each free by the gate summing to a real cost, the
rule-set exchange's +13%. This is the same pattern at a smaller size, and it is why the pair is measured end to end.

`SolveEasyMedium` +1.67%, `ParseEasy` +0.88% and `SolveMedium` +0.71% are below the gate's resolution for a single step
and are not attributed either.
`SolveEasyMedium` +1.54%, `ParseEasy` +0.78%, `SolveMedium` +0.65% and `SimplifyEasy` +0.50% are below the gate's
resolution for a single step and are not attributed either.

### Timings

These moved well outside this machine's run-to-run spread and agree in sign with the allocation beside them:

| benchmark | change |
|---|--:|
| `EvalTranscendentalFresh` | −99.50% |
| `EvalPolynomialFresh` | −98.81% |
| `EvalTranscendentalFresh` | −99.51% |
| `EvalPolynomialFresh` | −98.83% |
| `ParseHard` | −98.54% |
| `EvalTrigPrecise` | −98.20% |
| `ParseHard` | −98.17% |
| `SolveEasy` | −95.81% |
| `EvalTrig` | −95.47% |
| `EvalPolynomialFresh15Digits` | −94.63% |
| `SimplifyHard` | −43.90% |
| `SolveHard` | −37.69% |

`Derivate` was +5.15% (13.7 → 14.5 µs), in line with its allocation. The smaller rows from the same run are not worth
reading and are not quoted.
| `SolveEasy` | −96.62% |
| `EvalTrig` | −95.53% |
| `EvalPolynomialFresh15Digits` | −94.85% |
| `SimplifyHard` | −56.99% |
| `SolveHard` | −39.96% |

`Derivate` was +1.66% (14.5 → 14.7 µs). `SimplifyEasy` was −14.9% and `ParseEasy` −25.9% with their allocation flat or
up. One run does not say whether that is the code or the machine, so they are not read as either. The smaller rows from the same run are not
worth reading and are not quoted.

## The 1845th and the 1954th, measured together on one machine — the pair for 2.5.0

Expand Down
Loading