ERS: sort the hash lists once instead of inserting into a LinkedList by index

[dejmekradim]

Fixes #2456.

SortedHashList and SortedIndexedHashList keep their hashes in a LinkedList and find each insertion point by walking it with get(index). LinkedList.get(i) is O(i), so one add is O(n²) and building a list of n hashes is O(n³). Both lists are on the evidence-record generation path — SortedIndexedHashList from ERSArchiveTimeStampGenerator.getPartialHashtrees(), SortedHashList from BinaryTreeRootCalculator.computeRootHash() — so the cost lands on generateTimeStampRequest and generateArchiveTimeStamps.

Both classes expose only size(), getFirst() and toList(), so this collects into an ArrayList and sorts once, lazily, in the accessors.

The output order does not change. The old add inserted each hash after every element comparing <= to it, which is exactly where a stable sort of the insertion sequence puts it, and Collections.sort is stable. Checked two ways:

• testSortedHashListOrder (added) reproduces the original insertion algorithm in the test and asserts the list matches it element by element over 1,200 pseudo-random values including duplicates and arrays sharing a prefix with a longer one.
• Building the root over 2,000 data objects gives 810060f1eb145d7aa90c07d9c8970364c4f7c62cafd45b144f4a60d6a140b5c5 both before and after the change.

getFirst() still throws NoSuchElementException on an empty list, as LinkedList.getFirst() did.

Measured on JDK 21, timing generateTimeStampRequest over n ERSByteData objects:

data objects	before	after
2,000	7.0 s	0.27 s
10,000	347 s	0.67 s
100,000	~93 h (extrapolated)	1.5 s

Tests added to ERSTest:

• testSortedHashListOrder — the order-preservation check described above.
• testLargeDataObjectSet — a reduced hash tree over 2,000 data objects, reaching both sorted lists, asserting the root does not depend on the order the objects were added in. This took about seven seconds before the change and a quarter of a second after it, so it also serves as the regression guard.

A note on how the tests were run: ./gradlew :pkix:test does not build on my machine, for a reason unrelated to this change — :prov:compileJava25Java fails with error: release version 25 not supported, as I have JDK 21. I therefore compiled ERSTest directly with javac against the 1.80 jars and ran it both ways. Both added tests pass against stock 1.80 as well as against the patched classes (9.9 s stock, 1.5 s patched); testSortedHashListOrder passing on the unpatched code is the point of it. I would appreciate a CI run on a box with a JDK 25 toolchain.

Disclosure under CONTRIBUTING.md: this contribution was prepared with the assistance of a large-language-model-based tool. I have reviewed and understood the change, have run the tests, and have verified the before/after output equivalence independently by A/B-running the patched classes against the released bcpkix-jdk18on 1.80 jar.

[dejmekradim]

A correction and an addition on the timings, so the numbers in #2456 and in the description above can be read against each other.

They were measured under different machine load, and they disagree by about 3×. The table in #2456 (1,000: 0.36 s / 2,000: 2.5 s / 4,000: 25 s / 10,000: 347 s) comes from an earlier benchmark run; the "before" column in the description here was measured later, directly against the released bcpkix-jdk18on 1.80 jar, on a 4-core box that turned out to have two other JVMs pinned at ~95% CPU throughout. So the same 2,000-object measurement appears as 2.5 s in the issue and 7.0 s here. Re-running 10,000 objects under that same load gave 1,088 s, against the 347 s in the issue.

Both series are consistent with O(n³), independently: 2,000 → 10,000 is 5× the input, so cubic predicts 125×.

series	2,000	predicted at 10,000	measured at 10,000
issue #2456	2.5 s	312 s	347 s
description above	7.0 s	875 s	1,088 s

The absolute values are therefore only as good as the machine they were taken on, and the lower series understates the cost rather than exaggerating it. The ratio between before and after is the part that does not depend on the box.

Separately, a stronger equivalence check than the one in the description. The root over 10,000 data objects is byte-identical before and after the change:

aecfa9bce2999402e0ac2124ebca2bdfad0671721ad706a41c71450588f9e3ea

stock 1.80 in 1,088 s, patched in 0.67 s, same digest, and the same root again for the input in reverse order.

[dghgit]

Thanks for the PR! This is now in https://www.bouncycastle.org/betas let us know how it goes.

[dejmekradim]

Tested against bcpkix-jdk18on-1.87-SNAPSHOT from the betas page — thank you. Running the prebuilt beta jars also sidesteps the JDK 25 toolchain problem I hit with :pkix:test, so this is the real artifact rather than classes I compiled myself.

Output is unchanged. Our suite regenerates RFC 4998 roots and complete evidence records and compares them byte for byte against vectors frozen from 1.80: 374 comparisons — 8 leaf counts up to 257, every record at each, plus whole .ers files for n = 1, 3 and 5 — all match on the beta. Driving 1.83 and the beta over the same input with the same timestamp token, the digest over every record byte is identical at 2,000 leaves (165cd1e871fef37ad1d8ff8a326a7c7d82b5e4d30eebeaa40de4adf8e941e0ab) and at 4,000 (7282e793c150b951f9173e7265b28772f771a4c13377d32a555c4b47c37fd68b). A further 58 differential assertions pass, covering the shapes the sort could get wrong: already-sorted and reverse-sorted input, hashes differing only in the last byte, the all-zero and all-0xff ends of the comparator's range, runs of equal hashes straddling a split point, and data groups.

Performance, JDK 21, as root build (generateTimeStampRequest) / generateArchiveTimeStamps:

leaves	1.83	1.87-SNAPSHOT
1,000	0.50 s / 0.46 s	0.06 s / 0.27 s
2,000	2.42 s / 2.43 s	0.04 s / 0.41 s
4,000	19.93 s / 20.28 s	0.08 s / 0.67 s
10,000	—	0.08 s / 0.86 s
100,000	—	0.27 s / 3.82 s

Both columns matter because the two lists sit on different paths — SortedHashList under the root build, SortedIndexedHashList under generateArchiveTimeStamps — and both are now flat. One run each, same machine, load average 1.1–1.7 throughout. These are not comparable with the table in the description above: that harness timed ERSByteData objects, this one feeds stored hashes through a one-hash ERSData, so the two series should not be read against each other.

Good to ship from our side.
Thank you.