uucore: make Range::merge linear instead of quadratic - #14359
Conversation
|
GNU testsuite comparison: |
|
any reason why codspeed don't detect the improvements? |
|
The dedup is Indeed better, as you don't shift the queue every time You're doing a sort which is n*logarithmic to begin with. @sylvestre TLDR : the code is still better, but not as much as the clanker said |
| while j < ranges.len() && ranges[j].low <= ranges[i].high { | ||
| let j_high = ranges.remove(j).high; | ||
| ranges[i].high = max(ranges[i].high, j_high); | ||
| ranges.dedup_by(|a, b| { |
There was a problem hiding this comment.
pub fn merge(mut ranges: Vec<Self>) -> Vec<Self> {
ranges.sort_unstable_by_key(|r| r.low);
[...]
}Is better for two reasons :
We don't need a stable sort because we merge values, unstable sort is always more efficient.
Secondly, because we use the derived Ord trait, we compare both ends when we only need compare the low one.
Deepseek found another optimization.
This one converts the loop from O(n^2) to O(n) for ranges.
I ran a few options for this PR, including one that was smaller in size by 96 bytes, but this is faster overall.
LLM generated below here:
Range::merge(src/uucore/src/lib/features/ranges.rs) merged overlappingranges with
ranges.remove(j)inside awhileloop. BecauseVec::removeshifts the tail, heavily-overlapping range lists were O(n²). Only caller is
cut(viaRange::from_list).Replaced with a single
Vec::dedup_bypass that extends the bucket'shighon overlap. Output is unchanged — still sorted, disjoint, and adjacent ranges
are not merged. The only subtlety is that
dedup_by(a, b)passesa= newelement,
b= bucket, so the closure must extendb.high(the kept element),not
a.high(the dropped one).Measurement (same harness, release)
Both versions measured in one harness, same input, realistic overlapping ranges:
The O(n) pass is dramatically faster at 30k+ ranges, but real
cutinvocationsuse a handful of fields (argv caps ~30k, and a typical
-flist is 1-100), sothe merge is sub-microsecond, once-at-startup work there. The value of this
change is not a user-visible speedup: it removes the O(n²) blowup (a large
overlapping field list would otherwise hang
cut) and shrinks the binary by256 B.
dedup_bywas also the smallest/fastest of the four O(n) variants A/B'd(baseline, extra-
Vec, in-placeswap,dedup_by).dedup_bywas also the fastest of the four O(n) variants A/B'd (baseline,extra-
Vec, in-placeswap,dedup_by) in every all-overlap case; the inplace variants are not measurably faster and compile to a larger binary.
Verification
cargo fmtclean,cargo clippy --release --bin coreutilsclean (0 warnings)