Repository navigation
fix: build OrderedSet/OrderedCollection/OrderedMap in O(n log n) and stop OrderedSet set operations going quadratic [patch] - #111
Merged
Conversation
…stop OrderedSet set operations going quadratic [patch] The bulk constructors filled the backing list one binary-search insert at a time, shifting the list tail on every insert, so building from n unsorted or reverse-sorted items was O(n^2). OrderedSet.ToComparerSet goes through that constructor, which made IsSubsetOf, IsProperSubsetOf, IsProperSupersetOf, SetEquals, IntersectWith and SymmetricExceptWith quadratic in the size of the argument: over a minute for a 1,000,000-element descending array. The constructors now copy the input and run a stable merge sort (new internal StableSort) with the container's comparer: - OrderedSet drops adjacent equal elements, keeping the first, as Add does - OrderedCollection keeps equal elements in their original order, as Add does - OrderedMap throws the same ArgumentException Add throws when two keys compare equal OrderedSet set operations also reuse an OrderedSet argument that has the same comparer instead of copying it, IntersectWith compacts with a single RemoveAll, and SymmetricExceptWith merges the two sorted lists in one linear pass. Fixes #82 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K6bDUGMFsAQews3TnA2jXr
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Fixes #82
Summary
The sorted containers' bulk constructors filled the backing
List<T>one binary-searchInsertat a time, so building from n unsorted or reverse-sorted items was O(n²).OrderedSet.ToComparerSetgoes through that constructor, which madeIsSubsetOf,IsProperSubsetOf,IsProperSupersetOf,SetEquals,IntersectWithandSymmetricExceptWithquadratic in the size of the argument.StableSort(internal): a top-down merge sort with an insertion-sort cutoff. It takes from the left half on ties, so it is stable, and it skips the merge when two halves are already in order, so sorted input (such asClone) costs O(n).OrderedSet(IEnumerable<T>[, comparer]): copies, stable-sorts, then drops adjacent equal elements and keeps the first, which is what repeatedAddcalls did.OrderedCollection(IEnumerable<T>[, comparer]): copies and stable-sorts, so equal elements keep their input order, asAdd(upper-bound insert) does.OrderedMap(IDictionary[, comparer]): copies entries, checks for null keys, stable-sorts by key, and throws the sameArgumentException(same message) thatAddthrows when two keys compare equal under the comparer.OrderedSetset operations:ToComparerSetnow reuses anOrderedSet<T>argument that has the same comparer instead of copying it.IntersectWithcompacts in one pass withRemoveAllinstead of callingRemoveAtonce per removed element.SymmetricExceptWithnow does a single linear merge of the two sorted lists. The old code made repeatedRemoveAt,RemoveandAddcalls, each of which is O(n). A call on the set itself still empties it, and anOrderedSetargument is no longer modified.IsSupersetOfandOverlapsare unchanged.#80 is not addressed beyond what this naturally covers.
Testing
New tests:
Stopwatchwith a 5 s budget on 1,000,000 elements:OrderedSet,OrderedCollectionandOrderedMapOrderedSet.IsSubsetOf,SetEquals,IntersectWithandSymmetricExceptWithagainst a descending arrayOrderedSetkeeps the first occurrence of duplicates, using 1,000 items with a key comparer so the merge path runs.OrderedSethonours a case-insensitive comparer and a reverse comparer.Addon random input.OrderedMapthrowsArgumentExceptionon keys that compare equal, matchingAdd, and honours a custom comparer.OrderedCollectionkeeps equal elements in insertion order, using 1,000 items so the merge path runs.SymmetricExceptWithworks on the set itself and on anOrderedSetwith the same comparer.Revert proof: I stashed only the three production files (
OrderedSet.cs,OrderedCollection.cs,OrderedMap.cs) and ran the new tests. All 7 performance tests failed their budget. TheOrderedSet/OrderedCollectioncases took about 73–98 s and theOrderedMapconstructor took about 174 s, against a 5 s budget. The semantics tests passed on both paths, as expected, because they check behaviour that already existed. With the fix restored, all of them pass, and the whole suite runs in about 4 s.Full suite:
dotnet testreports 470/470 passed.dotnet build -c Releasereports 0 warnings and 0 errors.🤖 Generated with Claude Code
https://claude.ai/code/session_01K6bDUGMFsAQews3TnA2jXr
Generated by Claude Code