Skip to content

fix: make ContiguousSet and InsertionOrderSet IntersectWith/SymmetricExceptWith linear [patch] - #110

Merged
matt-edmondson merged 2 commits into
mainfrom
perf/83-set-compaction
Oct 7, 2026
Merged

matt-edmondson merged 2 commits into
mainfrom
perf/83-set-compaction

Conversation

@matt-edmondson

Copy link
Copy Markdown
Contributor

Fixes #83

What was wrong

ContiguousSet<T>.IntersectWith and SymmetricExceptWith called the public Remove(item) for each element they dropped. Each call rescans items from index 0 and shifts the tail, so dropping most of a set costs O(n²). InsertionOrderSet<T> had the same shape, with one List.RemoveAt per dropped element.

Change

  • ContiguousSet: a private RemoveWhere makes one compaction pass. It keeps a write index, removes dropped items from uniquenessSet, clears the vacated tail when SlotClearing.IsNeeded<T>() says to, and bumps version once, only if something was removed. Both operations use it.
  • InsertionOrderSet: List.RemoveAll with the same predicates.
  • Both are now O(n + m) and keep the surviving elements in insertion order. Items found only in other are still appended afterwards, as before.

Tests

  • *_RunsInLinearTime: n = 100k for ContiguousSet and n = 200k for InsertionOrderSet, each with a 1.5 s budget.
    • Against the old code in a Debug build they fail at 33 s and 32 s (ContiguousSet) and 15 s and 14 s (InsertionOrderSet).
    • With the fix, all four run in about a second combined, including test host startup.
  • New order and consistency tests:
    • Survivors keep insertion order, and items that were only in other are appended after them.
    • Contains and Add agree with the new contents.
    • An enumerator open during the operation is invalidated.
  • Full suite: 447/447 pass on net10.0. The library builds clean on every target.

This is a separate change from #109 (#78), which touches AsSpan in the same file. The two changes do not overlap.

🤖 Generated with Claude Code

https://claude.ai/code/session_01J6q8NVYVFUeBVAH6tJCHnU


Generated by Claude Code

claude added 2 commits October 7, 2026 10:30
…ExceptWith linear [patch]

Both operations dropped elements one at a time. ContiguousSet called Remove,
which rescans the array from index 0 and shifts the tail per element, and
InsertionOrderSet called List.RemoveAt per element. Dropping most of a set was
O(n²): about 2.3 s (Release) or 23 s (Debug) to intersect a 100k-element
ContiguousSet with an empty sequence.

ContiguousSet now drops elements in a single compaction pass that clears the
vacated slots and bumps the version once. InsertionOrderSet uses List.RemoveAll.
Both are O(n + m) and keep the surviving elements in order.

Fixes #83

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J6q8NVYVFUeBVAH6tJCHnU
# Conflicts:
#	Containers.Test/ContiguousSetTests.cs
@sonarqubecloud

sonarqubecloud Bot commented Oct 7, 2026

Copy link
Copy Markdown

@matt-edmondson
matt-edmondson merged commit dce4967 into main Oct 7, 2026
14 checks passed
@matt-edmondson
matt-edmondson deleted the perf/83-set-compaction branch October 7, 2026 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ContiguousSet.IntersectWith / SymmetricExceptWith are O(n²): intersecting a 100k-element set with an empty sequence takes ~2.3 s

2 participants