Skip to content

perf(join): fall back to chained build on pathological key duplication#259

Merged
singaraiona merged 10 commits into
masterfrom
perf/join-dup-build-fallback
Jun 13, 2026
Merged

perf(join): fall back to chained build on pathological key duplication#259
singaraiona merged 10 commits into
masterfrom
perf/join-dup-build-fallback

Conversation

@singaraiona

Copy link
Copy Markdown
Collaborator

Summary

Fixes the O(dup²) build pathology in the radix-parallel join at its source. The per-partition open-addressing hash build is row-granular, so a key with dup duplicate rows forms a dup-length linear-probe run — inserting the k-th scans k slots → quadratic build (a 10M single-key build took ~9.8s). The build now counts same-hash (per-key) duplicates during insert and, when one key exceeds RADIX_DUP_RUN_MAX (512), abandons the radix attempt and re-runs the whole join through the existing chained-HT path (O(1) insert / O(matches) probe — already the trusted reference for small joins and OOM recovery).

This is sub-project 4, piece 2. It complements the merged build-side selection (PR #251): that dodges the quadratic for INNER joins by building the smaller side; this fix covers LEFT/FULL (which can't swap) and any forced-build-duplicated-side case. ANTI joins use a separate exec path and are unaffected.

Perf (gate verdict: WIN)

  • Catastrophic (one giant build key): ~16× — INNER-no-swap and LEFT both ~2200ms → ~135ms.
  • Moderate (~100 rows/key): neutral, no trip — the same-hash counter measures true per-key duplication, immune to the dense-cluster collision-merge that a total-run-length signal (and lowering the load factor) could not separate.
  • Near-unique + ~10/key: neutral — the per-insert branchless count + single post-loop check costs nothing (a loop-cloning codegen trap from reading the knob in the hot loop was found and avoided).

Full data in `bench/bottleneck/join_dup_fallback_compare.md`.

Test Plan

  • Differential (`test_join_buildside.c`): auto-fallback == forced-chained-oracle multiset for INNER/LEFT/FULL; trip-boundary, not-sticky, no-trip control, force-on-ordinary
  • Correctness inherited from the chained path; no partial radix output leaks on a trip (cleanup runs before consolidation)
  • Full suite green under ASan+UBSan (3451/3453, 2 pre-existing skips); both abort points (mid-build goto, top-of-fn force bail) exercised
  • Release build warning-free; bench binary sanitizer-free

@singaraiona singaraiona merged commit 6f78153 into master Jun 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant