fix: PartitionedTopKRank counts tie rows twice in output_rows - #25535
Open
jayzhan211 wants to merge 1 commit into
Open
jayzhan211 wants to merge 1 commit into
jayzhan211 wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #25535 +/- ##
==========================================
- Coverage 82.38% 82.38% -0.01%
==========================================
Files 1138 1138
Lines 434491 434519 +28
Branches 434491 434519 +28
==========================================
+ Hits 357969 357981 +12
- Misses 54875 54884 +9
- Partials 21647 21654 +7 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
For a
RANK()window Top-N (PartitionedTopKExec: fn=rank), theoutput_rowsandoutput_batchesmetrics shown byEXPLAIN ANALYZEare too high whenever rows tie at the K-th value.PartitionedTopKRank::emitcallsrecord_outputon every tie batch before pushing it into theBatchCoalescer, and then callsrecord_outputagain on each completed batch coming out of that coalescer — which already contains those tie rows. Heap rows are only counted at the second site, so each tie row is counted twice and each tie batch adds a phantom output batch.Example: K = 2, one partition with values
5, 5, 10, 5. Three rows are emitted in one batch, but the metrics reportoutput_rows=4,output_batches=2.PartitionedTopK(ROW_NUMBER) andPartitionedTopKDenseRankonly record at the coalescer output and are not affected.What changes are included in this PR?
Remove the extra
record_outputcall on tie batches so every emitted row is counted once, at the coalescer output, as in the other two operators. Query results are unchanged; only the metrics are corrected.What is the testing strategy for this PR?
New unit test
test_partitioned_topk_rank_output_rows_counts_ties_once, which emits heap rows plus a boundary tie and assertsoutput_rows/output_batchesequal what the stream actually produced. It fails onmainwith(2, 4)vs(1, 3).Are there any user-facing changes?
output_rows/output_batchesreported forPartitionedTopKExecwithfn=rankare now accurate when ties are present. No API changes.