Describe the bug
Native Top-K queries may fail when all output columns are also sort keys.
In this case, the sort keys are stored separately and the data batch contains no columns. The sorted indices are truncated by the Top-K limit, but the zero-column batch still keeps the original input row count.
This row-count mismatch can cause the merge cursor to read past the retained sort-key rows.
To Reproduce
Run a Top-K query whose output contains only the sort column:
spark
.range(10)
.repartition(1)
.select("id")
.orderBy("id")
.limit(6)
.collect()The query fails in native SortExec with:
unsafe precondition(s) violated:
slice::get_unchecked requires that the index is within the slice
Expected behavior
The query should return the correct result.
Screenshots
Additional context
Describe the bug
Native Top-K queries may fail when all output columns are also sort keys.
In this case, the sort keys are stored separately and the data batch contains no columns. The sorted indices are truncated by the Top-K limit, but the zero-column batch still keeps the original input row count.
This row-count mismatch can cause the merge cursor to read past the retained sort-key rows.
To Reproduce
Run a Top-K query whose output contains only the sort column:
The query fails in native
SortExecwith:Expected behavior
The query should return the correct result.
Screenshots
Additional context