Plan V2 random access from known row selections by joseph-isaacs · Pull Request #10625 · vortex-data/vortex

joseph-isaacs · 2026-10-19T22:59:33Z

V2 random access resolves known row selections before allocating projection planners. It skips empty chunks, trims selected spans, and avoids masks and filtering for contiguous selections. A selection-bound physical plan limits segment discovery to reachable children; unchanged masks retain their derived indices across fields.

Whole dictionary dependencies retain their original plans and per-scan sharing. Removing the eager whole-file compatibility walk keeps unused layout children lazy; unsupported plans fail when reached. List layouts remain unsupported. The pipeline driver, compiler, and generic Planner trait are unchanged.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

codspeed · 2026-10-19T23:06:46Z

Merging this PR will regress 1 benchmark

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️

13 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

test_scan_filter[int_str]

test_scan[int_str_float]

multipolygon_mixed_validity

test_scan[int_str]

random_i128[0.01]

pcodec_compress_f64

decompress[u16, (4000, 256)]

take_varbin

test_scan_filter[int_float]

decompress[u32, (1000, 256)]

…and 3 more

ℹ️ 40 benchmarks have no flame graph

The flamegraph could not be generated because we failed to resolve the symbols in the profiling trace. This happens when we fail to harvest information about language frames.

test_scan_filter[int]

test_repeated_scan_scalar_at[int_str_float]

test_scan_filter[int_str]

test_repeated_scan_filter[int_str_float]

test_scan_scalar_at[int_str_float]

test_scan_filter_to_arrow_array[int_str_float]

test_scan_scalar_at[int]

test_repeated_scan_scalar_at[int_float]

test_scan[int_str_float]

test_repeated_scan_filter[int]

…and 30 more

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
❌ 1 regressed benchmark
✅ 2404 untouched benchmarks
⏩ 12 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
❌	Simulation	`take[routing/grouped_chunks/primitive/nonnull/chunks=32/indices=1000]`	385 µs	429.4 µs	-10.34%
⚡	Simulation	`chunked_opt_bool_canonical_into[(1000, 10)]`	65.4 µs	55.1 µs	+18.6%
⚡	WallTime	`bitpack_blocked_compress_avx512`	5.6 µs	4.8 µs	+17.19%
⚡	WallTime	`deferred_bool_avx512[i64, PerRowPerRow]`	4.1 µs	3.7 µs	+11.28%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing ji/v2-selection-planning (60b2d43) with develop (2ebe999)²}

12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
No successful run was found on develop (5e365e6) during the generation of this report, so 2ebe999 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

github-actions · 2026-10-19T23:13:53Z

Polar Signals Profiling Results

Latest Run

Status	Commit	Job	Attempt	Link
🟢 Done	`d326929`	random-access-bench	1	Explore Profiling Data

Previous Runs (4)

Status	Commit	Job	Attempt	Link
🟢 Done	`5012ca3`	random-access-bench	1	Explore Profiling Data
🟢 Done	`5012ca3`	compress-bench	1	Explore Profiling Data
🟢 Done	`3cd67a2`	random-access-bench	1	Explore Profiling Data
🟢 Done	`3cd67a2`	compress-bench	1	Explore Profiling Data

Powered by Polar Signals Cloud

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

github-actions · 2026-10-19T23:21:14Z

Benchmarks: Compression 📖

Commits: PR 5012ca31 vs base 61ff28d9

vortex / vortex-file-compressed / ns (0.999x ➖, 0↑ 0↓)

name	ns (PR / base / %diff)
compress time/Arade	768270786 / 758524188 / +1.3%
compress time/Bimbo	3295760967 / 3271273938 / +0.7%
compress time/CMSprovider	2455181381 / 2415200639 / +1.7%
compress time/Euro2016	253683851 / 254412764 / -0.3%
compress time/Food	307533615 / 309889717 / -0.8%
compress time/HashTags	500059604 / 504567333 / -0.9%
compress time/TPC-H l_comment canonical	569706853 / 565803887 / +0.7%
compress time/TPC-H l_comment chunked	570698544 / 565947036 / +0.8%
compress time/taxi	361123588 / 359860443 / +0.4%
compress time/wide table cols=100 chunks=1 rows=1000	4127947 / 4177299 / -1.2%
compress time/wide table cols=100 chunks=50 rows=1000	3985575 / 4222323 / -5.6%
compress time/wide table cols=1000 chunks=1 rows=1000	41675972 / 41049901 / +1.5%
compress time/wide table cols=1000 chunks=50 rows=1000	41761328 / 41957638 / -0.5%
compress time/wide table cols=10000 chunks=1 rows=1000	473650136 / 470827873 / +0.6%
compress time/wide table cols=10000 chunks=50 rows=1000	465507019 / 472878133 / -1.6%
decompress time/Arade	12042667 / 12018830 / +0.2%
decompress time/Bimbo	55576999 / 56279836 / -1.2%
decompress time/CMSprovider	36503983 / 37422648 / -2.5%
decompress time/Euro2016	11702141 / 11816148 / -1.0%
decompress time/Food	3258754 / 3298628 / -1.2%
decompress time/HashTags	34893259 / 33980554 / +2.7%
decompress time/TPC-H l_comment canonical	13348984 / 13338587 / +0.1%
decompress time/TPC-H l_comment chunked	13351732 / 13235338 / +0.9%
decompress time/taxi	8685926 / 8941793 / -2.9%
decompress time/wide table cols=100 chunks=1 rows=1000	1170347 / 1176628 / -0.5%
decompress time/wide table cols=100 chunks=50 rows=1000	1165305 / 1183174 / -1.5%
decompress time/wide table cols=1000 chunks=1 rows=1000	11526336 / 11375090 / +1.3%
decompress time/wide table cols=1000 chunks=50 rows=1000	11547811 / 11370878 / +1.6%
decompress time/wide table cols=10000 chunks=1 rows=1000	126041513 / 123760742 / +1.8%
decompress time/wide table cols=10000 chunks=50 rows=1000	125988537 / 123149143 / +2.3%

vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)

name	bytes (PR / base / %diff)
vortex-file-compressed size/Arade	142213756 / 142213756 / +0.0%
vortex-file-compressed size/Bimbo	458581212 / 458581212 / +0.0%
vortex-file-compressed size/CMSprovider	420794836 / 420794836 / +0.0%
vortex-file-compressed size/Euro2016	164125572 / 164130260 / -0.0%
vortex-file-compressed size/Food	38950752 / 38950752 / +0.0%
vortex-file-compressed size/HashTags	196170516 / 196170516 / +0.0%
vortex-file-compressed size/TPC-H l_comment canonical	178688232 / 178688232 / +0.0%
vortex-file-compressed size/TPC-H l_comment chunked	178688232 / 178688232 / +0.0%
vortex-file-compressed size/taxi	49998092 / 49998092 / +0.0%
vortex-file-compressed size/wide table cols=100 chunks=1 rows=1000	934080 / 934080 / +0.0%
vortex-file-compressed size/wide table cols=100 chunks=50 rows=1000	934080 / 934080 / +0.0%
vortex-file-compressed size/wide table cols=1000 chunks=1 rows=1000	9325680 / 9325680 / +0.0%
vortex-file-compressed size/wide table cols=1000 chunks=50 rows=1000	9325680 / 9325680 / +0.0%
vortex-file-compressed size/wide table cols=10000 chunks=1 rows=1000	93277680 / 93277680 / +0.0%
vortex-file-compressed size/wide table cols=10000 chunks=50 rows=1000	93277680 / 93277680 / +0.0%

vortex / vortex-file-compressed / ratio (0.998x ➖, 0↑ 0↓)

name	ratio (PR / base / %diff)
vortex:parquet-zstd ratio compress time/Arade	1.04566933 / 1.03502475 / +1.0%
vortex:parquet-zstd ratio compress time/Bimbo	1.66481097 / 1.64015989 / +1.5%
vortex:parquet-zstd ratio compress time/CMSprovider	2.31428162 / 2.2780073 / +1.6%
vortex:parquet-zstd ratio compress time/Euro2016	0.393920508 / 0.39467854 / -0.2%
vortex:parquet-zstd ratio compress time/Food	0.961306413 / 0.973404696 / -1.2%
vortex:parquet-zstd ratio compress time/HashTags	1.46282583 / 1.46470403 / -0.1%
vortex:parquet-zstd ratio compress time/TPC-H l_comment canonical	0.536118328 / 0.532902529 / +0.6%
vortex:parquet-zstd ratio compress time/TPC-H l_comment chunked	0.538363826 / 0.534069589 / +0.8%
vortex:parquet-zstd ratio compress time/taxi	1.8747174 / 1.87134793 / +0.2%
vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=1 rows=1000	1.19492388 / 1.18883001 / +0.5%
vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=50 rows=1000	1.10832116 / 1.21911144 / -9.1%
vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=1 rows=1000	1.16867258 / 1.15070208 / +1.6%
vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=50 rows=1000	1.16754198 / 1.16388149 / +0.3%
vortex:parquet-zstd ratio compress time/wide table cols=10000 chunks=1 rows=1000	1.24992685 / 1.26557569 / -1.2%
vortex:parquet-zstd ratio compress time/wide table cols=10000 chunks=50 rows=1000	1.24031736 / 1.26838178 / -2.2%
vortex:parquet-zstd ratio decompress time/Arade	0.245820557 / 0.245570846 / +0.1%
vortex:parquet-zstd ratio decompress time/Bimbo	0.622732001 / 0.63097111 / -1.3%
vortex:parquet-zstd ratio decompress time/CMSprovider	0.356254574 / 0.365313018 / -2.5%
vortex:parquet-zstd ratio decompress time/Euro2016	0.0781139707 / 0.0789361881 / -1.0%
vortex:parquet-zstd ratio decompress time/Food	0.130400876 / 0.13230067 / -1.4%
vortex:parquet-zstd ratio decompress time/HashTags	0.083410812 / 0.0810134218 / +3.0%
vortex:parquet-zstd ratio decompress time/TPC-H l_comment canonical	0.162270276 / 0.162533318 / -0.2%
vortex:parquet-zstd ratio decompress time/TPC-H l_comment chunked	0.162398279 / 0.161570429 / +0.5%
vortex:parquet-zstd ratio decompress time/taxi	0.148405876 / 0.152816464 / -2.9%
vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=1 rows=1000	0.573879832 / 0.576607428 / -0.5%
vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=50 rows=1000	0.56639388 / 0.57788421 / -2.0%
vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=1 rows=1000	0.487500127 / 0.482566342 / +1.0%
vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=50 rows=1000	0.493503241 / 0.483326112 / +2.1%
vortex:parquet-zstd ratio decompress time/wide table cols=10000 chunks=1 rows=1000	0.492064023 / 0.491314093 / +0.2%
vortex:parquet-zstd ratio decompress time/wide table cols=10000 chunks=50 rows=1000	0.49908994 / 0.492123032 / +1.4%
vortex:parquet-zstd size/Arade	0.551185597 / 0.551185597 / -0.0%
vortex:parquet-zstd size/Bimbo	1.19261532 / 1.19261532 / +0.0%
vortex:parquet-zstd size/CMSprovider	1.11649863 / 1.11649863 / -0.0%
vortex:parquet-zstd size/Euro2016	1.33458157 / 1.33461969 / -0.0%
vortex:parquet-zstd size/Food	1.09107276 / 1.09107276 / -0.0%
vortex:parquet-zstd size/HashTags	1.46946859 / 1.46946859 / -0.0%
vortex:parquet-zstd size/TPC-H l_comment canonical	1.12837977 / 1.12837977 / -0.0%
vortex:parquet-zstd size/TPC-H l_comment chunked	1.12837977 / 1.12837977 / -0.0%
vortex:parquet-zstd size/taxi	0.904392267 / 0.904392267 / -0.0%
vortex:parquet-zstd size/wide table cols=100 chunks=1 rows=1000	1.0017975 / 1.0017975 / -0.0%
vortex:parquet-zstd size/wide table cols=100 chunks=50 rows=1000	1.0017975 / 1.0017975 / -0.0%
vortex:parquet-zstd size/wide table cols=1000 chunks=1 rows=1000	1.00017975 / 1.00017975 / -0.0%
vortex:parquet-zstd size/wide table cols=1000 chunks=50 rows=1000	1.00017975 / 1.00017975 / -0.0%
vortex:parquet-zstd size/wide table cols=10000 chunks=1 rows=1000	1.00040408 / 1.00040408 / -0.0%
vortex:parquet-zstd size/wide table cols=10000 chunks=50 rows=1000	1.00040408 / 1.00040408 / -0.0%

vortex / parquet / ns (1.002x ➖, 0↑ 0↓)

name	ns (PR / base / %diff)
parquet_rs-zstd compress time/Arade	734716762 / 732856084 / +0.3%
parquet_rs-zstd compress time/Bimbo	1979660772 / 1994484780 / -0.7%
parquet_rs-zstd compress time/CMSprovider	1060882722 / 1060225153 / +0.1%
parquet_rs-zstd compress time/Euro2016	643997572 / 644607543 / -0.1%
parquet_rs-zstd compress time/Food	319912164 / 318356505 / +0.5%
parquet_rs-zstd compress time/HashTags	341844937 / 344484157 / -0.8%
parquet_rs-zstd compress time/TPC-H l_comment canonical	1062651328 / 1061739917 / +0.1%
parquet_rs-zstd compress time/TPC-H l_comment chunked	1060061089 / 1059687816 / +0.0%
parquet_rs-zstd compress time/taxi	192628280 / 192300126 / +0.2%
parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000	3454569 / 3513790 / -1.7%
parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000	3596047 / 3463443 / +3.8%
parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000	35660948 / 35673787 / -0.0%
parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000	35768588 / 36049751 / -0.8%
parquet_rs-zstd compress time/wide table cols=10000 chunks=1 rows=1000	378942285 / 372026642 / +1.9%
parquet_rs-zstd compress time/wide table cols=10000 chunks=50 rows=1000	375312829 / 372820031 / +0.7%
parquet_rs-zstd decompress time/Arade	48989666 / 48942414 / +0.1%
parquet_rs-zstd decompress time/Bimbo	89247058 / 89195583 / +0.1%
parquet_rs-zstd decompress time/CMSprovider	102466005 / 102439952 / +0.0%
parquet_rs-zstd decompress time/Euro2016	149808554 / 149692407 / +0.1%
parquet_rs-zstd decompress time/Food	24990277 / 24932814 / +0.2%
parquet_rs-zstd decompress time/HashTags	418330168 / 419443510 / -0.3%
parquet_rs-zstd decompress time/TPC-H l_comment canonical	82263889 / 82066786 / +0.2%
parquet_rs-zstd decompress time/TPC-H l_comment chunked	82215970 / 81916834 / +0.4%
parquet_rs-zstd decompress time/taxi	58528181 / 58513283 / +0.0%
parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000	2039359 / 2040605 / -0.1%
parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000	2057411 / 2047424 / +0.5%
parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000	23643760 / 23572075 / +0.3%
parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000	23399666 / 23526306 / -0.5%
parquet_rs-zstd decompress time/wide table cols=10000 chunks=1 rows=1000	256148605 / 251897399 / +1.7%
parquet_rs-zstd decompress time/wide table cols=10000 chunks=50 rows=1000	252436539 / 250240560 / +0.9%

vortex / parquet / bytes (1.000x ➖, 0↑ 0↓)

name	bytes (PR / base / %diff)
parquet size/Arade	258014282 / 258014282 / +0.0%
parquet size/Bimbo	384517292 / 384517292 / +0.0%
parquet size/CMSprovider	376887911 / 376887911 / +0.0%
parquet size/Euro2016	122979049 / 122979049 / +0.0%
parquet size/Food	35699500 / 35699500 / +0.0%
parquet size/HashTags	133497590 / 133497590 / +0.0%
parquet size/TPC-H l_comment canonical	158358238 / 158358238 / +0.0%
parquet size/TPC-H l_comment chunked	158358238 / 158358238 / +0.0%
parquet size/taxi	55283635 / 55283635 / +0.0%
parquet size/wide table cols=100 chunks=1 rows=1000	932404 / 932404 / +0.0%
parquet size/wide table cols=100 chunks=50 rows=1000	932404 / 932404 / +0.0%
parquet size/wide table cols=1000 chunks=1 rows=1000	9324004 / 9324004 / +0.0%
parquet size/wide table cols=1000 chunks=50 rows=1000	9324004 / 9324004 / +0.0%
parquet size/wide table cols=10000 chunks=1 rows=1000	93240004 / 93240004 / +0.0%
parquet size/wide table cols=10000 chunks=50 rows=1000	93240004 / 93240004 / +0.0%

vortex / arrow-ipc / ns (0.994x ➖, 0↑ 0↓)

name	ns (PR / base / %diff)
arrow-ipc compress time/Arade	99839690 / 100215714 / -0.4%
arrow-ipc compress time/Bimbo	621607647 / 635589410 / -2.2%
arrow-ipc compress time/CMSprovider	382820650 / 390948643 / -2.1%
arrow-ipc compress time/Euro2016	38040164 / 38595296 / -1.4%
arrow-ipc compress time/Food	33899558 / 34177711 / -0.8%
arrow-ipc compress time/HashTags	78732320 / 79122787 / -0.5%
arrow-ipc compress time/TPC-H l_comment canonical	940784435 / 940180187 / +0.1%
arrow-ipc compress time/TPC-H l_comment chunked	939789151 / 941274915 / -0.2%
arrow-ipc compress time/taxi	73890779 / 75583954 / -2.2%
arrow-ipc compress time/wide table cols=100 chunks=1 rows=1000	198535 / 204921 / -3.1%
arrow-ipc compress time/wide table cols=100 chunks=50 rows=1000	200741 / 201027 / -0.1%
arrow-ipc compress time/wide table cols=1000 chunks=1 rows=1000	2246616 / 2260609 / -0.6%
arrow-ipc compress time/wide table cols=1000 chunks=50 rows=1000	2215804 / 2218879 / -0.1%
arrow-ipc compress time/wide table cols=10000 chunks=1 rows=1000	25355881 / 25653899 / -1.2%
arrow-ipc compress time/wide table cols=10000 chunks=50 rows=1000	25221315 / 25493022 / -1.1%
arrow-ipc decompress time/Arade	217646901 / 212031427 / +2.6%
arrow-ipc decompress time/Bimbo	411726683 / 413855809 / -0.5%
arrow-ipc decompress time/CMSprovider	678660517 / 679048161 / -0.1%
arrow-ipc decompress time/Euro2016	83875015 / 83994413 / -0.1%
arrow-ipc decompress time/Food	67445624 / 66250501 / +1.8%
arrow-ipc decompress time/HashTags	160835762 / 161450843 / -0.4%
arrow-ipc decompress time/TPC-H l_comment canonical	624570753 / 623843409 / +0.1%
arrow-ipc decompress time/TPC-H l_comment chunked	624019140 / 624218751 / -0.0%
arrow-ipc decompress time/taxi	51099713 / 51288021 / -0.4%
arrow-ipc decompress time/wide table cols=100 chunks=1 rows=1000	320231 / 318607 / +0.5%
arrow-ipc decompress time/wide table cols=100 chunks=50 rows=1000	317030 / 320260 / -1.0%
arrow-ipc decompress time/wide table cols=1000 chunks=1 rows=1000	3469538 / 3583377 / -3.2%
arrow-ipc decompress time/wide table cols=1000 chunks=50 rows=1000	3594473 / 3479159 / +3.3%
arrow-ipc decompress time/wide table cols=10000 chunks=1 rows=1000	35255074 / 36848739 / -4.3%
arrow-ipc decompress time/wide table cols=10000 chunks=50 rows=1000	36274149 / 36411339 / -0.4%

vortex / arrow-ipc / bytes (1.000x ➖, 0↑ 0↓)

name	bytes (PR / base / %diff)
arrow-ipc size/Arade	691765162 / 691765162 / +0.0%
arrow-ipc size/Bimbo	3279590114 / 3279590114 / +0.0%
arrow-ipc size/CMSprovider	2422259682 / 2422259682 / +0.0%
arrow-ipc size/Euro2016	360582666 / 360582666 / +0.0%
arrow-ipc size/Food	255782338 / 255782338 / +0.0%
arrow-ipc size/HashTags	704785810 / 704785810 / +0.0%
arrow-ipc size/TPC-H l_comment canonical	7317023586 / 7317023586 / +0.0%
arrow-ipc size/TPC-H l_comment chunked	7317023586 / 7317023586 / +0.0%
arrow-ipc size/taxi	482329690 / 482329690 / +0.0%
arrow-ipc size/wide table cols=100 chunks=1 rows=1000	1257290 / 1257290 / +0.0%
arrow-ipc size/wide table cols=100 chunks=50 rows=1000	1257290 / 1257290 / +0.0%
arrow-ipc size/wide table cols=1000 chunks=1 rows=1000	12568506 / 12568506 / +0.0%
arrow-ipc size/wide table cols=1000 chunks=50 rows=1000	12568506 / 12568506 / +0.0%
arrow-ipc size/wide table cols=10000 chunks=1 rows=1000	125752506 / 125752506 / +0.0%
arrow-ipc size/wide table cols=10000 chunks=50 rows=1000	125752506 / 125752506 / +0.0%

github-actions · 2026-10-19T23:46:37Z

Benchmarks: Random Access 📖

Commits: PR d3269295 vs base 61ff28d9
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -9.0%
Engines: random-access No clear signal (-9.0%, environment too noisy confidence)
Vortex (geomean): hot 0.929x ➖
Parquet (geomean): hot 1.020x ➖
Shifts: Parquet (control) +2.0% · Median polish -1.0%

How to read Verdict and Engines

Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.

vortex / arrow-ipc / ns (1.048x ➖, 1↑ 2↓)

name	ns (PR / base / %diff)
random-access/arrow-tokio-local-disk	70479 / 67936 / +3.7%
random-access/arrow-tokio-local-disk-footer	166811 / 158515 / +5.2%
random-access/feature-vectors/correlated/arrow-tokio-local-disk	2569457 / 2553640 / +0.6%
random-access/feature-vectors/correlated/arrow-tokio-local-disk-footer	3401191 / 1913523 / +77.7% 🔴
random-access/feature-vectors/uniform/arrow-tokio-local-disk	31405306 / 54103073 / -42.0% 🟢
random-access/feature-vectors/uniform/arrow-tokio-local-disk-footer	31903513 / 29949513 / +6.5%
random-access/nested-lists/correlated/arrow-tokio-local-disk	57523 / 55338 / +3.9%
random-access/nested-lists/correlated/arrow-tokio-local-disk-footer	100990 / 97032 / +4.1%
random-access/nested-lists/uniform/arrow-tokio-local-disk	1036722 / 1025451 / +1.1%
random-access/nested-lists/uniform/arrow-tokio-local-disk-footer	1078018 / 1110731 / -2.9%
random-access/nested-structs/correlated/arrow-tokio-local-disk	52084 / 49037 / +6.2%
random-access/nested-structs/correlated/arrow-tokio-local-disk-footer	93286 / 85395 / +9.2%
random-access/nested-structs/uniform/arrow-tokio-local-disk	903415 / 792307 / +14.0% 🔴
random-access/nested-structs/uniform/arrow-tokio-local-disk-footer	917638 / 837566 / +9.6%
random-access/taxi/correlated/arrow-tokio-local-disk	120863 / 113520 / +6.5%
random-access/taxi/correlated/arrow-tokio-local-disk-footer	218034 / 212011 / +2.8%
random-access/taxi/uniform/arrow-tokio-local-disk	2799878 / 2663156 / +5.1%
random-access/taxi/uniform/arrow-tokio-local-disk-footer	3082513 / 2840547 / +8.5%

random-access / vortex-file-compressed / ns (0.929x ➖, 7↑ 1↓)

name	ns (PR / base / %diff)
random-access/feature-vectors/correlated/vortex-tokio-local-disk	242775 / 234005 / +3.7%
random-access/feature-vectors/correlated/vortex-tokio-local-disk-footer	1135356 / 1199748 / -5.4%
random-access/feature-vectors/uniform/vortex-tokio-local-disk	1025665 / 1370878 / -25.2% 🟢
random-access/feature-vectors/uniform/vortex-tokio-local-disk-footer	2132987 / 2576171 / -17.2% 🟢
random-access/nested-lists/correlated/vortex-tokio-local-disk	164583 / 200355 / -17.9% 🟢
random-access/nested-lists/correlated/vortex-tokio-local-disk-footer	304449 / 384975 / -20.9% 🟢
random-access/nested-lists/uniform/vortex-tokio-local-disk	674495 / 678080 / -0.5%
random-access/nested-lists/uniform/vortex-tokio-local-disk-footer	898668 / 879596 / +2.2%
random-access/nested-structs/correlated/vortex-tokio-local-disk	234332 / 234324 / +0.0%
random-access/nested-structs/correlated/vortex-tokio-local-disk-footer	427816 / 427555 / +0.1%
random-access/nested-structs/uniform/vortex-tokio-local-disk	551198 / 595545 / -7.4%
random-access/nested-structs/uniform/vortex-tokio-local-disk-footer	746466 / 862067 / -13.4% 🟢
random-access/taxi/correlated/vortex-tokio-local-disk	646985 / 596962 / +8.4%
random-access/taxi/correlated/vortex-tokio-local-disk-footer	1210993 / 1184712 / +2.2%
random-access/taxi/uniform/vortex-tokio-local-disk	1823434 / 2472705 / -26.3% 🟢
random-access/taxi/uniform/vortex-tokio-local-disk-footer	2602088 / 3425154 / -24.0% 🟢
random-access/vortex-tokio-local-disk	496032 / 383160 / +29.5% 🔴
random-access/vortex-tokio-local-disk-footer	875604 / 859433 / +1.9%

random-access / parquet / ns (1.020x ➖, 0↑ 1↓)

name	ns (PR / base / %diff)
random-access/feature-vectors/correlated/parquet-tokio-local-disk	112300498405 / 112308985889 / -0.0%
random-access/feature-vectors/correlated/parquet-tokio-local-disk-footer	113085691631 / 112844970780 / +0.2%
random-access/feature-vectors/uniform/parquet-tokio-local-disk	107130283200 / 106964234141 / +0.2%
random-access/feature-vectors/uniform/parquet-tokio-local-disk-footer	155782819430 / 107637654388 / +44.7% 🔴
random-access/nested-lists/correlated/parquet-tokio-local-disk	144331550 / 149164855 / -3.2%
random-access/nested-lists/correlated/parquet-tokio-local-disk-footer	144673068 / 144865077 / -0.1%
random-access/nested-lists/uniform/parquet-tokio-local-disk	146362502 / 144796783 / +1.1%
random-access/nested-lists/uniform/parquet-tokio-local-disk-footer	143918331 / 144804018 / -0.6%
random-access/nested-structs/correlated/parquet-tokio-local-disk	43817257 / 44577966 / -1.7%
random-access/nested-structs/correlated/parquet-tokio-local-disk-footer	43958599 / 44946796 / -2.2%
random-access/nested-structs/uniform/parquet-tokio-local-disk	43820434 / 44603223 / -1.8%
random-access/nested-structs/uniform/parquet-tokio-local-disk-footer	44330196 / 44777426 / -1.0%
random-access/parquet-tokio-local-disk	119858826 / 118927964 / +0.8%
random-access/parquet-tokio-local-disk-footer	118925133 / 117487949 / +1.2%
random-access/taxi/correlated/parquet-tokio-local-disk	179710553 / 176466380 / +1.8%
random-access/taxi/correlated/parquet-tokio-local-disk-footer	178142001 / 176544200 / +0.9%
random-access/taxi/uniform/parquet-tokio-local-disk	189840296 / 185059062 / +2.6%
random-access/taxi/uniform/parquet-tokio-local-disk-footer	191767299 / 189985177 / +0.9%

random-access / lance / ns (1.004x ➖, 0↑ 0↓)

name	ns (PR / base / %diff)
random-access/feature-vectors/correlated/lance-tokio-local-disk	213456 / 213609 / -0.1%
random-access/feature-vectors/correlated/lance-tokio-local-disk-footer	982768 / 1010450 / -2.7%
random-access/feature-vectors/uniform/lance-tokio-local-disk	1080394 / 1040514 / +3.8%
random-access/feature-vectors/uniform/lance-tokio-local-disk-footer	1857205 / 1869792 / -0.7%
random-access/lance-tokio-local-disk	544688 / 548107 / -0.6%
random-access/lance-tokio-local-disk-footer	1309726 / 1291762 / +1.4%
random-access/nested-lists/correlated/lance-tokio-local-disk	160286 / 160509 / -0.1%
random-access/nested-lists/correlated/lance-tokio-local-disk-footer	501576 / 488460 / +2.7%
random-access/nested-lists/uniform/lance-tokio-local-disk	895494 / 914257 / -2.1%
random-access/nested-lists/uniform/lance-tokio-local-disk-footer	1422264 / 1410465 / +0.8%
random-access/nested-structs/correlated/lance-tokio-local-disk	283187 / 279140 / +1.4%
random-access/nested-structs/correlated/lance-tokio-local-disk-footer	560679 / 563064 / -0.4%
random-access/nested-structs/uniform/lance-tokio-local-disk	2368214 / 2324098 / +1.9%
random-access/nested-structs/uniform/lance-tokio-local-disk-footer	2674994 / 2602463 / +2.8%
random-access/taxi/correlated/lance-tokio-local-disk	785866 / 796747 / -1.4%
random-access/taxi/correlated/lance-tokio-local-disk-footer	1837554 / 1823407 / +0.8%
random-access/taxi/uniform/lance-tokio-local-disk	8258121 / 8214856 / +0.5%
random-access/taxi/uniform/lance-tokio-local-disk-footer	8878184 / 8946234 / -0.8%

Use the historical layout-reader cache and outer Canonical materialization for both V1 and V2. Keep nested values lazy so the existing CI series compares equivalent work. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Drop the VORTEX_BENCH_SCAN=v1 switch and the V1-only layout-reader cache. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0161zMAb6XtH1LGJ52oLn9GG

Restore the develop format list and measurement names. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0161zMAb6XtH1LGJ52oLn9GG

tpchgen-cli 4.0.0 removed the --format and --parquet-compression flags. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0161zMAb6XtH1LGJ52oLn9GG

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0161zMAb6XtH1LGJ52oLn9GG

The tpchgen-cli 4.x fix lands separately in #10630. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0161zMAb6XtH1LGJ52oLn9GG

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs · 2026-10-20T10:11:49Z

Rust tests (windows-x64) failed on 60b2d43 in a test this PR doesn't touch: vortex-io runtime::tokio::tests::test_spawn_and_abort, at vortex-io/src/runtime/tokio.rs:220 (assert!(send.send(()).is_err())).

This PR has no changes in vortex-io. The test was last modified on develop in #10458. It checks timing: it expects a task that has just been cancelled to have already dropped its receiver, and that can race on the Windows runner. The other 10,127 tests in the job passed.

I don't know of an existing fix for it. I'm re-running the failed job once. If it fails again, the test needs a fix on develop, separately from this PR.

Generated by Claude Code

Resolve conflicts with #10625: it removed the V2 layout-support check, so ScanCache no longer caches one. Take develop's V2 benchmark take and reopen its file handles with the layout reader cache, so they share a ScanCache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014bhE34DXnMWjLtHgHpDUzX Signed-off-by: Claude <noreply@anthropic.com>

Plan V2 projections from known row selections

3cd67a2

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs added action/bench-compress Run only the compression benchmark on this PR action/bench-random-access Run only the random-access benchmark on this PR labels Oct 19, 2026 — with ChatGPT Codex Connector

github-actions Bot removed action/bench-compress Run only the compression benchmark on this PR action/bench-random-access Run only the random-access benchmark on this PR labels Oct 19, 2026

joseph-isaacs added the changelog/performance A performance improvement label Oct 19, 2026 — with ChatGPT Codex Connector

Preserve unchanged selection masks across projection fields

5012ca3

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs added action/bench-compress Run only the compression benchmark on this PR action/bench-random-access Run only the random-access benchmark on this PR labels Oct 19, 2026 — with ChatGPT Codex Connector

github-actions Bot removed action/bench-random-access Run only the random-access benchmark on this PR action/bench-compress Run only the compression benchmark on this PR labels Oct 19, 2026

Restore the random-access benchmark measurement contract

d326929

Use the historical layout-reader cache and outer Canonical materialization for both V1 and V2. Keep nested values lazy so the existing CI series compares equivalent work. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs added the action/bench-random-access Run only the random-access benchmark on this PR label Oct 20, 2026 — with ChatGPT Codex Connector

github-actions Bot removed the action/bench-random-access Run only the random-access benchmark on this PR label Oct 20, 2026

joseph-isaacs mentioned this pull request Oct 20, 2026

Stream selected V2 chunk planners from one file planner #10628

Closed

joseph-isaacs added the action/bench-random-access Run only the random-access benchmark on this PR label Oct 20, 2026

github-actions Bot removed the action/bench-random-access Run only the random-access benchmark on this PR label Oct 20, 2026

joseph-isaacs marked this pull request as ready for review October 20, 2026 09:38

robert3005 approved these changes Oct 20, 2026

View reviewed changes

joseph-isaacs and others added 3 commits October 20, 2026 09:40

bench: drop vortex-compact random-access changes

bcd6677

Restore the develop format list and measurement names. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0161zMAb6XtH1LGJ52oLn9GG

joseph-isaacs force-pushed the ji/v2-selection-planning branch from 9ed1dc0 to ce73df6 Compare October 20, 2026 09:42

joseph-isaacs and others added 3 commits October 20, 2026 09:52

bench: format random-access open path

894b5ab

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0161zMAb6XtH1LGJ52oLn9GG

Merge develop into ji/v2-selection-planning

60b2d43

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs merged commit cb39bb9 into develop Oct 20, 2026
120 of 122 checks passed

joseph-isaacs deleted the ji/v2-selection-planning branch October 20, 2026 10:13

joseph-isaacs mentioned this pull request Oct 20, 2026

feat(scan): cache V2 scan state across scans of one file #10631

Merged

Navigation Menu

Repository navigation

Plan V2 random access from known row selections - #10625

Plan V2 random access from known row selections#10625
joseph-isaacs merged 9 commits into
developfrom
ji/v2-selection-planning

joseph-isaacs commented Oct 19, 2026 •

edited

Loading

Uh oh!

codspeed Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

joseph-isaacs commented Oct 20, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

joseph-isaacs commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will regress 1 benchmark

Performance Changes

Footnotes

Uh oh!

github-actions Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Polar Signals Profiling Results

Latest Run

Uh oh!

github-actions Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks: Compression 📖

Uh oh!

github-actions Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks: Random Access 📖

Uh oh!

joseph-isaacs commented Oct 20, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

joseph-isaacs commented Oct 19, 2026 •

edited

Loading

codspeed Bot commented Oct 19, 2026 •

edited

Loading

github-actions Bot commented Oct 19, 2026 •

edited

Loading

github-actions Bot commented Oct 19, 2026 •

edited

Loading

github-actions Bot commented Oct 19, 2026 •

edited

Loading