Skip to content

Broaden the string-to-Arrow benchmarks - #9372

Open
robert3005 wants to merge 1 commit into
rk/array-capabilitiesfrom
rk/string-arrow-benches
Open

Broaden the string-to-Arrow benchmarks#9372
robert3005 wants to merge 1 commit into
rk/array-capabilitiesfrom
rk/string-arrow-benches

Conversation

@robert3005

@robert3005 robert3005 commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Stacked on #9371.

Broaden string to arrow benchmarks to codify the currently optimised set of exporters, taken from #9339

@codspeed-hq

codspeed-hq Bot commented Aug 11, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 11.71%

⚡ 1 improved benchmark
✅ 1961 untouched benchmarks
🆕 4 new benchmarks
⏩ 89 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation slice_primitive_tight_loop[10000] 473.8 µs 424.1 µs +11.71%
🆕 Simulation file_to_offset_array[binary_array] N/A 317.7 ms N/A
🆕 Simulation file_to_offset_array[string_array] N/A 317.7 ms N/A
🆕 Simulation file_to_offset_batches[binary_array] N/A 360.2 ms N/A
🆕 Simulation file_to_offset_batches[string_array] N/A 360.4 ms N/A

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing rk/string-arrow-benches (4fce0bc) with develop (ca7f626)2

Open in CodSpeed

Footnotes

  1. 89 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on rk/array-capabilities (c13f3f8) during the generation of this report, so develop (ca7f626) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@robert3005
robert3005 force-pushed the rk/string-arrow-benches branch from 324314a to cc7d5ed Compare August 11, 2026 23:35
Rework `vortex-arrow`'s string export benchmarks into a matrix over encoding
(offset, view, FSST, OnPair, Zstd), structure (flat, dict, chunked), operator
(identity, filter, take, slice, mask, zip), and nullability, exported to both
Arrow offset and view layouts, plus the same cases appended directly into
`VarBinBuilder` and `VarBinViewBuilder`.

Add a `string_to_arrow` benchmark in `vortex` that measures the end-to-end file
scan producing Arrow offset arrays, so the export is also covered with data laid
out by the default compressor rather than by hand.

Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005
robert3005 force-pushed the rk/string-arrow-benches branch from cc7d5ed to 4fce0bc Compare August 11, 2026 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant