Repository navigation
Vortex 0.90.0 results for DuckDB and DataFusion on c8g.metal-48xl, c7a.metal-48xl and c8g.4xlarge - #8
Open
joseph-isaacs wants to merge 1 commit into
Open
joseph-isaacs wants to merge 1 commit into
joseph-isaacs wants to merge 1 commit into
Conversation
…a.metal-48xl and c8g.4xlarge DataFusion: build vortex-datafusion-cli 0.90.0-55.2.0 (Vortex 0.90.0, DataFusion 55.2.0). DuckDB: build duckdb-vortex from source for both the single and the partitioned variant, pinned to a commit with Vortex 0.90.0 on DuckDB v2.0.0. DuckDB's own `INSTALL vortex` ships an older Vortex, and the partitioned variant was still on v0.44.0. Building against DuckDB 2.0 needs three fixes: link vortex into the shell explicitly (STATICALLY_LINK_EXTENSIONS), skip C++20 module scanning, which needs clang-scan-deps, and raise the open-file limit for many-core hosts. The install script now checks for vortex_version() rather than for the binary, since the plain shell is linked before the extension is built. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Vortex 0.90.0 results for the four Vortex systems on c8g.metal-48xl, c7a.metal-48xl and c8g.4xlarge, run on 2026-11-02 with one EC2 instance per system and machine (500 GB gp2, as in
run-benchmark.sh).This PR is against
ji/upstream-main-6160e81, a snapshot ofClickHouse/ClickBenchmain at6160e8148, because this fork'smainis 7,411 commits behind upstream. It is for review in the fork, not a submission upstream.Changes
datafusion-vortexanddatafusion-vortex-partitioned:CLI_TAG0.87.0-55.1.0→0.90.0-55.2.0(Vortex 0.90.0, DataFusion 55.2.0).duckdb-vortexandduckdb-vortex-partitioned: build duckdb-vortex59b149bf(Vortex 0.90.0 on DuckDB v2.0.0) from source instead ofINSTALL vortex. The build needed three DuckDB 2.0 workarounds:STATICALLY_LINK_EXTENSIONSincludingvortex,CMAKE_CXX_SCAN_FOR_MODULES=OFF, and a raised open-file limit.load,queryandcheckcall the built binary.*/results/20261102/.Results
Against the current upstream results (hot geomean with ClickBench's 10 ms offset, new/old):
Before submitting upstream
0.90.0-55.2.0exists. These runs built its head869db5f.ji/vortex-0.90.0; the scripts pin commit59b149bf. DuckDB v2.0.0's ownINSTALL vortexships Vortex from 2026-10-19 (0.88.0 plus 125 commits), which would be a simpler alternative.COPY … (FORMAT vortex). DataFusion writes the same file in about 100 s.Checks
validate-results.py --baseupstream main passes, with no warnings in the new files.vortex_version()= 0.90.0.🤖 Generated with Claude Code