Skip to content

feat: make infino runnable in the clickbench playground #1307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
naman-modi wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: make infino runnable in the clickbench playground #1307

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions infino/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,12 @@ queries, so `tuned=no`.
./install # installs rustup if absent, then cargo build --release
```

The release build is `opt-level = 3`, `lto = "fat"`, `codegen-units = 1`
(`bench/Cargo.toml`), a portable build with no `-C target-cpu`, matching the
`datafusion` variant's recipe. Pinned to a published crate version
(`infino = "0.1.10"`) so the result is reproducible from crates.io alone.
The release build is portable (no `-C target-cpu`), pinned to a published crate
version (`infino = "0.5.3"`) so the result is reproducible from crates.io alone.
LTO is sized to the machine by `install`: fat LTO + `codegen-units = 1` at
>= 12 GiB RAM (its single-pass link peaks ~7.6 GB), thin LTO below so the small
VMs still build. Reads use strong consistency, so a query issued right after
load sees every ingested row.

## Run (automated ClickBench flow)

Expand All @@ -45,7 +47,7 @@ seq 0 10 | xargs -P11 -I{} wget -q --continue \

INFINO_SRC="hits_*.parquet" INFINO_MAX_ROWS=10000000 ./load # ingest once
./start # launch the server
printf '%s\n' "SELECT COUNT(*) FROM hits" | ./query # row count + seconds
printf '%s\n' "SELECT COUNT(*) FROM hits" | ./query # result table + seconds
./stop # shut the server down
```

Expand All @@ -54,9 +56,11 @@ printf '%s\n' "SELECT COUNT(*) FROM hits" | ./query # row count + seco
`serve` opens the table once and answers one query per connection on
`INFINO_SOCK` (default `./infino.sock`). Timing wraps `query_sql` only, so the
socket round-trip is never counted. `./query` sends stdin to the server and
prints the row count to stdout and the elapsed seconds to stderr (the
ClickBench query-script contract). `./check` pings the server so the driver can
detect it coming up and going down.
prints the query result as a text table to stdout (a bounded preview: up to
1000 rows and `CLICKBENCH_OUTPUT_LIMIT` bytes, so a huge result cannot blow up)
and the elapsed seconds to stderr (the ClickBench query-script contract). The
result on stdout is also what the online playground shows the user. `./check`
pings the server so the driver can detect it coming up and going down.

## Environment

Expand Down
Loading
Loading