Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,27 @@ jobs:
enable-sccache: "true"
- run: cargo minimal-versions check --direct --workspace --ignore-private

rust-msrv:
name: "Rust (MSRV)"
timeout-minutes: 30
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=rust-msrv', github.run_id)
|| 'ubuntu-latest' }}
env:
RUSTFLAGS: "-A warnings"
steps:
- uses: runs-on/action@v2
if: github.repository == 'vortex-data/vortex'
with:
sccache: s3
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: ./.github/actions/setup-prebuild
with:
enable-sccache: "true"
- name: Check publishable crates against the MSRV
run: cargo hack --rust-version --no-dev-deps --ignore-private check --profile ci --all-features

rust-lint:
name: "Rust (lint)"
needs: duckdb-ready
Expand Down
14 changes: 14 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,20 @@ repository root, create or update the development environment with:
uv sync --all-packages
```

### Rust toolchain and MSRV

Workspace declares a Minimum Supported Rust Version through `rust-version`
in the root `Cargo.toml`, which the `Rust (MSRV)` CI job verifies by building
the publishable crates with exactly that toolchain:

```bash
cargo hack --rust-version --no-dev-deps --ignore-private check --all-features
```

Read the [Rust version compatibility policy](README.md#rust-version-compatibility-policy) before
changing `rust-version`; when the MSRV job fails, the usual fix is in the code or the dependency
update, not the MSRV.

### Python bindings

`vortex-data` is a mixed Python and Rust package. `uv` manages its Python environment, and
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ members = [
"vortex-spatial",
]
exclude = ["java/testfiles", "wasm-test"]
resolver = "2"
resolver = "3"

[workspace.package]
authors = ["Vortex Authors <hello@vortex.dev>"]
Expand Down
36 changes: 36 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,10 @@ uv sync --all-packages
See the [development workflows](CONTRIBUTING.md#development-workflows) for Python binding and
documentation development, including Maturin rebuilds, targeted tests, and documentation checks.

`rust-toolchain.toml` pins the toolchain used for development and CI, and is kept on the latest
stable release. Building Vortex as a dependency only requires a toolchain that satisfies the
[Rust version compatibility policy](#rust-version-compatibility-policy).

### Benchmarking

Use `vx-bench` to run benchmarks comparing engines (DataFusion, DuckDB) and formats (Parquet, Vortex):
Expand Down Expand Up @@ -154,6 +158,38 @@ static GLOBAL_ALLOC: MiMalloc = MiMalloc;

## Project Information

### Rust Version Compatibility Policy

**The policy: Vortex supports the four most recent stable minor releases.** Writing the latest
stable release as `1.N`, that means `1.N`, `1.N-1`, `1.N-2`, and `1.N-3` all build Vortex, so the
three minor releases older than the latest stable release. Only the minor version is constrained;
Minimum Supported Rust Version (MSRV) declared in `Cargo.toml` must be **no newer than `1.N-3`**,
patch releases are never a factor.

An MSRV *older* than `1.N-3` is always acceptable — supporting extra releases cannot break the
guarantee. An MSRV *newer* than `1.N-3` does not meet the policy. For example, once `1.98` is the
latest stable release:

| Declared MSRV | Status |
| --- | --- |
| `1.94` or older | Acceptable — supports more releases than required |
| `1.95` | Exactly on policy |
| `1.96` or newer | Does not meet the policy |

The MSRV is raised in occasional deliberate steps rather than on every Rust release, so it drifts
relative to that bound.

How the policy is applied:

- The MSRV is declared once, as `rust-version` in the root `Cargo.toml`, and inherited by every
crate in the workspace. That value, not this document, is the source of truth.
- The toolchain pinned in `rust-toolchain.toml` tracks the latest stable release and is independent
of the MSRV. It is what contributors and most CI jobs build with.
- CI enforces the declared MSRV in the `Rust (MSRV)` job, which builds the publishable crates with
exactly that toolchain. When it fails, the first choices are to express the code without the
newer Rust feature, or to hold back the dependency update that raised the requirement. Raising
`rust-version` is a last resort.

### License

Licensed under the Apache License, Version 2.0.
Expand Down
2 changes: 1 addition & 1 deletion rust-toolchain.toml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[toolchain]
channel = "1.95"
channel = "1.97.1"
components = ["rust-src", "rustfmt", "clippy", "rust-analyzer"]
profile = "minimal"
12 changes: 2 additions & 10 deletions vortex-array/src/expr/stats/bound.rs
Original file line number Diff line number Diff line change
Expand Up @@ -139,9 +139,7 @@ impl<T: PartialOrd> PartialOrd<T> for LowerBound<T> {
fn partial_cmp(&self, other: &T) -> Option<Ordering> {
match &self.0 {
Exact(lhs) => lhs.partial_cmp(other),
Inexact(lhs) => lhs
.partial_cmp(other)
.and_then(|o| if o == Ordering::Less { None } else { Some(o) }),
Inexact(lhs) => lhs.partial_cmp(other).filter(|&o| o != Ordering::Less),
Absent => None,
}
}
Expand Down Expand Up @@ -241,13 +239,7 @@ impl<T: PartialOrd> PartialOrd<T> for UpperBound<T> {
fn partial_cmp(&self, other: &T) -> Option<Ordering> {
match &self.0 {
Exact(lhs) => lhs.partial_cmp(other),
Inexact(lhs) => lhs.partial_cmp(other).and_then(|o| {
if o == Ordering::Greater {
None
} else {
Some(o)
}
}),
Inexact(lhs) => lhs.partial_cmp(other).filter(|&o| o != Ordering::Greater),
Absent => None,
}
}
Expand Down
2 changes: 1 addition & 1 deletion vortex-bench/src/tpcds/tpcds_benchmark.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ impl TpcDsBenchmark {
Url::from_directory_path(&data_dir_with_sf).map_err(|_| {
anyhow!(
"Failed to create URL from directory path: {:?}",
&data_dir_with_sf
data_dir_with_sf
)
})
}
Expand Down
2 changes: 1 addition & 1 deletion vortex-bench/src/tpch/benchmark.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ impl TpcHBenchmark {
Url::from_directory_path(&data_dir_with_sf).map_err(|_| {
anyhow::anyhow!(
"Failed to create URL from directory path: {:?}",
&data_dir_with_sf
data_dir_with_sf
)
})
}
Expand Down
2 changes: 1 addition & 1 deletion vortex-bench/src/utils/file.rs
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ pub fn resolve_data_url(remote_data_dir: Option<&str>, local_subdir: &str) -> Re
None => {
let data_dir = data_dir().join(local_subdir);
Url::from_directory_path(&data_dir).map_err(|_| {
anyhow::anyhow!("Failed to create URL from directory path: {:?}", &data_dir)
anyhow::anyhow!("Failed to create URL from directory path: {:?}", data_dir)
})
}
Some(remote_data_dir) => {
Expand Down
2 changes: 1 addition & 1 deletion vortex-datafusion/src/v2/source.rs
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,7 @@ impl fmt::Debug for VortexDataSource {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
f.debug_struct("VortexScanSource")
.field("schema", &self.leftover_schema)
.field("projection", &format!("{}", &self.projected_projection))
.field("projection", &format!("{}", self.projected_projection))
.field("filter", &self.filter.as_ref().map(|e| format!("{}", e)))
.field("limit", &self.limit)
.finish()
Expand Down
9 changes: 5 additions & 4 deletions vortex-duckdb/src/table_function.rs
Original file line number Diff line number Diff line change
Expand Up @@ -827,10 +827,11 @@ pub fn statistics(bind_data: &TableFunctionBind, column_index: usize) -> Option<
let MultiLayoutChild::Opened { reader, .. } = &children[0] else {
return None;
};
let stats_sets = match reader.as_any().downcast_ref::<FileStatsLayoutReader>() {
Some(inner) => inner.file_stats().stats_sets(),
None => return None,
};
let stats_sets = reader
.as_any()
.downcast_ref::<FileStatsLayoutReader>()?
.file_stats()
.stats_sets();
// Columns with pushed projection expression output expression results,
// and not column values
if bind_data.column_fields[column_index]
Expand Down
2 changes: 1 addition & 1 deletion vortex-python/src/iter/python.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ impl Iterator for PythonArrayIterator {
if array.dtype() != &self.dtype {
Err(PyTypeError::new_err(format!(
"ArrayIterator dtype mismatch. Expected {:?}, got {:?}",
&self.dtype,
self.dtype,
array.dtype()
)))
} else {
Expand Down
2 changes: 1 addition & 1 deletion vortex-python/src/scalar/factory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ fn scalar_helper_inner(value: &Bound<'_, PyAny>, dtype: Option<&DType>) -> PyRes
if names != dtype.names() {
return Err(PyValueError::new_err(format!(
"Dictionary field names {:?} do not match target dtype names {:?}",
&names,
names,
dtype.names()
)));
}
Expand Down
2 changes: 1 addition & 1 deletion vortex-web/crate/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name = "vortex-web-wasm"
version = "0.1.0"
edition = "2024"
rust-version = "1.91.0"
rust-version = { workspace = true }
license = "Apache-2.0"
description = "WASM bindings for the Vortex web explorer"
publish = false
Expand Down
Loading