ci: run crate tests with cargo-nextest and Rust jobs on ev-runner-x-large - #233
Conversation
| toolchain: stable | ||
| components: rustfmt, clippy | ||
| - name: Install nextest | ||
| uses: taiki-e/install-action@nextest |
| override: true | ||
| components: rustfmt, clippy | ||
| - name: Install nextest | ||
| uses: taiki-e/install-action@nextest |
Replace the `cargo test -p ev-cli -p ev-enclave` step in both `clippy_check_cli` jobs (lint-and-test-cli and release-cli-version-staging) with `cargo nextest run`, installing nextest via taiki-e/install-action. nextest runs each test in its own process, so the two enclave-build tests annotated with `#[serial_test::serial]` are no longer serialized by serial_test's in-process mutex. Add .config/nextest.toml with a `serial` test group (max-threads = 1) covering those tests to preserve the previous one-at-a-time behaviour. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RQHtsSNmrd1N7QCVRhYXCv
Move the jobs that invoke the Rust compiler off ubuntu-latest onto the self-hosted ev-runner-x-large runner: - clippy_check_cli (lint-and-test-cli, release-cli-version-staging) - compile-ubuntu (build-and-publish) compile-macos stays on macos-latest since it builds the x86_64-apple-darwin target and needs a macOS host. Jobs that don't compile Rust (get-version, upload-artifacts-to-s3, release-cli-version) remain on ubuntu-latest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RQHtsSNmrd1N7QCVRhYXCv
819288f to
d7b26a9
Compare
| run: cargo build | ||
| - name: Test project | ||
| run: cargo test -p ev-cli -p ev-enclave | ||
| run: cargo nextest run -p ev-cli -p ev-enclave |
There was a problem hiding this comment.
can we also add this in a precommit hook if we have one that runs tests? i worry about tests running locally being different to what runs in the cloud
There was a problem hiding this comment.
or is there any way for us to proxy cargo test to nextest in the background
There was a problem hiding this comment.
We can, I don't love precommit hooks for tests. I think they work better for things that run almost instantly, such as formatting. I don't believe we can replace what the default cargo test action does :( I think this will be fine, nextest is setup in most if not all of our rust repos, so if it did fail in CI but not locally, a developer could then run nextest locally and get the faster/prettier output; the other main difference is running tests in parallel.
Why
Two CI changes for the crate workflows:
clippy_check_clijobs fromcargo testtocargo-nextest— a next-generation test runner with per-test process isolation, better output, and faster runs.ubuntu-latestonto the self-hostedev-runner-x-largerunner.How
nextest migration
taiki-e/install-action@nextestand swapcargo test -p ev-cli -p ev-enclave→cargo nextest run -p ev-cli -p ev-enclavein both jobs that run the suite:.github/workflows/lint-and-test-cli.yml.github/workflows/release-cli-version-staging.yml.config/nextest.tomlwith aserialtest group (max-threads = 1). This is the one non-mechanical part: nextest runs each test in its own process, so the two enclave-build tests annotated#[serial_test::serial](test_choose_output_dir,test_reproducible_enclave_builds_with_pinned_version) are no longer serialized —serial_test's mutex is in-process only. Both writecert.pem/key.peminto, and build from, the crate working directory, so running them concurrently would race. The test group restores the previous one-at-a-time behaviour. Confirmed it resolves to exactly those two tests viacargo nextest show-config test-groups.Runner change
Set
runs-on: ev-runner-x-largeon the three jobs that compile Rust:lint-and-test-cli.ymlclippy_check_cliubuntu-latestev-runner-x-largerelease-cli-version-staging.ymlclippy_check_cliubuntu-latestev-runner-x-largebuild-and-publish.ymlcompile-ubuntuubuntu-latestev-runner-x-largecompile-macoswas left onmacos-latest— it builds thex86_64-apple-darwintarget and needs a macOS host, andev-runner-x-largeappears to be a Linux runner. Ifev-runner-x-largeis actually macOS-capable, say so and I'll switch it too. Jobs that don't compile Rust (get-version,upload-artifacts-to-s3,release-cli-version) stay onubuntu-latest.Verification
Ran both runners locally over the same scope:
cargo test -p ev-cli -p ev-enclavecargo nextest run -p ev-cli -p ev-enclaveThe 5 failures are identical across both and purely environmental — the Docker-based enclave-build / synthetic-enclave tests, and the sandbox has no Docker daemon. They pass in CI (which provides Docker). This confirms the nextest migration is behaviour-preserving.
Notes
cargo nextest run(which does not run doctests) has the same coverage ascargo testhere.clippy_check_clijobs were migrated together for consistency. The#[serial_test::serial]source annotations were left untouched — harmless under nextest, and still applied by localcargo testruns.🤖 Generated with Claude Code
https://claude.ai/code/session_01RQHtsSNmrd1N7QCVRhYXCv