[LeeroyHannigan]

What

Two jobs ran tests that CI also runs elsewhere in the same run.

mongodb-rust

CI's mongodb-rust job no longer runs the whole workspace's cargo test. It ran devtools/run-mongodb-tests --rust-live -- --rust --rust-integration. The --rust suite of devtools/run-tests runs cargo test over all 14 workspace crates. The job now runs devtools/run-mongodb-tests --rust-live -- --rust-integration.

devtools/run-mongodb-tests now names both Rust suites in the condition of its storage-level sections (records_live and the migration tests). The old condition matched the substring --rust, which --rust-integration also contains. Without --rust, the old condition selects these sections only because of that match. The new condition names --rust-integration, so the job does not depend on the substring.

The integration shards

Every shard of run-integration (PostgreSQL) and run-integration-sqlite ran tests/test_backup_store_config.py and tests/test_backup_job.py by name. Each SQLite shard also ran the store configuration suite inside devtools/run-tests.

• PostgreSQL: the new run-integration-backup job runs the two files once per run, with the shards' PostgreSQL service, the S3 mock and the same environment. The integration job needs it. The CLI lifecycle list of devtools/run-tests still runs the store configuration suite's PostgreSQL cells on one shard.
• SQLite: the shards run neither file. devtools/run-tests skips its SQLite backup store block in a run with --shard. The shards no longer start the S3 mock, whose one reader was that block. run-integration-sqlite-s3 already ran both files on the same build with an S3 mock.

Why

The mongodb-integration check waits for six jobs, and mongodb-rust finishes last. On main's push run for 96a6b6f, the workspace cargo test took 12.2 of the job's 31.8 minutes. Cargo compiled the test binaries in 4 min 58 s and ran them in 7 min 12 s. The Test workflow runs the same test binaries on stable and on 1.88.0.

The integration check waits for the shards. On this PR's run for d9528e6 (run 42637473269), the PostgreSQL shards took 16 min 52 s to 18 min 14 s. The two files took 2 min 52 s to 3 min 8 s of that, and their S3 mock 22 to 28 s more. The SQLite shards took 20 min 6 s to 21 min 52 s. The two files took 4 min 28 s to 4 min 30 s of that. The run inside devtools/run-tests took 2 min 45 s to 2 min 46 s more. Without these runs, the PostgreSQL shards should take 13 min 38 s to 14 min 44 s. The SQLite shards should take 12 min 26 s to 14 min 7 s. Both are projections from the same run.

Testing done

mongodb-rust

• The tests that need MongoDB still run against a replica set:
  • records_live and the five migration test files run in the storage-level sections of run-mongodb-tests. The migration tests run with EXTENDDB_TEST_MONGODB_REQUIRED=1.
  • The data_engine and backup_engine live tests and the restore fence cells run under --rust-live.
  • The 12 snapshot_engine tests run in the mongodb-snapshot job, with EXTENDDB_TEST_MONGODB_REQUIRED=1.
• Of the 14 workspace crates, only crates/storage-mongodb reads EXTENDDB_TEST_MONGODB_URI or EXTENDDB_TEST_MONGODB_REQUIRED. The Test workflow runs every test binary of the workspace cargo test. It has no MongoDB server, so the migration and snapshot_engine tests skip there.
• The Test workflow does not set EXTENDDB_ADMIN_USER and EXTENDDB_ADMIN_PASSWORD, which run-mongodb-tests exports. When the environment has them, an in-memory or dev SQLite open in crates/storage-sqlite passes them to bootstrap_dev_rows. The workspace tests passed both with and without them.
• The old and new conditions give the same result for --rust, --rust-integration, --rust --rust-integration, --pytest --comprehensive --parallel and --pytest --parallel --filter backup. They also give the same result for all 255 combinations of the eight suite flags of run-tests.
• bash -n passes on devtools/run-mongodb-tests. shellcheck reports one info finding there: SC2329, for the cleanup function that trap calls. Main's copy of the script gives the same finding. actionlint passes on the workflow.
• CI on d9528e6 passed. Minutes on main's push run for 96a6b6f and on this PR's run:

Measure	Main, 96a6b6f	This PR, d9528e6
mongodb-rust run time	31.8	17.7
The workspace cargo test in mongodb-rust	12.2	not run
build-mongodb run time	6.5	4.4
mongodb-integration, from the start of the workflow run	38.6	22.4

• The removed section accounts for 12.2 of the 14.1 minutes that mongodb-rust saved. The Rust cache likely explains the other 1.9 minutes and the 2.1 minutes that build-mongodb saved. Main's run started without one and saved the cache that this PR's run restored.
• The other sections of mongodb-rust ran the same tests in both runs, and all passed. The storage-level sections ran 6 records_live tests and 41 migration tests. The Rust integration suite ran 514 tests. The live test sections ran 19 data_engine tests, 41 backup engine tests and 3 restore fence cells. This PR's log has no === Rust tests (debug) section.

The integration shards

• On run 42637473269, run-integration-sqlite-s3 passed every test that a SQLite shard passed in either of its runs of the two files. Shard 2's run by name passed 26 tests, and its run inside devtools/run-tests passed 13 tests of the store configuration suite.
• The new job has the same services as a PostgreSQL shard and the same steps for the two files. From reading the two files' fixtures, nothing they read comes from an earlier shard step: each PostgreSQL cell runs extenddb init on a catalog of its own.
• With a stub SQLite binary, the block of devtools/run-tests runs the store configuration suite in a run without --shard. For shards 1/3 and 2/3 it prints a skip line.
• actionlint passes on the workflow. bash -n passes on devtools/run-tests, and shellcheck gives the same finding there as on main's copy.
• CI on d833c71 passed. Minutes for the jobs of the Integration workflow on this PR's runs for d9528e6 (run 42637473269) and d833c71 (run 42691010109):

Job	d9528e6	d833c71
run-integration shards	16.9 to 18.2	12.8 to 14.8
run-integration-sqlite shards	20.1 to 21.9	10.2 to 13.8
run-integration-backup	not run	4.4
integration, from the start of the workflow run	26.5	19.8

• On run 42691010109, run-integration-backup passed 17 tests and skipped 15, as each PostgreSQL shard's run of the two files did on run 42637473269. run-integration-sqlite-s3 gave the same counts as on that run. Its run in devtools/run-tests passed 12 and skipped 5, and its run by name passed 28 and skipped 4. Each SQLite shard printed the skip line.

Checklist

• I have read CONTRIBUTING.md
• All tests pass (cargo test --workspace)
• Code is formatted (cargo fmt --check)
• Clippy is clean (cargo clippy -- -W clippy::pedantic)
• I have added or updated tests for new functionality
• I have updated documentation if behavior changed
• Breaking changes are noted below (if any)
• If this changes the wire protocol, Storage trait, auth model, on-disk
  format, or public CLI surface, an RFC has been accepted or is linked
  below. Otherwise, an ADR captures the decision (link below).

ADR / RFC: n/a

---

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.