Skip to content

ci: split platform crates from workspace catch-all - #822

Merged
ruvnet merged 2 commits into
mainfrom
ci/split-core-platform
Aug 13, 2026
Merged

ci: split platform crates from workspace catch-all#822
ruvnet merged 2 commits into
mainfrom
ci/split-core-platform

Conversation

@ruvnet

@ruvnet ruvnet commented Aug 13, 2026

Copy link
Copy Markdown
Owner

Summary

  • move 17 service/distributed crates and 9 native Node bindings into a dedicated core-platform test shard
  • reduce the core-and-rest catch-all from 78 packages to 50
  • move ruvector-graph-condense-wasm into the WASM shard and correct the duplicated decompiler exclusion

Why

The post-merge and PR validations for #821 remained in the catch-all test step beyond the historical 155–165 minute completion window. Five preceding runs timed out at 240 minutes with compile/link processes still active. This additional split keeps full test/doctest coverage while reducing cold-cache linker pressure.

Validation

  • workflow YAML parses successfully
  • all 26 core-platform package names exist in workspace metadata
  • no core-platform package overlaps another explicit shard
  • catch-all package count verified at 50
  • git diff --check passes

The `core-and-rest` catch-all shard was not sharding. Its `packages:` value
is a YAML folded scalar (`>-`) that contained `#`-prefixed lines *inside*
the scalar. Those are content, not YAML comments: folding joins the whole
block onto one line, and when `run:` expands `${{ matrix.packages }}` the
shell treats the first `#` as the start of a comment and truncates the rest
of the command.

The effective command was therefore:

    cargo nextest run --no-fail-fast --workspace

0 of 162 `--exclude` flags survived. Every shard split landed in iters
230-240 was inert — the catch-all kept building and testing all 210
workspace crates, which is why it kept drifting into the job timeout no
matter how many crates were hoisted out of it. The doctest step had the
same truncation. `core-and-rest-wasm` had the same defect but its inline
comment was trailing prose with no flags after it, so all 29 `-p` flags
survived; fixed anyway so the pattern does not get copied.

Move every comment above the `packages:` key at mapping level, where YAML
strips it — the pattern the `core-platform` entry already used. The folded
scalars now contain only `--workspace`, `--exclude <crate>`, and
`-p <crate>` tokens.

Verified by parsing the workflow with PyYAML and diffing each shard's
effective package set against `cargo metadata`: 162 excludes / 50 effective
packages in the catch-all, 203 of 210 crates covered, no crate built twice.

Also add `.config/nextest.toml` with a 10-minute per-test kill switch.
`ruvector-delta-index::tests::test_insert_and_search` hangs indefinitely in
DeltaHnsw insert/search rather than failing, and with no timeout it consumed
3h52m of the 240-minute `core-platform` budget. Hold that crate out of CI
until it is fixed (#825).

Co-Authored-By: claude-flow <ruv@ruv.net>
@ruvnet

ruvnet commented Aug 13, 2026

Copy link
Copy Markdown
Owner Author

Review — 2 blocking issues found and fixed

Reviewed against this PR's own CI run logs. The shard split arithmetic is correct, but two defects made it ineffective. Both are fixed in 09cccfe.


Finding 1 (critical): every --exclude in the catch-all shard was silently discarded

packages: is a YAML folded scalar (>-), and the core-and-rest entry had #-prefixed lines inside the scalar. Those are content, not YAML comments. Folding joins the block onto one line, and when the run: step expands ${{ matrix.packages }} the shell treats the first # as a comment and truncates everything after it.

The command that actually ran was:

cargo nextest run --no-fail-fast --workspace

0 of 162 --exclude flags survived. The catch-all shard was building and testing all 210 workspace crates on every run. That explains the behaviour the timeout bumps in iters 228-240 were chasing: every shard split landed so far has been inert, so the catch-all kept drifting into the cap no matter how many crates were hoisted out of it. The doctest step (cargo test --doc ${{ matrix.packages }}) had the same truncation.

core-and-rest-wasm had the same defect, but its inline comment was trailing prose with no flags after it, so all 29 -p flags survived. Fixed anyway so the pattern is not copied forward.

Fix: moved every comment above the packages: key at mapping level, where YAML strips it — the pattern the new core-platform entry already used correctly. The folded scalars now contain only --workspace, --exclude <crate>, and -p <crate> tokens.

Finding 2 (critical): a hung test can burn the entire job budget

ruvector-delta-index::tests::test_insert_and_search (crates/ruvector-delta-index/src/lib.rs:731) hangs indefinitely in DeltaHnsw insert/search rather than failing. It sat in a nextest SLOW loop for 3h52m on run 31675583441 until core-platform was cancelled at the 240-minute cap. The test builds its input from an unseeded rand::thread_rng(), so the hang is nondeterministic — which is why it only surfaced once this PR gave the crate its own shard.

There was no .config/nextest.toml, so nothing bounded it.

Fix:


Shard coverage — verified

Parsed the workflow with PyYAML and diffed each shard's effective package set against cargo metadata:

shard mode excludes effective pkgs
vector-index -p list 0 6
rvagent -p list 0 10
ruvix -p list 0 16
ml-research-heavy -p list 0 4
ml-research-rest -p list 0 6
core-and-rest-heavy -p list 0 9
core-and-rest-wasm -p list 0 29
research-nightly -p list 0 12
core-and-rest-examples -p list 0 36
core-platform (new) -p list 0 25
core-and-rest --workspace 162 50
  • 26 crates moved into core-platform by this PR; 25 remain after holding out ruvector-delta-index. Each has a matching --exclude in the catch-all.
  • No crate is built twice — the previously double-built ruvector-decompiler is fixed (it is in core-and-rest-heavy and excluded from the catch-all exactly once).
  • 203 of 210 crates covered. The 7 uncovered are all deliberate: the 4 hailo crates (hailort-sys, ruvector-hailo, ruvector-hailo-cluster, ruvector-mmwave), ruvllm-wasm (wasm-target-only tests SIGABRT on native), timesfm (tested by its own dedicated step with --features candle), and ruvector-delta-index (ruvector-delta-index test_insert_and_search hangs indefinitely (DeltaHnsw insert/search) #825).

Only after Finding 1 is fixed do those 162 excludes take effect — so this is the first run where the shard split will actually reduce the catch-all's workload, from 210 crates to 50.

Note on scope

Two pre-existing --exclude targets are not workspace members: ruvector-postgres and timesfm-wasm. Harmless and unrelated to this PR, so left alone.


Review and fixes generated by Claude.

@ruvnet
ruvnet merged commit 6c7cb70 into main Aug 13, 2026
43 of 46 checks passed
13obbyMack pushed a commit to 13obbyMack/ruvector that referenced this pull request Aug 15, 2026
The terminate-after cap added in ruvnet#822 (10 minutes) was sized for the
regular shards and killed legitimately long tests on the first sharded
run of main: ruvector-mincut subpolynomial tests and
ruvector-nervous-system pattern_separation_collision_rate all TIMEOUT
at exactly 600s in run 31714159600, failing ml-research-heavy and
core-and-rest-heavy. Historical successful runs of those shards take
up to ~75 minutes with individual tests exceeding 10 minutes by design.

Scope a slow-timeout override to the 13 packages of the two heavy
shards: 300s SLOW reporting period, killed after 18 periods (90 min) —
still bounding a genuine hang at 1.5h instead of the 4h job budget.

Co-Authored-By: claude-flow <ruv@ruv.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant