Skip to content

fix(cargo-gamma): close actual scheduled mutation gaps - #359

Open
martin-kolinek wants to merge 42 commits into
mainfrom
makolnek-microsoft-scheduled-build-timings
Open

martin-kolinek wants to merge 42 commits into
mainfrom
makolnek-microsoft-scheduled-build-timings

Conversation

@martin-kolinek

Copy link
Copy Markdown
Collaborator

Summary

  • Address all six mutation survivors across four source sites from the actual post-merge scheduled run.
  • Precisely exclude the equivalent default-value replacement and the native canonicalization forwarding mutation, retaining scripted result/error-policy coverage.
  • Split platform-specific path identity and transient-error classification into cfg-gated function items so Gamma 0.4.0 does not mutate disabled tail expressions.
  • Leave generated workflows, Gamma 0.4.0, the 100% score threshold, and the zero-flaky gate unchanged.

Verification

The source scheduled run exposed only the two exhaustive mutation-gate failures: https://github.com/microsoft/ox-tools/actions/runs/42976692409

The actual full anvil-scheduled workflow passed at this PR head, ac54232: https://github.com/microsoft/ox-tools/actions/runs/42996589390

  • All 14 validation jobs passed; the failure publisher correctly skipped.
  • Linux: 59,858/59,858 mutants killed. Windows: 59,598/59,598 killed. Zero survivors, timeouts, OOMs, or uncovered mutants on either platform.
  • Regular tests, advisories, and runtime analysis passed on all four platforms. Feature powersets and benchmark compilation passed on both exhaustive platforms.
  • Total elapsed approximately 2h26m. This was the real scheduled workflow dispatched on the branch, not the former protected diagnostic harness.

Focused regressions passed on Linux and Windows; all-feature library Clippy passed on Windows. Broad local library runs were not green due to developer-host Cargo configuration, Windows symlink privilege, and WSL-mounted filesystem limitations. The complete hosted workflow is the acceptance evidence.

martin-kolinek and others added 30 commits November 3, 2026 17:32
…stic

Replace native process startup and subsecond timing assertions with the existing listing-supervision seam and an exhausted scripted output budget. Keep a live reader sender to model an escaped descendant without launching processes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exclude 19 expensive generated-text contract assertions only under Miri. Keep native/cargo-careful execution, artifact rendering and ownership checks, and parser/planner coverage unchanged; document the safety tradeoff.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover xversion as a missing version and a leading NUL as malformed YAML. Both exact prefix-stripping mutants from the scheduled Gamma campaign now fail their focused behavioral regressions; production parsing remains unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep complete native/cargo-careful failure-injection and adoption matrices. Under Miri retain the successful plan, first/middle/last resolver failures, both TOML shapes, Unicode, both newline styles, and both catalog orders without repeating every Cartesian combination.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Start the timeout assertion only after the contained descendant is ready, then exercise the real supervisor with a zero budget. Keep this OS acceptance behind system-tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep expensive planner policy matrices, YAML permutations, and generated-text scans native/cargo-careful only. Retain focused parser and lifecycle coverage, add a small BOM/Unicode/CRLF flow round trip, and shrink builder fixtures without skipping their contracts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Inject signalling into the rejected-ID tests while retaining native termination acceptance and mutation coverage. Linux killpg(1, SIGKILL) broadcasts beyond the test process tree.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run the unchanged Anvil group on ubuntu-latest and publish bounded host resource, process, kernel, and phase snapshots to a separate live check.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The active run retains its immutable observer and source snapshot. Keep the lasting change limited to safe process-group guard testing and its design documentation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Check the exact child arguments before launching the current harness. Add a construction-only regression for the empty-user-arguments path and require exactly one passing native child.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Capture synced pre-launch identities and spawned PIDs, preserve campaign journals without saved environments, and observe a fresh hosted runner outside a bounded workload scope.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run the campaign as a dedicated unprivileged user on the disposable hosted VM. Bound its entire root-owned user slice so automatic systemd sibling scopes cannot leave the memory/task ceiling; observe from outside that identity with no job token in the campaign.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The hosted image rejects MemoryOOMGroup as a systemctl assignment. Set and verify memory.oom.group through the root-owned slice directly, and keep diagnostic output writable by the runner and isolated campaign identity.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The runner home prevents the dedicated user from reaching the checkout. Grant only directory traversal on its ancestors and verify checkout/tool/output access before launching the protected campaign.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run 42267402403 continues at immutable commit ebadc79. Retain the production selection guards without diagnostic hooks or workflow; session artifacts preserve the harness and verified user-slice limits.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A benign Linux lookup probe confirms CARGO_HOME/bin precedes PATH. Install and byte-compare the diagnostic coordinator in the isolated user's Cargo bin before running the unchanged group, retaining the already verified user-slice limits.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The active hosted run retains the instrumented coordinator and independent observer at 493b9f7. Restore the lasting production fixes without leaving one-off workflow or tracing code.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Repeat the bounded hosted investigation with allowlisted baseline failure messages; do not publish saved environment overrides.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Retained baseline errors show the dedicated user's temporary root was group-writable. Set its mode explicitly and use a private campaign umask, preserving cache ownership checks and the root-managed resource boundary.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…umasks

Use the existing private-directory helper before cache claiming, so preparation does not create and then reject its own redirected cache. Reuse the private temporary root for CLI fixtures. Remove ineffective diagnostic-only umask overrides.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep external boundary counter loss fatal and sample that boundary independently of currently observed child processes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Inject the native relaunch attempt into the once-only policy so guard mutations fail in-process instead of recursively creating sibling systemd scopes. Cover unmarked result propagation and document the non-spawning contract.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The protected campaign identified recursive relaunch mutant ddda37216735, ordinal 50167. Preserve the collected evidence as session artifacts and remove the temporary hosted workflow, external observer, and synced launch tracing.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reuse the independently enforced Linux user-slice limits and fail-closed observer while executing the unchanged full-workspace exhaustive recipe. Build the current workspace coordinator on both platforms, retain mutation evidence, and keep production sources and generated Anvil artifacts unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run 42598115692 passed the full Windows tier in 2h20m. Linux completed mutation execution without runner OOM or a safety stop, but failed the 100% score gate at 99.0%: 548 survivors, five timeouts, and two individually memory-limited mutants. The recursive guard mutation was killed in 78 ms. Preserve reports in session artifacts and remove the temporary hosted harness.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the 100 percent retained-mutant gate. Exclude native ownership and I/O paths that need the deferred resource backend, add deterministic policy regressions, and avoid collecting disabled cfg tails.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Retain the existing whole-user resource limits and fail-closed observer. Finalize diagnostic evidence before removing the user slice so successful teardown cannot erase required counters.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
martin-kolinek and others added 12 commits November 9, 2026 22:00
Retain the external memory, swap, task, and observer limits while rerunning feature combinations and benchmark compilation. The unchanged product sources already passed the full Linux mutation gate; Windows validation continues in its existing run.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The two-job retry reached the protective stop with two rustc processes using about 6.4 GiB RSS combined. Keep the existing safety boundary and serialize Cargo compilation; retain memory.stat to distinguish resident allocations from charged file cache.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Linux mutation and remaining feature/benchmark checks passed in separate protected runs; Windows full exhaustive tier passed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep injected process-group guards, use upstream scripted build supervision, and adapt the listing deadline regression to separate output results.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run both full-workspace exhaustive groups. Preserve independent Linux observation, earlier reclaim, external memory/task limits, and serial compilation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove redundant local-item attribute visits, keep platform executable-name helpers cfg-gated, and make initial cache claims private. Strengthen cfg-tail and error diagnostic regressions and restore the Linux-only interrupt test gate.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exclude only the dispatch adapter's native resume error-to-panic mutation. Scripted resume policy tests still cover checkpoint and delegation error propagation without replacing the harness process.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the temporary hosted diagnostic harness after both exhaustive jobs pass. Exercise fresh campaign creation independently and gate explicit permission mutation behind system-tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep build-only helpers private and split independent injected directory-sync failures without changing runtime behavior or native test boundaries.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the scheduled Gamma 0.4.0 coordinator and strict gate. Scope one equivalent default and one native adapter mutation, and move platform selection to item cfgs so disabled tail expressions are not mutated.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 11, 2026 13:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The supplied review reports no active issues with the targeted mutation-testing changes.

No open issues 🎉

What changed in this PR

This PR addresses scheduled mutation-testing gaps in cargo-gamma-lib without changing runtime behavior or relaxing the mutation-score and zero-flaky gates.

Changes:

  • Splits platform-specific path comparison and transient-error classification into separately cfg-gated functions.
  • Adds targeted mutation-skip comments for an equivalent default replacement and the native canonicalization adapter, retaining scripted policy coverage.
  • Leaves generated workflows and Gamma 0.4.0 unchanged.
File Description
crates/​cargo-gamma-lib/​src/​exec/​workspace.rs Separates Windows and non-Windows path-identity functions.
crates/​cargo-gamma-lib/​src/​exec/​verdict.rs Separates Unix and non-Unix transient-error classifiers.
crates/​cargo-gamma-lib/​src/​exec/​testing_state.rs Adds a targeted skip comment for native canonicalization forwarding.
crates/​cargo-gamma-lib/​src/​cfg/​cfgs.rs Adds a targeted skip comment for an equivalent default-value mutation.

Review effort: Balanced

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.5%. Comparing base (a093f71) to head (ac54232).

Additional details and impacted files
@@          Coverage Diff          @@
##            main    #359   +/-   ##
=====================================
  Coverage   99.5%   99.5%           
=====================================
  Files        351     351           
  Lines      64349   64353    +4     
=====================================
+ Hits       64050   64054    +4     
  Misses       299     299           
Flag Coverage Δ
linux 99.4% <100.0%> (ø)
linux-arm 99.4% <100.0%> (ø)
scheduled 99.5% <100.0%> (+<0.1%) ⬆️
windows 99.5% <100.0%> (-0.1%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@martin-kolinek
martin-kolinek enabled auto-merge (squash) November 11, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants