Repository navigation
fix: stabilize scheduled checks and close Linux mutation coverage gaps - #357
Conversation
…stic Replace native process startup and subsecond timing assertions with the existing listing-supervision seam and an exhausted scripted output budget. Keep a live reader sender to model an escaped descendant without launching processes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exclude 19 expensive generated-text contract assertions only under Miri. Keep native/cargo-careful execution, artifact rendering and ownership checks, and parser/planner coverage unchanged; document the safety tradeoff. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover xversion as a missing version and a leading NUL as malformed YAML. Both exact prefix-stripping mutants from the scheduled Gamma campaign now fail their focused behavioral regressions; production parsing remains unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep complete native/cargo-careful failure-injection and adoption matrices. Under Miri retain the successful plan, first/middle/last resolver failures, both TOML shapes, Unicode, both newline styles, and both catalog orders without repeating every Cartesian combination. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Start the timeout assertion only after the contained descendant is ready, then exercise the real supervisor with a zero budget. Keep this OS acceptance behind system-tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep expensive planner policy matrices, YAML permutations, and generated-text scans native/cargo-careful only. Retain focused parser and lifecycle coverage, add a small BOM/Unicode/CRLF flow round trip, and shrink builder fixtures without skipping their contracts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Inject signalling into the rejected-ID tests while retaining native termination acceptance and mutation coverage. Linux killpg(1, SIGKILL) broadcasts beyond the test process tree. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run the unchanged Anvil group on ubuntu-latest and publish bounded host resource, process, kernel, and phase snapshots to a separate live check. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The active run retains its immutable observer and source snapshot. Keep the lasting change limited to safe process-group guard testing and its design documentation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Check the exact child arguments before launching the current harness. Add a construction-only regression for the empty-user-arguments path and require exactly one passing native child. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Capture synced pre-launch identities and spawned PIDs, preserve campaign journals without saved environments, and observe a fresh hosted runner outside a bounded workload scope. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run the campaign as a dedicated unprivileged user on the disposable hosted VM. Bound its entire root-owned user slice so automatic systemd sibling scopes cannot leave the memory/task ceiling; observe from outside that identity with no job token in the campaign. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The hosted image rejects MemoryOOMGroup as a systemctl assignment. Set and verify memory.oom.group through the root-owned slice directly, and keep diagnostic output writable by the runner and isolated campaign identity. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The runner home prevents the dedicated user from reaching the checkout. Grant only directory traversal on its ancestors and verify checkout/tool/output access before launching the protected campaign. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run 42267402403 continues at immutable commit ebadc79. Retain the production selection guards without diagnostic hooks or workflow; session artifacts preserve the harness and verified user-slice limits. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A benign Linux lookup probe confirms CARGO_HOME/bin precedes PATH. Install and byte-compare the diagnostic coordinator in the isolated user's Cargo bin before running the unchanged group, retaining the already verified user-slice limits. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The active hosted run retains the instrumented coordinator and independent observer at 493b9f7. Restore the lasting production fixes without leaving one-off workflow or tracing code. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Repeat the bounded hosted investigation with allowlisted baseline failure messages; do not publish saved environment overrides. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Retained baseline errors show the dedicated user's temporary root was group-writable. Set its mode explicitly and use a private campaign umask, preserving cache ownership checks and the root-managed resource boundary. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…umasks Use the existing private-directory helper before cache claiming, so preparation does not create and then reject its own redirected cache. Reuse the private temporary root for CLI fixtures. Remove ineffective diagnostic-only umask overrides. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep external boundary counter loss fatal and sample that boundary independently of currently observed child processes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Inject the native relaunch attempt into the once-only policy so guard mutations fail in-process instead of recursively creating sibling systemd scopes. Cover unmarked result propagation and document the non-spawning contract. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The protected campaign identified recursive relaunch mutant ddda37216735, ordinal 50167. Preserve the collected evidence as session artifacts and remove the temporary hosted workflow, external observer, and synced launch tracing. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reuse the independently enforced Linux user-slice limits and fail-closed observer while executing the unchanged full-workspace exhaustive recipe. Build the current workspace coordinator on both platforms, retain mutation evidence, and keep production sources and generated Anvil artifacts unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run 42598115692 passed the full Windows tier in 2h20m. Linux completed mutation execution without runner OOM or a safety stop, but failed the 100% score gate at 99.0%: 548 survivors, five timeouts, and two individually memory-limited mutants. The recursive guard mutation was killed in 78 ms. Preserve reports in session artifacts and remove the temporary hosted harness. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the 100 percent retained-mutant gate. Exclude native ownership and I/O paths that need the deferred resource backend, add deterministic policy regressions, and avoid collecting disabled cfg tails. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Retain the existing whole-user resource limits and fail-closed observer. Finalize diagnostic evidence before removing the user slice so successful teardown cannot erase required counters. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Cache preclaiming can leave newly created directories world-writable under umask 000, and cfg-disabled local items can incorrectly block declared-return-type propagation.
2 open issues
What changed in this PR
This PR stabilizes Gamma’s Linux validation and reduces Anvil’s Miri workload, while retaining the strict mutation gate for the documented mutation scope.
Changes:
- Makes process-supervision tests deterministic, isolates signalling and relaunch guards, and adds private cache creation during preparation.
- Fixes configuration-aware tail-expression selection and adds focused behavioral regressions with localized mutation exclusions.
- Reduces repeated Miri test matrices and generated-text scans, with corresponding design documentation.
| File | Description |
|---|---|
crates/cargo-heather/src/bin/cargo-heather/run.rs |
Documents native header-publication mutation exclusions. |
crates/cargo-gamma-unsafe/src/support.rs |
Excludes native capability discovery from mutation testing. |
crates/cargo-gamma-unsafe/src/interrupt.rs |
Adds registry regressions and targeted mutation exclusions. |
crates/cargo-gamma-unsafe/src/identity.rs |
Documents the native credential boundary. |
crates/cargo-gamma-unsafe/src/group.rs |
Tests signal guards through injected callbacks. |
crates/cargo-gamma-unsafe/src/cgroup.rs |
Adds cgroup policy regressions and native-boundary exclusions. |
crates/cargo-gamma-unsafe/docs/DESIGN.md |
Documents process-group isolation and mutation scope. |
crates/cargo-gamma-process/src/process_tree.rs |
Adds localized lifecycle mutation exclusions. |
crates/cargo-gamma-process/docs/IMPLEMENTATION.md |
Clarifies native lifecycle mutation boundaries. |
crates/cargo-gamma-lib/tests/cli.rs |
Uses private temporary CLI workspaces. |
crates/cargo-gamma-lib/src/paths.rs |
Documents a native path-resolution exclusion. |
crates/cargo-gamma-lib/src/exec/workspace.rs |
Uses private directory creation and platform-specific branches. |
crates/cargo-gamma-lib/src/exec/verdict.rs |
Guards native libtest launches against recursive selection. |
crates/cargo-gamma-lib/src/exec/testing_state.rs |
Extracts and tests executable-permission masking. |
crates/cargo-gamma-lib/src/exec/test_listing.rs |
Replaces timing-dependent listing coverage with scripted supervision. |
crates/cargo-gamma-lib/src/exec/sync.rs |
Adds synchronization failure regressions and targeted exclusions. |
crates/cargo-gamma-lib/src/exec/rustc_wrapper.rs |
Documents native capture-directory cleanup exclusions. |
crates/cargo-gamma-lib/src/exec/relaunch.rs |
Injects relaunch attempts and strengthens command tests. |
crates/cargo-gamma-lib/src/exec/measure.rs |
Documents native compiler-evidence publication exclusions. |
crates/cargo-gamma-lib/src/exec/manifest.rs |
Tests portable paths and documents native creation exclusions. |
crates/cargo-gamma-lib/src/exec/copy.rs |
Separates platform-specific symlink creation. |
crates/cargo-gamma-lib/src/exec/build/tests.rs |
Waits for descendant readiness before timeout acceptance. |
crates/cargo-gamma-lib/src/exec/build/splices.rs |
Documents native instrumentation-publication exclusions. |
crates/cargo-gamma-lib/src/exec/build/invoke.rs |
Extracts supervision of an already-adopted process tree. |
crates/cargo-gamma-lib/src/elements/publication.rs |
Adds localized filesystem-publication exclusions. |
crates/cargo-gamma-lib/src/discover/plan.rs |
Runs Windows package-URL coverage on every platform. |
crates/cargo-gamma-lib/src/discover/hints.rs |
Documents native hint-file read exclusions. |
crates/cargo-gamma-lib/src/discover/glob.rs |
Uses compile-time platform branches for normalization. |
crates/cargo-gamma-lib/src/discover/diff.rs |
Tests inventoried absolute paths without filesystem probes. |
crates/cargo-gamma-lib/src/commands/verdict_log.rs |
Documents native progress-log repair exclusions. |
crates/cargo-gamma-lib/src/commands/run.rs |
Documents native resume-relaunch exclusions. |
crates/cargo-gamma-lib/src/commands/dispatch.rs |
Documents native relaunch dispatch exclusions. |
crates/cargo-gamma-lib/src/ci/summary.rs |
Documents native summary-file opening exclusions. |
crates/cargo-gamma-lib/docs/design/system-tests.md |
Defines deterministic acceptance and mutation boundaries. |
crates/cargo-gamma-lib/docs/DESIGN.md |
Describes private directory creation during preparation. |
crates/cargo-gamma-engine/src/ops/collect/collector/tests.rs |
Adds configuration-aware typed-tail regression coverage. |
crates/cargo-gamma-engine/src/ops/collect/collector.rs |
Selects the last enabled statement for typed tails. |
crates/cargo-gamma-engine/src/model/identity.rs |
Excludes non-progress normalization mutations. |
crates/cargo-gamma-engine/docs/DESIGN.md |
Specifies configuration-aware return-type propagation. |
crates/cargo-each/src/run.rs |
Documents native PATH lookup exclusions. |
crates/cargo-coverage-gate/src/source_filter.rs |
Tests preserving workspace sections after compiler filtering. |
crates/cargo-coverage-gate/src/bin/cargo-coverage-gate/report_file.rs |
Documents native file-durability exclusions. |
crates/cargo-coverage-gate/src/bin/cargo-coverage-gate/collect.rs |
Tests fallback classification and separates platform cleanup messages. |
crates/cargo-aprz-lib/src/facts/crates/tables/table_mgr.rs |
Documents platform-specific archive-path mutation equivalence. |
crates/cargo-aprz-lib/src/facts/collector.rs |
Documents native cache-creation exclusions. |
crates/cargo-aprz-lib/src/facts/advisories/archive.rs |
Adds localized download and archive-path exclusions. |
crates/cargo-aprz-lib/src/commands/github_credentials.rs |
Extracts and tests executable-permission policy. |
crates/cargo-anvil/src/run.rs |
Reduces repeated planner matrices under Miri. |
crates/cargo-anvil/src/region.rs |
Excludes non-progress rendering mutations. |
crates/cargo-anvil/src/dependabot.rs |
Adds focused parsing tests and reduces Miri matrices. |
crates/cargo-anvil/src/catalog/builder.rs |
Uses a small catalog for builder tests. |
crates/cargo-anvil/src/anvil/artifacts/region.rs |
Skips exhaustive embedded-config checks under Miri. |
crates/cargo-anvil/src/anvil/artifacts/mod.rs |
Skips exhaustive catalog scans under Miri. |
crates/cargo-anvil/src/anvil/artifacts/justfile.rs |
Skips selected generated-recipe scans under Miri. |
crates/cargo-anvil/src/anvil/artifacts/github.rs |
Skips selected workflow-text checks under Miri. |
crates/cargo-anvil/src/anvil/artifacts/container.rs |
Skips selected container-text checks under Miri. |
crates/cargo-anvil/src/anvil/artifacts/ado.rs |
Skips selected pipeline-text checks under Miri. |
crates/cargo-anvil/docs/design/checks.md |
Documents retained Miri coverage and workload reductions. |
Review effort: Balanced
Keep injected process-group guards, use upstream scripted build supervision, and adapt the listing deadline regression to separate output results. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run both full-workspace exhaustive groups. Preserve independent Linux observation, earlier reclaim, external memory/task limits, and serial compilation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Normal cache-claim paths still leave new caches world-writable under umask 000, while temporary privileged diagnostic tooling and overly permissive diagnostic assertions remain unresolved.
5 open issues
Review effort: Balanced
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #357 +/- ##
=======================================
- Coverage 99.5% 99.5% -0.1%
=======================================
Files 351 331 -20
Lines 64328 61842 -2486
=======================================
- Hits 64029 61545 -2484
+ Misses 299 297 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Remove redundant local-item attribute visits, keep platform executable-name helpers cfg-gated, and make initial cache claims private. Strengthen cfg-tail and error diagnostic regressions and restore the Linux-only interrupt test gate. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exclude only the dispatch adapter's native resume error-to-panic mutation. Scripted resume policy tests still cover checkpoint and delegation error propagation without replacing the harness process. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Campaign-directory creation lacks independent test coverage, and the mode-preservation test introduces native permission dependencies into the default unit suite instead of being gated behind system-tests.
5 open issues
2 resolved since last review
Review effort: Balanced
Remove the temporary hosted diagnostic harness after both exhaustive jobs pass. Exercise fresh campaign creation independently and gate explicit permission mutation behind system-tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
[copilot speaking] [non-blocking] cargo-gamma-lib's floor is still |
Keep build-only helpers private and split independent injected directory-sync failures without changing runtime behavior or native test boundaries. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>


🤖
Summary
Mutation scope
Keep the strict 100% mutation score and zero-flaky gate. Add localized, reasoned exclusions for native resource/I/O boundaries requiring the deferred ownership/backend refactor, plus demonstrated equivalent or non-contract mutations. These boundaries are excluded, not claimed as mutation-covered; there are no blanket file/crate exclusions.
Complete exhaustive verification
Integrated
mainat08a51545in merge7202bbe9, preserving upstream scripted build supervision and macOS process support.Full-workspace exhaustive run 42812857111 passed on both platforms at
2c010a4a, including mutation testing, feature powersets, and benchmark compilation:Both platforms had zero survivors, timeouts, out-of-memory verdicts, or uncovered mutants. Jobs run in parallel, so exhaustive workflow wall time is approximately the longer job, not the sum.
Linux's independent containment observer finalized successfully: peak 7.7384 GiB charged memory, zero outer-boundary max/OOM/PID-limit events, no controlled stop, and campaign exit 0. Early reclaim remained enabled with unchanged external hard limits and serial compilation.
The final follow-up removes the two temporary diagnostic workflow/observer files and strengthens only the existing cache-permission acceptance regression: campaign creation is exercised before workspace preclaiming can create its directory, and the explicit existing-directory permission change is gated behind
system-tests. No production behavior changed after the passing exhaustive head. Fresh ordinary PR CI applies to that final follow-up; the complete exhaustive evidence above is specifically for2c010a4a.The earlier integrated run 42792368403 safely exposed two redundant visitor calls and one native resume-adapter survivor. Commits
ca336ae2and2c010a4aaddressed these before the successful full rerun. Focused Linux regressions and the native Windows diagnostic regression passed; full-workspace Clippy passed on both platforms.Miri evidence
Earlier profiled Miri run 41806126868 completed in Windows 1h07m03s, Linux 1h44m58s, Linux ARM 1h29m51s, and Windows ARM 1h56m34s. These measurements preceded upstream integration and are not a new current-head profile. Miri belongs to runtime analysis, not the exhaustive group above.
Runtime and diagnostic findings
Gamma is the largest part of exhaustive execution, but not the only long check: optimized benchmark compilation adds approximately 28 minutes on Windows and 49 minutes on Linux in the final measured run. Linux Gamma's actual testing phase took approximately 97m28s, with approximately 25m15s fixed preparation/build/baseline overhead.
The later Linux benchmark stops were compiler allocations plus charged file cache, not a renewed mutant process storm. Earlier cgroup reclaim resolved those stops without raising the existing hard limits. Temporary diagnostic harness settings are not new Anvil defaults, and the harness is removed from the final changes.
These are observed post-adoption runtimes, not a controlled speedup comparison against the previous mutation tool.