Skip to content

test(cli): kill the surviving mutants of the framework context - #804

Merged
blafourcade merged 14 commits into
nextfrom
test/framework-survivors
Sep 9, 2026
Merged

test(cli): kill the surviving mutants of the framework context#804
blafourcade merged 14 commits into
nextfrom
test/framework-survivors

Conversation

@blafourcade

@blafourcade blafourcade commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

🎯 What & why

The framework mutation scope held at a floor of 69 with a third of its mutants alive: 1186 survived and 384 uncovered of 5649, the largest hole in the tree after #795 brought every other context to 85–97. A behaviour change in install, sync, restore or doctor could pass the suite.

🛠️ How it works

  • 584 new its across nine families (domain, adapters, install/setup, plugin, clean, status/doctor, restore/uninstall, marketplace flows, translators/shared), 40 new test files, 73 extended, 6 new doubles under tests/helpers/ports/. Each it is one intention in a describe by responsibility; whole-structure assertions.
  • Every new test was made red first by applying the mutation by hand and restored after; 892 red lines recorded, one sample per family in the last commit body.
  • No src/ change: git diff --stat origin/next...HEAD -- cli/src is empty.
  • Score 72.2 → 95.4 (run-mutation.mjs framework, 5388 killed, 0 timeouts, 225 survived, 36 uncovered). Floor 69 → 93.
  • Survivors left alive are argued equivalent by reason class in the issue thread (guards no registered profile can reach, isAiToolId over already-typed ids, fields never read back, fallbacks filtered on the next line, values just saved, swallowing catches, pure-domain no-ops).

🧪 How to verify

  • cd cli && node scripts/run-mutation.mjs framework --force → score ≥ 93.
  • git diff --stat origin/next...HEAD -- cli/src → empty.
  • Locally green: tsc, biome check . (904 files), arch (126), knip, unit+integration (4864), golden e2e, type honesty; pre-commit and pre-push ran on every commit.

⚠️ Heads-up

Two things the tests found in src/ and did not fix, filed as bugs: global/restore-all-use-case.ts:81 treats an interactive "select nothing" as "no selection" and restores everything; uninstall-mcp-exclusion-use-case.ts:40 rebuilds merge entries for files it never edited, so with the sectionKey: null every real install records, it strips nothing.

🔗 Linked issue

Closes #798

✅ I certify

  • I DO CERTIFY I READ EACH LINE OF THE PULL REQUEST BECAUSE I AM A SOFTWARE ENGINEER, NOT A AI PUPPY.

🤖 Generated with Claude Code

https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb

blafourcade and others added 11 commits September 9, 2026 19:34
@blafourcade @claude
test(cli): add the test doubles the framework survivors need
2574897 
Recording, choosing and checkbox prompters, a faulting file adapter, a stub asset provider and a stub AI tool, plus a save counter on the in-memory manifest repository and select and input recorders on the scripted prompter. Every double is consumed by the tests in the following commits.

Framework mutation score before this series: 72.2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
test(cli): kill the surviving mutants of the framework domain
7fb9e43 
Manifest, tool entries, tracked files, native registrations, mcp exclusions, serialization, setup flow, config capability, project context, tool recommendations, installed plugins and rules, markdown references, install scope, marketplace source drift and the plugin source resolver: 91 tests, each shown red first against the mutant it names.

Framework mutation score: 72.2 before the series, 95.4 after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
test(cli): kill the surviving mutants of the framework adapters
9b6dd6a 
Plugin distribution reader, user source references, manifest and user manifest repositories, manifest file io and the environment adapter: 34 tests, each shown red first against the mutant it names.

Framework mutation score: 72.2 before the series, 95.4 after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
test(cli): kill the surviving mutants of the install and setup use cases
45a06bc 
Runtime and IDE config install, AI and IDE tool install, config install, content sections, gitignore, init, setup and its tools, machine scope, marketplace source and project context steps, update decisions and one-tool update: 63 tests, each shown red first against the mutant it names.

Framework mutation score: 72.2 before the series, 95.4 after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
test(cli): kill the surviving mutants of the plugin use cases
39cb49b 
Plugin add, install, install from marketplace, remove, update, search, target resolution and helpers: 68 tests, each shown red first against the mutant it names.

Framework mutation score: 72.2 before the series, 95.4 after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
test(cli): kill the surviving mutants of the clean use cases
f17df2c 
Clean and clean user scope, including the shared reference guard: 61 tests, each shown red first against the mutant it names.

Framework mutation score: 72.2 before the series, 95.4 after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
test(cli): kill the surviving mutants of the status and doctor use cases
cc00161 
Status, status all, doctor, doctor all, registration, references, tracked files, merge files, layout and plugin checks: 65 tests, each shown red first against the mutant it names.

Framework mutation score: 72.2 before the series, 95.4 after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
test(cli): kill the surviving mutants of the restore and uninstall us…
3498f1e 
…e cases

Restore, restore all, restore all plugins, tool files, merge and regular files, restore decisions, tool distribution generation, uninstall, uninstall tools, ide, plugin and mcp exclusion: 76 tests, each shown red first against the mutant it names. The mcp exclusion use case had no test before.

Framework mutation score: 72.2 before the series, 95.4 after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
test(cli): kill the surviving mutants of the marketplace flows
15b5067 
Marketplace sync settings, remove and check: 60 tests, each shown red first against the mutant it names.

Framework mutation score: 72.2 before the series, 95.4 after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
test(cli): kill the surviving mutants of the translators and shared s…
5b6728b 
…teps

Built tree, flat and marketplace translators, project hooks materializer, ensure built marketplace, catalog identity, marketplace registration, project hooks removal, user scope files, cache purges, shared source references, plugin drift, plugin files and native calls: 66 tests, each shown red first against the mutant it names.

Framework mutation score: 72.2 before the series, 95.4 after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
test(cli): raise the framework mutation floor to 93
5fd5998 
Measured 95.4 after the survivor series (5388 killed, 225 survived, 36 uncovered of 5649), up from 72.2 (4029 killed, 50 timed out, 1186 survived, 384 uncovered). The floor is the measured score minus two.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade
blafourcade requested a review from a team as a code owner September 9, 2026 17:41
This was referenced Sep 9, 2026
Closed
Closed
@blafourcade @claude
Merge branch 'next' into test/framework-survivors
589561d 
The install use case lost its Environment in #802 (the token now travels as an argument), so
the merged test keeps every new case and drops the three that published a token through it,
and the adapter test loses the write it no longer has.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade blafourcade mentioned this pull request Sep 9, 2026
Merged
1 task
blafourcade and others added 2 commits September 9, 2026 19:56
@blafourcade @claude
test(cli): make two framework survivor tests separator-agnostic
9d5ee0b 
The in-memory file adapter normalises every key to forward slashes, so a listing is compared to the normalised form rather than to a platform join. The user source references adapter joins its own path, so the expected error messages are built with the same join and the unparsable-JSON reason is matched as a substring rather than a slash-specific regex. Both failed only on Windows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
@blafourcade @claude
Merge remote-tracking branch 'origin/next' into test/framework-survivors
2798832 
Brings in the catalog's recommended field under metadata; the seeded raw catalogs in the framework tests already spell it that way and every new test is kept.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60
blafourcade added a commit that referenced this pull request Sep 9, 2026
@blafourcade @claude
chore(ci): install expect without the runner's third-party apt source (…
1e8ce3f 
…#808)

The coverage job's `apt-get update` fails a few times a day on the Google Chrome source the
runner image ships, whose mirror serves a stale index ("Hash Sum mismatch"), and the whole
job goes red before a test runs; it hit #804 and #807 within an hour. `expect` comes from
ubuntu's own archive, so the chrome source is removed before the update.


Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
@blafourcade
blafourcade merged commit 0268042 into next Sep 9, 2026
36 checks passed
@blafourcade blafourcade mentioned this pull request Sep 10, 2026
Merged
1 task
blafourcade added a commit that referenced this pull request Sep 10, 2026
@blafourcade @claude
fix(cli): an interactive restore that ticks nothing restores nothing (#…
42f572c 
…813)

* fix(cli): an interactive restore that ticks nothing restores nothing

`RestoreAllUseCase` forwarded an empty checkbox answer as `files: []`, which the restore
reads as "no selection made", the same as a non-interactive run, and every drifted file was
restored. An empty selection now ends the run with nothing restored; a run where nothing
drifted, so no checkbox was shown, still restores the plugin files the picker never offers.

Red first: the new test failed with `expected '{ "respectGitignore": false, …' to be
'EDITED OUTSIDE THE CLI'`.

Closes #805

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60

* test(cli): the picker tests say what an empty selection now means

Two tests from #804 pinned the behaviour #805 fixes: an empty selection forwarded as
`files: []`. They now assert that the run never delegates when nothing was ticked, and that
a run with no drifted entry delegates with no selection at all.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x4ms5qcGuZgYhCxfdHMUb
AIDD-Session-Id: 4acc9a1c-19bc-4468-b8b6-e86644bcba60

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@blafourcade