Repository navigation
Open
Labels
area: test-suiteThe validator's own tests, mutation testing and fuzzingThe validator's own tests, mutation testing and fuzzingtrackingRoadmap or umbrella issue tracking several pieces of workRoadmap or umbrella issue tracking several pieces of work
Description
Alternative spellings and action-input defaults are untested
From the full mutmut run of fixes at 1c2f869 (27,670 of 32,420 mutants checked; see #308).
275 surviving mutants in 107 functions drop one of several accepted spellings, and nothing fails:
- an option alias (
-c/--config,-cp/--classpath,-b/--build); - a program variant (
mypy/dmypy,msbuild/MSBuild); - a subcommand (
checkFormat/checkFormatAll).
A further 22 survivors drop the default of a GitHub Action input or a configuration key (inputs.get("folder-path", "."), preset.get("inherits", [])).
Each alias was added because real projects use it. Without a test, a refactor can drop it silently, and a project that uses that spelling flips from PASS to FAIL.
Functions with the most survivors
| Function | Survivors | Example mutant |
|---|---|---|
tools.coverage._lcov |
11 | read = _configuration("lcov", command, facts, _named(_LCOV_PROGRAMS, "--config-file")(command), ".lcovrc") |
ecosystems.swift._swiftlint |
10 | fixes = has_option(argv, "--fix", "--autocorrect") or argv[1:2] in (("--fix",), ("autocorrect",)) |
facts.formatting._keeps |
8 | if argv[1] == "commit" or (argv[1] == "diff" and has_option(argv, "--exit-code", "--quiet")): |
tools.coverage._llvm_cov |
8 | shown = has_option(options, "-show-branches", "--show-branches") |
ecosystems.openapi._oasdiff_action |
6 | return (("oasdiff", "breaking", inputs.get("base", ""), inputs.get("revision", "")),) |
ecosystems.php.tools._read_phpmd |
6 | rules = tuple(rule.get("ref", "") for rule in root.iter("rule")) |
ecosystems.jvm.fuzzing._names_a_class_or_jar |
6 | return has_option(argv, "--target_class", "--cp", "-cp", "--classpath") or any( |
ecosystems.dotnet.tools._nuget_audit |
6 | restores = starts_with(argv, ("msbuild",), ("MSBuild",), *(("dotnet", restore) for restore in _RESTORES)) |
ecosystems.docker.tools._scout_action |
6 | commands = inputs.get("command", "quickview").split(",") |
ecosystems.common_lisp._run_tests_action |
5 | system = inputs.get("asdf-system", "") |
ecosystems.jvm._wired |
5 | plugins = facts.where(Setting, tool="Maven", key=f"plugin.{wiring.maven_plugin}") |
ecosystems.dotnet.tools._coverlet |
5 | if any(collector.strip("\"'").lower() == "xplat code coverage" for collector in collectors): |
ecosystems.javascript.analysis._tsconfig |
5 | named = option_values(resolved(command), "-p", "--project", "-b", "--build") |
build_systems.meson.evaluation.Evaluation.target |
4 | return single(Object("target", name or "")) |
build_systems.meson.evaluation.Evaluation.run_target |
4 | return single(Object("run_target", names[0] if names else "")) |
ecosystems.scala._mill_scalafmt |
4 | if mill("checkFormat", "checkFormatAll")(command): |
ecosystems.dart._dart_format |
4 | if has_option(argv, "--output", "-o") and any(word in ("none", "show", "json") for word in argv[2:]): |
ecosystems.sql._squawk_action |
4 | targets = inputs.get("files", "").split() or [inputs.get("pattern", "migrations/*.sql")] |
ecosystems.kubernetes._kube_linter_action |
4 | configuration = ("--config", inputs["config"]) if inputs.get("config") else () |
ecosystems.r._rcmdcheck |
4 | args = inputs.get("args", 'c("--no-manual", "--as-cran")') |
ecosystems.haskell.tools._hpc |
4 | instrumented = (_builds(argv, "test", "build", "run") and has_option(argv, "--enable-coverage", "--coverage")) |
ecosystems.haskell.options._hpack_component |
4 | condition = frozenset({str(when.get("condition", ""))}) |
ecosystems.jvm.analysis._checkstyle_rules |
4 | for setting in facts.where(Setting, tool="Checkstyle", key="modules"): |
ci.gitlab._job_facts |
3 | tags = json.dumps(job.get("tags", defaults.get("tags", [])), default=str) |
ecosystems.documentation._markdown_link_check_action |
3 | return (("markdown-link-check", inputs.get("folder-path", ".")),) |
ecosystems.dart._publishable |
3 | return str(_pubspec(text).get("publish_to", "")).strip("'\"") != "none" |
ecosystems.graphql._graphql_inspector_action |
3 | return (("graphql-inspector", "diff", *inputs.get("schema", "").split()),) |
ecosystems.haskell.options.ghc_options |
3 | negated(requiring(found.removeprefix("not "))) if found.startswith("not ") else requiring(found) |
ecosystems.jvm.analysis._read_checkstyle |
3 | names = tuple(module.get("name", "") for module in root.iter("module")) |
ecosystems.dotnet.tools._sdks |
3 | imported = (element.get("Sdk") for element in root.iter("Import")) |
ecosystems.dotnet.tools._read_run_settings |
3 | names = (collector.get("friendlyName", "") for collector in root.iter("DataCollector")) |
ecosystems.ruby.tools._rake_tasks |
3 | defaults = (found.value for found in facts.where(Setting, tool="Rake", key="default")) |
pretty.print_reports |
2 | console.print(text.Text(f"{'':{_BADGE_WIDTH + 1}}{detail}", style="dim")) |
pretty._summary |
2 | return text.Text("no checks were run", style="dim") |
build_systems.cmake.ctest._registered |
2 | if facts.where(Setting, tool="CTest", key="enables"): |
Action-input and configuration defaults
| Function | Survivors | Example mutant |
|---|---|---|
ci.gitlab._job_facts |
4 | tags = json.dumps(job.get("tags", defaults.get("tags", [])), default=str) |
ecosystems.julia._runtest |
4 | depwarn = f"--depwarn={inputs.get('depwarn', 'yes')}" |
build_systems.cmake.presets._inherited |
2 | inherits = preset.get("inherits", []) |
build_systems.cmake.presets._variables |
2 | own = presets[name].get("cacheVariables", {}) |
ci.github_actions._with_inputs |
2 | return _INPUT.sub(lambda match: inputs.get(match[1], ""), value) # type: ignore[return-value] |
ecosystems.python.hatch._environments |
2 | document = tool.get("hatch", {}) if isinstance(tool, Mapping) else {} |
ecosystems.python.tox._toml_tables |
2 | document = document.get("tool", {}).get("tox") if isinstance(document, Mapping) else None |
ci.github_actions._runner_os |
1 | runner = substitute_matrix(json.dumps(job.get("runs-on", "")), combination) |
ecosystems.r._rcmdcheck |
1 | args = inputs.get("args", 'c("--no-manual", "--as-cran")') |
ecosystems.common_lisp._run_tests_action |
1 | system = inputs.get("asdf-system", "") |
ecosystems.haskell.options._hpack_component |
1 | condition = frozenset({str(when.get("condition", ""))}) |
Proposal
- Parametrise the claim tests over every alternative spelling each reader lists (
subTestper alias). A shared helper could read the reader's own alias tuples, so a newly added alias is tested automatically. - For each action reader, add one test with the input omitted and assert the default the reader assumes (
markdown-link-checkwith nofolder-pathchecks.).
🤖 Generated with Claude Code
Metadata
Metadata
Assignees
Labels
area: test-suiteThe validator's own tests, mutation testing and fuzzingThe validator's own tests, mutation testing and fuzzingtrackingRoadmap or umbrella issue tracking several pieces of workRoadmap or umbrella issue tracking several pieces of work
Activity
Test every alternative spelling a reader accepts, and the action read…
dart format only shows with --output joined to its value too
leftibot commentedon Nov 18, 2026
Progress in leftibot#80:
tests/test_aliases.pychecks that every alternative spelling reads the same as its siblings and differently from leaving the option out:--rc;The action readers' defaults and inputs are pinned. It found that
dart format --output=none(the option joined to its value) counted as a rewriting run. Fixed.Remaining: the alias tuples of the other readers on the list.
🤖 Generated with Claude Code
https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt