Skip to content

Alternative spellings and action-input defaults are untested (297 surviving mutants) #302

Open

Description

@leftibot

Alternative spellings and action-input defaults are untested

From the full mutmut run of fixes at 1c2f869 (27,670 of 32,420 mutants checked; see #308).

275 surviving mutants in 107 functions drop one of several accepted spellings, and nothing fails:

  • an option alias (-c/--config, -cp/--classpath, -b/--build);
  • a program variant (mypy/dmypy, msbuild/MSBuild);
  • a subcommand (checkFormat/checkFormatAll).

A further 22 survivors drop the default of a GitHub Action input or a configuration key (inputs.get("folder-path", "."), preset.get("inherits", [])).

Each alias was added because real projects use it. Without a test, a refactor can drop it silently, and a project that uses that spelling flips from PASS to FAIL.

Functions with the most survivors

Function Survivors Example mutant
tools.coverage._lcov 11 read = _configuration("lcov", command, facts, _named(_LCOV_PROGRAMS, "--config-file")(command), ".lcovrc")
ecosystems.swift._swiftlint 10 fixes = has_option(argv, "--fix", "--autocorrect") or argv[1:2] in (("--fix",), ("autocorrect",))
facts.formatting._keeps 8 if argv[1] == "commit" or (argv[1] == "diff" and has_option(argv, "--exit-code", "--quiet")):
tools.coverage._llvm_cov 8 shown = has_option(options, "-show-branches", "--show-branches")
ecosystems.openapi._oasdiff_action 6 return (("oasdiff", "breaking", inputs.get("base", ""), inputs.get("revision", "")),)
ecosystems.php.tools._read_phpmd 6 rules = tuple(rule.get("ref", "") for rule in root.iter("rule"))
ecosystems.jvm.fuzzing._names_a_class_or_jar 6 return has_option(argv, "--target_class", "--cp", "-cp", "--classpath") or any(
ecosystems.dotnet.tools._nuget_audit 6 restores = starts_with(argv, ("msbuild",), ("MSBuild",), *(("dotnet", restore) for restore in _RESTORES))
ecosystems.docker.tools._scout_action 6 commands = inputs.get("command", "quickview").split(",")
ecosystems.common_lisp._run_tests_action 5 system = inputs.get("asdf-system", "")
ecosystems.jvm._wired 5 plugins = facts.where(Setting, tool="Maven", key=f"plugin.{wiring.maven_plugin}")
ecosystems.dotnet.tools._coverlet 5 if any(collector.strip("\"'").lower() == "xplat code coverage" for collector in collectors):
ecosystems.javascript.analysis._tsconfig 5 named = option_values(resolved(command), "-p", "--project", "-b", "--build")
build_systems.meson.evaluation.Evaluation.target 4 return single(Object("target", name or ""))
build_systems.meson.evaluation.Evaluation.run_target 4 return single(Object("run_target", names[0] if names else ""))
ecosystems.scala._mill_scalafmt 4 if mill("checkFormat", "checkFormatAll")(command):
ecosystems.dart._dart_format 4 if has_option(argv, "--output", "-o") and any(word in ("none", "show", "json") for word in argv[2:]):
ecosystems.sql._squawk_action 4 targets = inputs.get("files", "").split() or [inputs.get("pattern", "migrations/*.sql")]
ecosystems.kubernetes._kube_linter_action 4 configuration = ("--config", inputs["config"]) if inputs.get("config") else ()
ecosystems.r._rcmdcheck 4 args = inputs.get("args", 'c("--no-manual", "--as-cran")')
ecosystems.haskell.tools._hpc 4 instrumented = (_builds(argv, "test", "build", "run") and has_option(argv, "--enable-coverage", "--coverage"))
ecosystems.haskell.options._hpack_component 4 condition = frozenset({str(when.get("condition", ""))})
ecosystems.jvm.analysis._checkstyle_rules 4 for setting in facts.where(Setting, tool="Checkstyle", key="modules"):
ci.gitlab._job_facts 3 tags = json.dumps(job.get("tags", defaults.get("tags", [])), default=str)
ecosystems.documentation._markdown_link_check_action 3 return (("markdown-link-check", inputs.get("folder-path", ".")),)
ecosystems.dart._publishable 3 return str(_pubspec(text).get("publish_to", "")).strip("'\"") != "none"
ecosystems.graphql._graphql_inspector_action 3 return (("graphql-inspector", "diff", *inputs.get("schema", "").split()),)
ecosystems.haskell.options.ghc_options 3 negated(requiring(found.removeprefix("not "))) if found.startswith("not ") else requiring(found)
ecosystems.jvm.analysis._read_checkstyle 3 names = tuple(module.get("name", "") for module in root.iter("module"))
ecosystems.dotnet.tools._sdks 3 imported = (element.get("Sdk") for element in root.iter("Import"))
ecosystems.dotnet.tools._read_run_settings 3 names = (collector.get("friendlyName", "") for collector in root.iter("DataCollector"))
ecosystems.ruby.tools._rake_tasks 3 defaults = (found.value for found in facts.where(Setting, tool="Rake", key="default"))
pretty.print_reports 2 console.print(text.Text(f"{'':{_BADGE_WIDTH + 1}}{detail}", style="dim"))
pretty._summary 2 return text.Text("no checks were run", style="dim")
build_systems.cmake.ctest._registered 2 if facts.where(Setting, tool="CTest", key="enables"):

Action-input and configuration defaults

Function Survivors Example mutant
ci.gitlab._job_facts 4 tags = json.dumps(job.get("tags", defaults.get("tags", [])), default=str)
ecosystems.julia._runtest 4 depwarn = f"--depwarn={inputs.get('depwarn', 'yes')}"
build_systems.cmake.presets._inherited 2 inherits = preset.get("inherits", [])
build_systems.cmake.presets._variables 2 own = presets[name].get("cacheVariables", {})
ci.github_actions._with_inputs 2 return _INPUT.sub(lambda match: inputs.get(match[1], ""), value) # type: ignore[return-value]
ecosystems.python.hatch._environments 2 document = tool.get("hatch", {}) if isinstance(tool, Mapping) else {}
ecosystems.python.tox._toml_tables 2 document = document.get("tool", {}).get("tox") if isinstance(document, Mapping) else None
ci.github_actions._runner_os 1 runner = substitute_matrix(json.dumps(job.get("runs-on", "")), combination)
ecosystems.r._rcmdcheck 1 args = inputs.get("args", 'c("--no-manual", "--as-cran")')
ecosystems.common_lisp._run_tests_action 1 system = inputs.get("asdf-system", "")
ecosystems.haskell.options._hpack_component 1 condition = frozenset({str(when.get("condition", ""))})

Proposal

  • Parametrise the claim tests over every alternative spelling each reader lists (subTest per alias). A shared helper could read the reader's own alias tuples, so a newly added alias is tested automatically.
  • For each action reader, add one test with the input omitted and assert the default the reader assumes (markdown-link-check with no folder-path checks .).

🤖 Generated with Claude Code

https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt

Activity

  1. leftibot commented on Nov 18, 2026

    @leftibot
    CollaboratorAuthor

    Progress in leftibot#80:

    tests/test_aliases.py checks that every alternative spelling reads the same as its siblings and differently from leaving the option out:

    • lcov reporting and measuring, and branch and condition coverage by option or --rc;
    • llvm-cov's single- and double-dash options;
    • tsc's project and build options;
    • the JVM Jazzer's class, classpath and jar;
    • NuGet auditing on any restore;
    • mill's format checks;
    • dart format's output options;
    • Haskell coverage on any build.

    The action readers' defaults and inputs are pinned. It found that dart format --output=none (the option joined to its value) counted as a rewriting run. Fixed.

    Remaining: the alias tuples of the other readers on the list.

    🤖 Generated with Claude Code

    https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt

  2. added
    area: test-suiteThe validator's own tests, mutation testing and fuzzing
    trackingRoadmap or umbrella issue tracking several pieces of work
    on Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: test-suiteThe validator's own tests, mutation testing and fuzzingtrackingRoadmap or umbrella issue tracking several pieces of work

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions