Repository navigation
Description
Pin small interfaces: allow_unknown, the CLI surface and parse-error positions
From the full mutmut run of fixes at 1c2f869 (27,670 of 32,420 mutants checked; see #308). Three small, mechanical gaps.
1. allow_unknown defaults on policy judges (18 survivors, 18 functions)
Flipping allow_unknown: bool = False to True survives on every judge (judge_analysis, judge_coverage, judge_formatting, judge_tests, judge_pinning, judge_settings, …). Production always passes the value explicitly (judging.py), and test_cli pins the unknown-is-error rule, so these are equivalent today. But a new caller that forgets the argument would silently allow unknowns.
Fix: make allow_unknown a required keyword argument (no default). This removes the hazard and the 18 mutants.
2. CLI surface (128 survivors, 18 functions)
| Function | Survivors | Example mutant |
|---|---|---|
cli._parser |
26 | -prog="project_validator", description="Validate a project's build, CI and tooling against best practices." | |
cli._add_check_options |
22 | -parser.add_argument("--build", action="store_true", help="configure and build the project to list its tests") |
scaffold.main |
16 | -parser = argparse.ArgumentParser(prog="python -m project_validator.scaffold", description=__doc__.split("\n") |
cli._add_output_options |
10 | -choices=("text", "json", "markdown"), || +choices=None, |
cli._add_scope_options |
8 | -metavar="DIR", || +metavar=None, |
cli._options |
8 | -build=arguments.build, || +build=None, |
cli._print_listing |
8 | -print(json.dumps(check_list(checks), indent=2)) || +print(json.dumps(check_list(checks), indent=None)) |
cli._add_listing_options |
6 | -help="print what each check recognises for each ecosystem (markdown; as JSON with --format json) and exit", \ |
cli._run |
5 | -print(json.dumps(_dump(context, arguments), indent=2)) || +print(json.dumps(_dump(context, arguments), inde |
cli._mebibytes |
4 | -size = 0 || +size = None |
cli._format_json |
3 | -return json.dumps(report_dicts(reports), indent=2) || +return json.dumps(report_dicts(reports), indent=None |
cli._format_list |
3 | -width = max((len(c.name) for c in checks), default=0) || +width = max((len(c.name) for c in checks), defaul |
--build'sstore_truecan becomeNone, so no test passes--build.- The
--format/--colorchoicescan be dropped. - The defaults (
path=.,--cmake,--ctest) can be dropped. - The JSON
indentcan change.
Fix:
- one test of
parse_args([])defaults; - one that rejects an invalid
--format; - one that passes
--build(it can mock the build); - a golden
--helpsnapshot; - a golden
--list --format jsonsnapshot.
3. Parse-error positions (23 survivors, 6 functions)
| Function | Survivors | Example mutant |
|---|---|---|
ecosystems.docker.dockerfile.parse |
11 | -raise ParseError(f"unknown instruction {instruction.keyword}", instruction.line, 1) || +raise ParseError(f" |
text.check_json |
7 | -raise JsonSyntaxError("malformed JSON", line, position - text.rfind("\n", 0, position)) || +raise JsonSynta |
build_systems.cmake.ctest._run |
2 | -raise BuildError(f"could not run {command[0]}: {error}") from error || +raise BuildError(None) from error |
build_systems.meson.syntax.parse |
1 | -raise MesonSyntaxError(parser.token.line, "nested too deeply to read") from error || +raise MesonSyntaxErro |
ecosystems.data._check_deep_json |
1 | -raise ParseError(str(error), error.line, error.column) from error || +raise ParseError(str(None), error.lin |
ecosystems.haskell.cabal._lines |
1 | -raise CabalSyntaxError(f"line {number}: tab in indentation") || +raise CabalSyntaxError(None) |
Dockerfile and check_json errors are only checked with assertRaises. Their line and column, which users see, can change without failing.
Fix: assert (line, column) in parse-error tests.
🤖 Generated with Claude Code
Activity
Test that every policy judge requires allow_unknown
Test that --allow-unknown relaxes an unknown ci-lint answer
Require allow_unknown on every policy judge, and pass it from every c…
Pin the command line's surface: defaults, options, choices, help and …
Assert where parse errors are reported, and what they say
Test that content after a JSON value is reported where it starts
Report content after a JSON value where that content starts
leftibot commentedon Nov 18, 2026
Progress in leftibot#80:
allow_unknownis now a required keyword on every judge, including theJudgeprotocol, with a contract test (tests/test_judges.py). Making it required showed a real bug: theci-lintcheck never passed--allow-unknownto its judge, so an unknown answer stayed an error. It is fixed, with a test.action-pinning,secret-scanningandlicensecan never be unknown; they take the argument and discard it, like the rest.tests/test_cli_options.pypins:Optionseach flag builds;--root, and the label and suffixes in the files it writes.tests/test_parse_errors.pyasserts line, column and message for JSON, Dockerfile, Meson, Cabal and build-command errors. It found a bug: JSON trailing content was reported at the end of the value, not where the content starts. Fixed.Nothing from this issue remains open.
🤖 Generated with Claude Code
https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
leftibot commentedon Nov 19, 2026
Fixed on main (as of e2e2286). Every problem listed here is covered by:
Checked against the commit messages and the regression suite (no expected failure is left for this issue). Reopen if a case is still wrong.
🤖 Generated with Claude Code
https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt