Repository navigation
Merged
Conversation
A symlink whose target does not exist was listed as a file, and the first content probe that opened it raised FileNotFoundError, so the extraction failed and nothing was judged (argo-cd, jekyll and os-lib keep such symlinks as test fixtures). Outside a git checkout the walk now skips them; the git listing already kept only real files. Part of cpp-best-practices#180 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
ToolRun.covers asks owner() for every (run, unit) pair, and owner() walked every unit through PurePosixPath, so a monorepo of U units and R runs cost O(U²·R) path operations: nushell took 208 s, uutils/coreutils 558 s, haskell/cabal and rust-lang/cargo more than ten minutes. within() is now a string-prefix test on the facts' normalised paths and owner() is cached per directory and set of units: nushell 4 s, coreutils 14 s, cabal 20 s, cargo 15 s, with the same results. cabal and cargo join the corpus. Part of cpp-best-practices#181 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
The splitter stripped `if`/`then` but not `[[`, so `if [[ "$TOTAL" -eq 0 || "$TOTAL" -eq 1 ]]` became a command whose
program is $TOTAL, an unknown when nothing in the job is seen to set it (148 ERROR lines in uutils/coreutils); a test
continued on the next line made every continuation line a command (Humanizer, scikit-learn, Homebrew), and
`repos=("$X")` a command named after its first element. A conditional expression now runs nothing, over as many
lines as it takes, and `name=(…)` is an assignment.
Part of cpp-best-practices#190
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
The body of `cat > f <<'EOF' … EOF` was split into words like any other line, so prose became commands (llama.cpp's 96 ERROR lines quoting a story), an apostrophe in it opened a quote the lexer could not close (cmake_template's 22 checks in ERROR), and Lisp forms in package-lint's `$(cat - <<EOF …)` were unknown programs. The lines after `<<WORD` up to the line that is `WORD` are now dropped before the script is split; here-strings (`<<<`) are untouched. Part of cpp-best-practices#191 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
GitHub's `$/.github/actions/x` and `$/.github/workflows/w.yml` name an action or reusable workflow of the same repository at the triggering commit; the Actions reader already followed them as local, but the pinning question only knew `./`, so junit5 (18 of 55), koreader (17 of 17), pytest, poetry, vite and rspec were told to pin them. Part of cpp-best-practices#209 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
The local action's path was `f"{directory}/action.yml"`, and the root's directory is the empty string, so `uses: ./`
looked for `/action.yml` and every GitHub Action repository's self-test was "not read" on every check that reads CI
commands (1656 ERROR lines in docker/build-push-action). The root action is `action.yml`; a composite action's steps
are now read as the job's commands. The GitHub Action fixture's snapshots lose the unknowns and gain those commands.
Part of cpp-best-practices#210
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
`a && b` and `a || b` evaluated to true or false, but GitHub yields the operand that decides them, so ternaries
rendered as booleans: fmt's `CXXFLAGS="${{ matrix.sanitizers && '-fsanitize=address,undefined' || '' }}"` became
`CXXFLAGS=" true"` (its two sanitizer builds unseen), mkdocs' `+py=${{ matrix.py || matrix.python-version }}` became
`+py=true`. The short-circuits a condition relies on when a side is unknown are kept. A runner given by such an
expression (`runs-on: ${{ matrix.os || 'ubuntu-latest' }}`) is now evaluated before its OS is named, so flask's
Windows and macOS jobs are no longer linux.
Part of cpp-best-practices#214
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…bytes The pattern held three literal backspace characters where `\b` was meant, so `error_prone_core` and `-Xplugin:ErrorProne` could never match and a Maven build running Error Prone (guava) had "no static analyzer". Part of cpp-best-practices#251 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
`.github/renovate.json5` (prettier, vite, okhttp, junit5, opentelemetry-proto, clap, starship, ruff), `.gitlab/ renovate.json(5)` and `.renovaterc.json5` were not looked at, so those projects had "no automated dependency updates". Scala Steward, also named in the issue, is not covered here. Part of cpp-best-practices#245 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
rbenv's and bash-it's CI run `test/run`, a script that ends in `exec bats …`; the script was followed but `exec` was taken for the program, so bats never ran as far as tests-run could see. `exec` is a launcher, like `env` and `nice`. Part of cpp-best-practices#193 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
Composer scripts whose entry is `@php vendor/bin/phpstan analyse` (monolog, composer) and steps running `php "vendor/bin/phpunit"` (PHP_CodeSniffer) recorded a command whose program is `php`, so PHPStan and PHPUnit never ran as far as the checks could see. `php` is now a launcher of the script after its options, and symfony's `simple-phpunit` (composer) is one of PHPUnit's program names, for tests-run as for coverage. Part of cpp-best-practices#200 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
spf13/cobra runs its tests through richgo, which only colours go test's output; tests-run, coverage and the race detector now recognise it wherever they recognise `go test`. Part of cpp-best-practices#259 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
mkdocs keeps its tests in `mkdocs/tests/*_tests.py` and discovers them with `unittest discover -p "*tests.py"`; only `test_*.py` and `*_test.py` were test files, so it had "no tests found". The suffix counts under a `test`/`tests` directory, where it cannot be mistaken for a module named that way. Part of cpp-best-practices#255 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…hout a space nushell runs every lint job as `cargo --config lints.toml clippy …`: only a toolchain (`+nightly`) was skipped before the subcommand, so clippy "did not run". serde denies `-Dclippy::pedantic`, which the `\bclippy::pedantic` pattern missed (no word boundary after `-D`). A shared `cargo_argv` now drops the toolchain and cargo's global options for the clippy and unsafe-code matchers, and the lint-settings pattern allows the same options. Part of cpp-best-practices#287 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
ripgrep was credited "-D warnings" from `RUSTDOCFLAGS: -D warnings` on its `cargo doc` step, which denies rustdoc's lints only; nothing passed it to rustc or clippy. A line that sets RUSTDOCFLAGS no longer counts. Part of cpp-best-practices#287 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
Rust had no warnings-as-errors practice, so the check was "not applicable to Rust" although tokio and serde set `RUSTFLAGS: -Dwarnings` on every job (and ripgrep's failure hint, from its shell scripts, said to run shellcheck). Denying warnings anywhere the build or CI reads it now passes: RUSTFLAGS, clippy's `-D warnings`, `[lints.rust] warnings = "deny"`, `#![deny(warnings)]`. The Rust fixtures' snapshots gain the result (well-configured PASS, bare FAIL); the profile page documents it. Part of cpp-best-practices#287 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
… || yields its operand
`java-version: ${{ matrix.java || '17' }}` (scalameta/scalafmt) rendered as "true" before && and || returned the
deciding operand; it now names JDK 17 and JDK 21, so the regression test's decorator goes. Also wraps two test lines
the renamed Python test-file pattern made too long.
Part of cpp-best-practices#274
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
rustup runs the channel `rust-toolchain.toml` pins for every plain `cargo`, but a job that named no version read as "Rust default" (nushell pins 1.97.1). A profile's toolchain patterns can now name the files that pin the default (`pinned_by`, read once per project into the CI jobs); Rust names `rust-toolchain.toml` and the older `rust-toolchain`. Part of cpp-best-practices#287 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…n it names none mlugg/setup-zig with no `version` installs the manifest's `minimum_zig_version`, but such jobs (zig-clap, zls, libxev, ghostty) read as "Zig default". The Zig profile now names build.zig.zon as the file that pins it. Part of cpp-best-practices#288 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
`role()` stopped at the first source root (`src`, `pkg`, `internal`), because the directory names below one are the code's own (`com/example/`). Test data kept there was therefore product data: deliberately malformed fixtures failed data-wellformed (golangci-lint's `pkg/…/testdata/`, liquibase's `src/test/resources/`, vite's `src/node/…/__tests__/ fixtures/`) and fixture projects became product units. Below a source root, a test directory (`test`, `tests`, `testdata`, `__tests__`, `fixtures`) now gives the test role; nothing else does. The JVM and Scala fixtures' test sources change role in the facts snapshots; no result changes. vite keeps the issue for its `playground/`. Part of cpp-best-practices#184 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
A `.dir-locals.el` holds an editor's per-directory settings; it made clj-kondo, GNOME/glib, rubocop, frog and others Emacs Lisp projects, failing six checks each for a language they do not have. Editor settings files are no sources. The rest of the issue (languages found only in fixtures, tooling or docs assets) is left: it would change the rule that every non-vendored source is the product's until `--auxiliary` names it. Part of cpp-best-practices#182 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
dbus-broker's CI passes `${CI_MESON_ARGS}`, a workflow variable holding `-D werror=true`; the variable was resolved,
but only `-Dwerror=true` in one word counted, so warnings-as-errors failed. Meson takes `-D`'s option in the same word
or the next, for `werror` and `warning_level` alike.
Part of cpp-best-practices#196
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…unknown Recipes were split with every `$(VAR)` kept as written, so `GO ?= go` then `$(GO) test -race -coverprofile …` (gin, protovalidate), `BIN := ./_boot/dune.exe` then `$(BIN) runtest` (dune) and `make test-with-flags TEST_FLAGS=…` (golang-migrate) ran nothing the checks knew, and no unknown said so. The makefile's assignments (`=`, `:=`, `?=`, `+=`, with `override` or `export`), make's built-in variables (`CC`, `CXX`, `AR`, `RM`…) and the command line's `NAME=value` are now expanded into each recipe, and a recipe whose program is still a variable is an unknown naming it. Two Make tests expected `$(CC)` literally; they now expect the compiler their makefile sets. Part of cpp-best-practices#197 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…n rules, CMake's own Four gaps, each silent until now. A rule line ending in a comment (`lint: deps ## Lint the spec`, the self-documenting idiom of digitalocean/openapi and docker-stacks) was not a rule. `include other.mk` was not followed, and a goal no rule defined expanded to nothing; included makefiles the project holds are now read and an undefined goal is an unknown (cowboy's erlang.mk targets). Pattern rules (`%:` of sphinx-quickstart's Makefile, clap's `lint-%:`) now match the goals they cover, with `$@` and `$*` filled in. And make in a directory with no makefile but a CMakeLists.txt (uber/h3's in-source `cmake .`) builds that directory's CMake targets instead of being unknown. A CMake test that listed `make lint` among the commands the CMake expander ignores no longer lists it. Part of cpp-best-practices#198 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…ow-unknown
With `--allow-unknown`, an unknown answer of Swift's hardening practice (it forbids `-Ounchecked`) fell through to
FAIL with nothing after the colon ("runtime safety checks are compiled away: ") whenever an unrelated reusable
workflow was unread (swift-argument-parser, swift-nio, vapor, SwiftLint, dart-lang/http, bloc). Judged by what could
be read, nothing forbidden found is a pass.
Part of cpp-best-practices#243
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
pyrightconfig.json (llama.cpp), turbo.json (graphql-eslint), .luarc.json (koreader), asv.conf.json (scikit-learn) and Nerdbank.GitVersioning's version.json (Humanizer) are documented to take `//` comments and trailing commas, and their tools read them so; data-wellformed failed each project for one. They join tsconfig.json and the other known names, with .eslintrc.json, biome.json and deno.json. Part of cpp-best-practices#232 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…urses The corpus's first CI run recorded "1:100001: Expecting value" for JSONTestSuite's 100000-deep array where the local recording had "1:100001: malformed JSON": the JSON reader falls back to its non-recursive checker only when Python's decoder raises RecursionError, which depends on the interpreter. The test fixes the decoder's behaviour each way with a mock and asks for one report; it is an expected failure until the reader decides by depth. Part of cpp-best-practices#290 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
… decoder would do The reader tried Python's decoder first and fell back to its own non-recursive checker on RecursionError; whether the decoder recursed that deep depended on the interpreter, so the same file was "Expecting value" on one machine and "malformed JSON" on another. Nesting is now measured first (one pass, strings skipped) and anything deeper than 500 goes straight to the checker. Part of cpp-best-practices#290 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…itten A glob entry ignored an output only when its text was one of the spellings the output names, so `*.pyc*` (scikit-learn), `site*/` (mkdocs), `blib*`/`cover_db*` (tidyall), `*.gz` (l3build), pandoc's whitelist (`/*` then `!tracked`), deno std's `sub/target` and magit's `/lisp/*.elc` all "did not ignore" what they ignore. An entry is now also read as git reads it (`*` within a component, `**` across, a trailing `/` for directories, a slash anchoring it) against the output's paths placed where the language writes them: its units' directories, and, for a file output, its sources' directories. `src/build/` still does not ignore a build directory at the root. Part of cpp-best-practices#236 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…tten Only the root ignore files were read, and an output was placed at the project's root, so a unit's own .gitignore (PostgREST's and golangci-lint's docs/.gitignore, koyo's and laravel's npm units, guava's Gradle integration tests) and a root entry anchored to the unit (`docs/_build/`, rich) did not count. An output is now placed in the units that write it (those built with its build system, else those of its language; a format's beside its own files, Sphinx's _build next to its conf.py), and an ignore file of the same name in a directory above it applies there. A nested file's entry is reported with its path. Part of cpp-best-practices#237 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
JSON and deep-JSON errors give the line and column of the token that breaks the value, Dockerfile problems the instruction's line, Meson and Cabal errors name the line, and a failing build command reports its last five lines of errors, or names the program it could not run. Until now these were only checked with assertRaises, so the mutation run's 23 survivors could move them freely. Part of cpp-best-practices#306 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
check_json reports trailing content at the end of the value ("[1]\n x" at 1:4) instead of where
the content that breaks it starts (2:2), as its docstring promises. Found while asserting
parse-error positions.
Part of cpp-best-practices#306
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
check_json skips the whitespace after a complete value before reporting trailing content, so the line and column point at the content that breaks the file, as its docstring says. Part of cpp-best-practices#306 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
Near-miss tests for the recognisers with the most surviving condition mutants: - golangci-lint configurations that disable go vet or staticcheck, turn the defaults off, or enable them again; - Ruby test runners against lookalike programs, scripts and rake tasks; - Module::Build actions only for ./Build and actions that depend on test; - SwiftLint's fixing runs, and the plugin only for builds; - git commands that check, keep, show or ignore changes; - GitLab's dependency scanning template only; - mix format and cljfmt by mode; - one VCS's ignore entries not counting for another's file; - GitHub status functions only without arguments; - Julia compat bounding every dependency; - the .NET security analyzer settings. `dotnet test --no-build` counting as a build is recorded as an expected failure. Part of cpp-best-practices#301 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
dotnet test, run, pack and publish build the project first, so they run the analyzers it references; with --no-build they run what an earlier step built and analyze nothing. The well-configured .NET fixture's `dotnet test --no-build` step no longer counts as an analysis (its `dotnet build` step still does), so only its facts snapshot changes. Part of cpp-best-practices#301 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
A project-wide -w silences every warning, so compiler-warnings should not pass on the -W flags it overrides, and warnings-as-errors has nothing left to treat as an error. Both pass today. Found while designing the score's disabled-diagnostics model (cpp-best-practices#295). Part of cpp-best-practices#301 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
-w (MSVC's /w and /W0) silences every warning, so like a flag's own -Wno- form it takes back the -W flags and -pedantic that compiler-warnings looks for, and leaves warnings-as-errors nothing to treat as an error. As with -Wno-, only a project-wide option counts; one a target sets does not. Part of cpp-best-practices#301 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…fixes, patterns Comments and blank lines keep a recipe open and other lines close it; += appends, ?= keeps the first value, and a comment after a value is dropped; includes and prerequisites accumulate across lines and rules; continued lines are numbered by their first line; the @, - and + prefixes are not part of a command; and a pattern rule covers only the goals its prefix and suffix match, with the stem in $* and the goal in $@. Part of cpp-best-practices#301 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
`in` needs an array; adding to an array appends an item or each item, keeping its condition; a comparison with a truth takes its condition from either side; indexing needs a dictionary and a string key; expressions are written back as they read; is_variable holds only for an assigned variable; a subdirectory reached twice is evaluated once, under the condition it is reached under; and a run target without a command runs nothing. Part of cpp-best-practices#301 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
Presets' anyOf and allOf conditions with more than one condition; an option without a default is OFF; a project's own <name>_IS_TOP_LEVEL says whether it is the top project; and add_test without a name registers nothing, which today it does (an empty test), recorded as an expected failure. Part of cpp-best-practices#301 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
`add_test()` and `add_test(NAME)` are errors in CMake and register nothing; they were recorded as a test with an empty name. The command still counts as the project using test registration. Part of cpp-best-practices#301 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
A tsconfig that extends a local base ("./tsconfig.base.json", "./tsconfig.base", or
"../tsconfig.base.json" from a subdirectory) never finds the base's strict setting: the joined
path keeps its "./" or "..", so it never matches the base file's path. Found while testing the
extends depth limit.
Part of cpp-best-practices#303
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
The path of a local base ("./tsconfig.base", "../tsconfig.base.json") is normalised before it is
looked up, so the base's settings are found and strict mode set there counts.
Part of cpp-best-practices#303
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
A shared helper builds a chain at the limit and one link past it for GitLab includes, extends and !reference tags, GitHub reusable workflows, tsconfig bases and CMake nested calls. Each is followed at its limit and not past it. Three readers stop without saying why: - GitLab extends past the limit claim the template is "not defined"; - !reference tags past it drop their commands with no unknown; - CMake calls past it vanish with no unknown. These are recorded as expected failures. Part of cpp-best-practices#303 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…, CMake nesting - GitLab extends nesting past its limit is unknown "the templates it extends nest more than 10 levels deep" rather than "a template it extends is not defined". - A !reference left unresolved past the limit makes its job unknown instead of dropping its commands silently. - A CMake call, include or add_subdirectory nested past MOST_NESTED records "calls, includes or subdirectories nest more than 24 deep" for its file, where it used to vanish. Part of cpp-best-practices#303 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
- GitLab template jobs and their marker commands, and GitLab jobs placed in the file they come from with the runner their own or default tags choose, and an unknown at the job that extends an undefined template. - pre-commit.ci's job and command with the hooks it skips. - tox commands placed in their section, running where tox.ini is, under exact and partial factor selection, and an unknown at a tox.ini that does not parse. - pre-commit hooks placed at their repository and index, and an unknown that hides the settings of a configuration that does not parse. - The settings readers that match code they do not parse mark every setting heuristic at the file they read. - The run helpers keep the reads and heuristic flag they are given. Part of cpp-best-practices#304 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…ers' defaults Each spelling reads the same as its siblings and differently from leaving it out: lcov reporting (genhtml, --list, -l, --summary) and measuring (geninfo, --capture), branch and condition coverage by option or --rc, llvm-cov's single- and double-dash options, tsc's project and build options, the JVM Jazzer's class, classpath and jar, NuGet auditing on any restore, mill's format checks, dart format's output options, and Haskell coverage on any build. Each action reader's defaults and inputs are pinned. `dart format --output=none` (the option joined to its value) is not recognised as only showing: recorded as an expected failure. Part of cpp-best-practices#302 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
The output mode is read as the --output or -o option's value, so `--output=none`, `-o=show` and the space-separated forms all only show the result instead of counting as a run that rewrites. Part of cpp-best-practices#302 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
tests/snapshots/registry.json records each known tool's name, languages, the levels it can provide with how to get each (the text of every hint that names it), what it reads, and each check's name, description and aliases. A change to any of them shows in review like the fixture snapshots, and is recorded with UPDATE_SNAPSHOTS=1. The mutation run's message survivors were mostly this text. Part of cpp-best-practices#305 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…ut can not see - A check built by answered_check is run: each ecosystem's answer is judged with the context's allow_unknown, and the results are labelled and folded. The real checks are built at import, so mutmut never exercised the run closure. - The mutation step caps each worker's memory: runaway mutants grew to about 14 GB in the full run. - The fresh-import architecture test is left out of mutation runs, where it only cost 70 s per import-time mutant. These two are recorded as expected failures. Part of cpp-best-practices#307 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
…t of mutation runs The mutation step sets `ulimit -v 4194304` before mutmut runs, so a runaway mutant fails as a kill instead of exhausting the runner. pyproject.toml's mutmut configuration deselects the fresh-import architecture test, which mutmut mapped to every function that runs at import but which fails only on a mutant that breaks an import: those mutants now report as having no tests, which is what they have, instead of surviving after 70 seconds each. Part of cpp-best-practices#307 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
This was referenced Nov 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on cpp-best-practices#291 (the corpus): fixes for the issues its audit filed, test first.
Each bug's failing test was committed as an expected failure (in cpp-best-practices#291's commit for most, on its own here for cpp-best-practices#290); each fix commit removes the decorator, so the suite fails as an unexpected success if a fix regresses, and drops the issue from the corpus manifest's per-project list. Commit messages say "Part of #N"; the issues stay open for you to close.
Fixed (one commit each unless noted)
[[ … ]]tests (over several lines too) andname=(…)arrays ([[ … ]]conditional expressions (also across lines) and array assignments are read as commands with a variable program cpp-best-practices/project_validator#190); here-document bodies (Heredoc bodies are tokenised as shell commands cpp-best-practices/project_validator#191);exec <program>(exec <program> …in a followed script is not read as running the program cpp-best-practices/project_validator#193).$/same-repository references are local for pinning (action-pinning:$/.github/actions/xand$/.github/workflows/x.yml(same-repository references) are reported "not pinned" cpp-best-practices/project_validator#209);uses: ./reads the rootaction.yml(uses: ./(the repository's own action) is never read:_local("./")yields "" and the path becomes "/action.yml" cpp-best-practices/project_validator#210);&&/||evaluate to the deciding operand, andruns-onexpressions are evaluated (GitHub expressions:&&/||evaluate to booleans instead of the operand value (_both/_either), so ternaries,runs-onfallbacks and+py=${{ a || b }}are wrong cpp-best-practices/project_validator#214).$(GO) test,GO ?= godefaults, command-line overridesmake test FLAGS=…cpp-best-practices/project_validator#197); comments on rule lines,include, pattern rules, undefined goals, and make in a directory CMake configured in place (Makefile reading: a rule with a trailing## commentis rejected,includeis not followed, unknown targets yield nothing, pattern rules and CMake-configured directories are not understood cpp-best-practices/project_validator#198).*.pyc*,site*/, whitelists, anchored paths) (vcs-ignore does not apply gitignore semantics: anchored patterns (/lisp/*.elc, docs/_build/), globs (site*/, *.pyc*, blib*, *.gz), whitelist files (/* + !name) cpp-best-practices/project_validator#236); a unit's own ignore file and root entries anchored to it (vcs-ignore reads only the root .gitignore although the unit that has the output carries its own (and Sphinx's _build lives under docs/) cpp-best-practices/project_validator#237)..dir-locals.elis not an Emacs Lisp source (part of A language present only in auxiliary files (fixtures, tooling, docs assets, editor dotfiles) becomes a product ecosystem and its FAILs decide the project cpp-best-practices/project_validator#182)..github/renovate.json5cpp-best-practices/project_validator#245);php <script>andsimple-phpunit(php <script>(and Composer's@php vendor/bin/x) is not a launcher, so PHPStan, PHPUnit and phpcs runs are missed cpp-best-practices/project_validator#200);richgo test(Go: richgo (a go test output wrapper) is not go test cpp-best-practices/project_validator#259);*_tests.pyundertests/(Python tests:*_tests.py(mkdocs/tests/*_tests.py, discovered withunittest discover -p "*tests.py") is not a test file pattern cpp-best-practices/project_validator#255); Meson's spaced-D werror=true(Workflow- and job-levelenv:variables are not substituted into command lines (meson setup ${CI_MESON_ARGS}) cpp-best-practices/project_validator#196); JSON files their tools read with comments (data-wellformed fails JSON files that are JSON-with-comments by their tool's specification (pyrightconfig.json, turbo.json, .luarc.json, version.json, asv.conf.json, tsconfig/jsconfig) cpp-best-practices/project_validator#232); deep JSON reported the same on every Python (data-wellformed's message for a 100000-deep JSON file depends on the Python build ("malformed JSON" vs "Expecting value") cpp-best-practices/project_validator#290).cargo --config <file> clippyis not clippy,-Dclippy::pedanticmisses the regex, RUSTFLAGS -Dwarnings is warnings-as-errors (check SKIPs for Rust), RUSTDOCFLAGS credited, unsafe_code warn ≠ forbid, rust-toolchain.toml pins the version, release profile inheritance cpp-best-practices/project_validator#287): clippy behind cargo's global options;-Dclippy::pedantic;RUSTDOCFLAGSnot credited; a warnings-as-errors practice;rust-toolchain.tomlpins a job's toolchain (a newpinned_byfor toolchain patterns, also used by Zig'sbuild.zig.zon, Zig: a library with no product binary is asked for a ReleaseSafe build; an AFL packaging shim is a C fuzz target; setup-zig withoutversioninstalls build.zig.zon's minimum_zig_version;zig build fuzz/zig build test --fuzzcpp-best-practices/project_validator#288).--allow-unknown(hardening: aforbidspractice whose answer is UNKNOWN falls through to FAIL with empty evidence under --allow-unknown ("runtime safety checks are compiled away: ") cpp-best-practices/project_validator#243); ci-lint is unknown when a job or command that could run the linter was not read (ci-lint WARNs "CI does not lint its configuration" when the lint job calls a reusable workflow that could not be read cpp-best-practices/project_validator#217).3/3.x,1.27/1.27.0), no "default" beside pinned versions,3.14tdistinct (platform-matrix counts "default" as a version and the same version spelled two ways as two (1.22 vs 1.22.x, 3 vs 3.x, JDK default vs default, current vs current CS); 3.14t is folded into 3.14 cpp-best-practices/project_validator#222); every version a setup action installs,node-version-file, pnpm's runtime (Setup actions: a multi-linedotnet-version: |/java-version: |installs every listed version,node-version-fileand matrix-drivennode-versionare not read cpp-best-practices/project_validator#216); toolchain patterns no longer match cache keys, refs, ranges or cross compilers (Toolchain patterns match free text: a cache key becomes "Rust 9.14.1", an action SHA becomes "Python 17", a cross-linker variable becomes a Nix system, "OTP 0", a Swift token on a clang-tidy line cpp-best-practices/project_validator#218).apple/swift-nio/.github/workflows/unit_tests.yml@main) is not read although the file is in the checkout cpp-best-practices/project_validator#212);defaults.run.working-directoryat workflow and job level, and a computed directory is unknown (Workflow-leveldefaults.run.working-directoryis ignored, and a composite action'sworking_directoryinput is attributed to the root unit cpp-best-practices/project_validator#213).TOX_ENV, hatch scripts (tox environments that fall back to [testenv] and TOX_ENV-selected envs are not expanded;hatch run env:scriptis not resolved cpp-best-practices/project_validator#199);deno task --cwd(deno task --cwd <dir> <task>: options before the task name are taken as the task cpp-best-practices/project_validator#201).MIT + file LICENSEstub, -or-later, LICENSE-CODE/LICENSE-TEXT pairs, Data Packagelicenses, dist.ini cpp-best-practices/project_validator#238, license compares a test fixture's manifest (tests/fixtures/invalid_pyproject/pyproject.toml declares INVALID) and a combined LICENSE file against the project's license cpp-best-practices/project_validator#239): more licence texts (EPL, LPPL, Artistic, the Perl dual licence, ODbL, CC-BY, PostgreSQL), blockquoted texts, a Data Package descriptor'slicensesstanding in for a file,License.pm/license.tnot licence files; fixture manifests are not compared, and a combined file is the licence it opens with.NOT DEFINED PROJECT_NAME,CMAKE_PROJECT_NAME STREQUAL PROJECT_NAME,string(COMPARE EQUAL …); out-of-source-build misses cmake_template's own PreventInSourceBuilds.cmake cpp-best-practices/project_validator#203, CMake constructs the evaluator skips: COMPILE_FLAGS and C_EXTENSIONS target properties, wrapper functions around add_custom_target,cmake -Ptest runners,*.cmake.intemplates parsed as build files cpp-best-practices/project_validator#204, Meson: a parenthesised condition continued over several lines fails to parse ("unexpected '\n'") cpp-best-practices/project_validator#205, sanitizers: an option whose default is a variable (${SUPPORTS_ASAN}from check_cxx_source_compiles) is "missing", not unknown cpp-best-practices/project_validator#206): pre-3.21 top-level guard idioms andstring(COMPARE EQUAL);REAL_PATH-resolved in-source checks;COMPILE_FLAGS; a branched value stays one argument inset_target_propertiesand an option default takes every branch's value;@PACKAGE_INIT@templates skipped;add_custom_commandoutputs run for the targets depending on them;cmake -P scriptfollowed intoexecute_process; Meson lines continued afterand/or.-Wshadow=global,-Weverything, project-wide-Wno-*negations, and a per-compiler "found" detail.:inside a flow sequence is rejected (PyYAML), and@TOKEN@template placeholders fail a file Gradle substitutes cpp-best-practices/project_validator#233, data-wellformed lists every template/test-data file as a detail line (577, 730 lines) and says "no XML/TOML/Jupyter file to parse" while the map detected them cpp-best-practices/project_validator#231): YAML 1.2 flow scalars starting with:;@NAME@templates unread, not malformed; unread files counted per reason; a format found only in test data says so.uses: |value keeps its trailing newline cpp-best-practices/project_validator#220, tests-run: a passing part says only "tests found in N file(s)"; site configurations, notebooks and specifications are called tests; the FAIL line does not say which language failed cpp-best-practices/project_validator#229, Small message fixes: cxx-extensions prints the example value (c_std=c17) not the fact (c11); raco fmt hint is empty; "4 of 4 actions" for reusable workflows; map job names with ${{ inputs.* }} cpp-best-practices/project_validator#250): each unread subject and clause said once, one unit's languages with the same result named together; job names rendered as GitHub shows them; tests-run says what was found and which runner CI uses; the standard found, a raco fmt hint, reusable workflows named as such.default: nonewith staticcheck listed as disabled, gosec excluded for the only path cpp-best-practices/project_validator#249): buf lint's rules need alintsection; a NoHttpCheck-only Checkstyle analyses nothing; golangci-lint'sdefault: none; gosec excluded everywhere scans nothing.ci:section;prek(a pre-commit reimplementation) is not a pre-commit runner cpp-best-practices/project_validator#221, status-badges: badges of other projects in a comparison table, stale Travis/CircleCI badges, a datahub badge read as Jenkins, reference-style markdown links, RTD and Codecov badges missed cpp-best-practices/project_validator#240, --map reports a language and its suffix as unrecognised at once (CUDA/.cu, Objective-C/.m) and lists files the profiles read (.freeze, .xctestplan, .resolved, .Plugin, .rda, .eex, .asciidoc, .tftpl, .zsh-theme) cpp-best-practices/project_validator#241, security-policy: the Kubernetes SECURITY_CONTACTS convention and a SECURITY.md with an obfuscated address + PGP fingerprint are not recognised cpp-best-practices/project_validator#242, dependency-management: an empty .gitmodules is "submodules", Composer's default type is not library, R DESCRIPTION and Julia Project.toml say "no dependencies are declared", license-notice third_party/, a project's own submodule is "third-party" cpp-best-practices/project_validator#246, committed-artifacts flags directories and files by name, not by what they are: source dirs named build/, fixture node_modules/.class/.egg-info/.bundle, bin/ scripts, zero-byte fixtures, a Julia coverage/ env, a sample presets file cpp-best-practices/project_validator#235): pre-commit.ci by its badge and prek-action; badges read by what they show; no unrecognised-suffix noise for claimed languages and known data files; SECURITY_CONTACTS and written-out addresses; empty.gitmodules, notice-onlythird_party/, Composer's default type, Julia's Pkg and R's DESCRIPTION as package managers; committed outputs judged by role and content.make/bazel/ninjamark a job C/C++, ubuntu jobs are "gcc", GitLab macOS jobs "gcc", Windows R jobs "msvc", clang-only jobs credit gcc cpp-best-practices/project_validator#228): a nested unit's definitions are its own;make/devenvalone mark no C/C++ job, only compiling jobs get a default compiler,gcc-multilibnames no compiler.make/bazel/ninjamark a job C/C++, ubuntu jobs are "gcc", GitLab macOS jobs "gcc", Windows R jobs "msvc", clang-only jobs credit gcc cpp-best-practices/project_validator#228, Swift settings are read by regex over any text: a commented-out.treatAllWarnings(as: .error), a patch file name,swift-tools-version: 6despiteswiftLanguageModes: [.v5]; ci-build-types demands sanitizers the sanitizers check skips cpp-best-practices/project_validator#244): Swift settings read from code, and a Swift sanitizers practice that ci-build-types agrees with.warning = trueis never read, Cucumber features are not tests, project runner scripts (test/runner, brew tests/style/typecheck, ruby test/*.rb) are not runners cpp-best-practices/project_validator#257, Dart: analysis_optionsinclude:in list form or an explicit rule list is "no lint set"; Flutter integration tests under example/integration_test are not tests cpp-best-practices/project_validator#258, Ansible/Kubernetes: helm-unittest suites in charts/*/unittests via ct.yaml additional-commands, yamllint as an Ansible linter, molecule's pytest verifier, antsibull-nox cpp-best-practices/project_validator#261, Typst: tytanic (tt run), the standard package test runner, is unknown; typstyle via its action + git diff is not a formatting gate cpp-best-practices/project_validator#263, Perl: t/, xt/, inc/ and corpus/ are not auxiliary roles;./Build authortestruns the tests; Test::Perl::Critic in the tests is perlcritic; Test::Warnings in test classes cpp-best-practices/project_validator#269, Shell: urchin tests (suffix-less files under test/), .zsh-theme scripts and .zunit tests, bash versions as a toolchain axis cpp-best-practices/project_validator#275, Emacs Lisp:eask run command <name>(a user-defined Eask command running ert) is not a test run cpp-best-practices/project_validator#279): JUnit 5 and JUnit 3 tests, multiplatform andtest/source sets; .NET test projects by framework or Arcade SDK; more JavaScript runners,test/,nub, xo and strict tsconfig bases; base-Rtests/*.R; Rakefile warnings andruby test/x.rb; Dart lint-set forms,integration_test/and the emulator action; helm-unittest'sunittests/and chart-testing's commands; tytanic and the typstyle action; Module::Build actions, Test::Perl::Critic,t/libclasses,corpus//inc/; zsh themes and zunit; Eask commands and Leiningen aliases.hadolint-dockerpre-commit hook is not credited cpp-best-practices/project_validator#260, Terraform:tofu -chdir=<dir> validate/fmt -checkwith -chdir before the subcommand is not matched; .tftpl/.tpl templates are unrecognised; clowdhaus/terraform-min-max versions cpp-best-practices/project_validator#262, Julia:format(".")+git diff+exit(1)is a formatting gate; biome via biome_jll in a julia -e one-liner cpp-best-practices/project_validator#267, Lua/LaTeX: plenary's-c "PlenaryBustedDirectory …"is credited as busted's -c coverage flag (luacov false positive); Teal (.tl, tl check) unknown; l3build unknown; Neovim versions are the toolchain axis of a plugin cpp-best-practices/project_validator#268, Scala:scalafixCheckalias, sbt-typelevel/sbt-tpolecat warning sets,++${{ matrix.scala }}cross versions read as "Scala default", JDK from a fallback expression, embedded H2 Flyway scripts cpp-best-practices/project_validator#274, Nix: treefmt'snix fmt -- --ciis a formatting gate (and treefmt.toml says what runs: statix, deadnix, nixfmt); agit diff --quietin anifcondition is not a failing step; devenv git-hooks anddevenv shell -- cmdcpp-best-practices/project_validator#285, OCaml:(flags …)of an ocamllex stanza or a test fixture's(env …)are taken as the project dropping :standard; opam's own-compiler matrix (ocamlv) is "1 OCaml version";ocaml/setup-ocaml/lint-fmtis the ecosystem's format check cpp-best-practices/project_validator#286, Rust:cargo --config <file> clippyis not clippy,-Dclippy::pedanticmisses the regex, RUSTFLAGS -Dwarnings is warnings-as-errors (check SKIPs for Rust), RUSTDOCFLAGS credited, unsafe_code warn ≠ forbid, rust-toolchain.toml pins the version, release profile inheritance cpp-best-practices/project_validator#287, Zig: a library with no product binary is asked for a ReleaseSafe build; an AFL packaging shim is a C fuzz target; setup-zig withoutversioninstalls build.zig.zon's minimum_zig_version;zig build fuzz/zig build test --fuzzcpp-best-practices/project_validator#288, OpenAPI/AsyncAPI detection matches"openapi":/asyncapi:keys inside JSON Schemas that describe OpenAPI documents, and redocly.yaml's theme section cpp-best-practices/project_validator#272, Anecho X=… >> $GITHUB_ENVinside a shellifbranch is taken as unconditional, and later steps withif: env.X != …are dropped silently cpp-best-practices/project_validator#284): hadolint's docker hook;terraform -chdir=; a Julia git-diff gate; nvim's-c; sbt aliases,project/*.scala, sbt-typelevel,'++ 2.12'; treefmt's--ciand its formatters; OCaml lexer flags, cram fixtures,ocamlv, lint-fmt; unsafe code forbidden in every crate; Zig example executables and commented AFL loops; API descriptions by their versioned key; conditional$GITHUB_ENVexports.shell:wrapper (msys2 {0},script -q -e -c "bash {0}") makes the whole step one argument of the wrapper program cpp-best-practices/project_validator#194, Maven<defaultGoal>and Gradle convention plugins (gradle/plugins/**.gradle.kts) are not read cpp-best-practices/project_validator#202, SQL: PostgreSQL extensions (PGXS Makefile, *.control, sql/*.sql, pg_regress/TAP tests) are not recognised; scan-build --status-bugs is not a static analyzer run cpp-best-practices/project_validator#276, Rust:cargo --config <file> clippyis not clippy,-Dclippy::pedanticmisses the regex, RUSTFLAGS -Dwarnings is warnings-as-errors (check SKIPs for Rust), RUSTDOCFLAGS credited, unsafe_code warn ≠ forbid, rust-toolchain.toml pins the version, release profile inheritance cpp-best-practices/project_validator#287, Zig: a library with no product binary is asked for a ReleaseSafe build; an AFL packaging shim is a C fuzz target; setup-zig withoutversioninstalls build.zig.zon's minimum_zig_version;zig build fuzz/zig build test --fuzzcpp-best-practices/project_validator#288): a binary launcher or generated configure is not read as shell; msys2 andscript -c "bash {0}"shells are split; a baremvnruns the POM's defaultGoal; PostgreSQL extensions are SQL andmake installcheckruns pg_regress; a Cargo profile builds what it inherits; Zig'sstd.testing.fuzzandzig build test --fuzz.for/caseheaders read as commands (The shell splitter records afor x in …; doloop header as a command namedforcpp-best-practices/project_validator#292),dune fmt --preview(OCaml:dune fmt --previewis read as rewrite-only, though it fails on a diff cpp-best-practices/project_validator#293), arithmetic expansion split into a command (The shell splitter reads arithmetic expansion$((…))as a command named((cpp-best-practices/project_validator#294).Not done, needs your decision (and cases left open)
--auxiliarynames it.web/test/browser) count for every unit by design; the regression test now pins the nested-package case only.nix fmt -- --ciis a formatting gate (and treefmt.toml says what runs: statix, deadnix, nixfmt); agit diff --quietin anifcondition is not a failing step; devenv git-hooks anddevenv shell -- cmdcpp-best-practices/project_validator#285'sgit diff --quietin anif: the job commits the change back, which counts as enforcement by design; the regression test stays an expected failure until you say otherwise.scan-build --status-bugsalone cannot pass static analysis; should it count as one of them, or as a third accepted analyzer?tofu -chdir=<dir> validate/fmt -checkwith -chdir before the subcommand is not matched; .tftpl/.tpl templates are unrecognised; clowdhaus/terraform-min-max versions cpp-best-practices/project_validator#262), antsibull-nox and molecule's testinfra verifier (Ansible/Kubernetes: helm-unittest suites in charts/*/unittests via ct.yaml additional-commands, yamllint as an Ansible linter, molecule's pytest verifier, antsibull-nox cpp-best-practices/project_validator#261), Cucumber features and suffix-less runner scripts (Ruby: a Rakefile'swarning = trueis never read, Cucumber features are not tests, project runner scripts (test/runner, brew tests/style/typecheck, ruby test/*.rb) are not runners cpp-best-practices/project_validator#257), urchin and bash-version axes (Shell: urchin tests (suffix-less files under test/), .zsh-theme scripts and .zunit tests, bash versions as a toolchain axis cpp-best-practices/project_validator#275), R sanitizer images and comment-triggered styler jobs (R: tests/*.R scripts run by R CMD check are not tests (data.table), sanitizer images (rocker/r-devel-san) are the R way to run ASan/UBSan, styler on issue_comment is not a gate cpp-best-practices/project_validator#256), Groovy/Spock (JVM test discovery is hard-wired to src/test/{java,kotlin} + @Test: JUnit 3 (guava-tests/test), Kotlin multiplatform jvmTest, @TestFactory, Groovy/Spock are missed cpp-best-practices/project_validator#252), and the larger ecosystem gaps (BEAM: Gleam is unknown (gleam.toml, .gleam, gleam test/format), erlang.mk is not a build system (rebar.config is its generated artifact), Earthly targets cpp-best-practices/project_validator#264, Protobuf: schema-declared judges example modules whose own buf.yaml disables PACKAGE_DEFINED; link-check and markdownlint do not apply to a Protobuf-only repo's Markdown docs cpp-best-practices/project_validator#271, JSON Schema/OpenAPI: AsyncAPI CLIvalidateover examples/ is a schema-examples check;yarn test→ git-dependency program should be unknown; SchemaStore's tsconfig schemas taken as TS configs; library-driven validation harnesses cpp-best-practices/project_validator#273, Common Lisprtsuites (testing/*.lisp + .testvec) and Clojurelein test-allproject.clj aliases are not recognised cpp-best-practices/project_validator#265's Common Lisp part, Lua/LaTeX: plenary's-c "PlenaryBustedDirectory …"is credited as busted's -c coverage flag (luacov false positive); Teal (.tl, tl check) unknown; l3build unknown; Neovim versions are the toolchain axis of a plugin cpp-best-practices/project_validator#268's remaining parts, SQL: PostgreSQL extensions (PGXS Makefile, *.control, sql/*.sql, pg_regress/TAP tests) are not recognised; scan-build --status-bugs is not a static analyzer run cpp-best-practices/project_validator#276's TAP and C-coverage parts, Flutter's generated platform directories make a Dart project a C/C++, Swift and JVM project cpp-best-practices/project_validator#185, A shell.nix or a flake whose only outputs are devShells is judged as a Nix build of the product cpp-best-practices/project_validator#187, Racket's standard multi-package layout (X, X-lib, X-doc, X-test) fails tests-run for every package whose tests live in the -test package cpp-best-practices/project_validator#189, Commands the reader cannot resolve are dropped silently instead of recorded as unknown:${PREFIX}tool,${{ env.X }} test, all-expression steps, missing scripts, docker run, earthly, unknown composite actions cpp-best-practices/project_validator#195, Maven<defaultGoal>and Gradle convention plugins (gradle/plugins/**.gradle.kts) are not read cpp-best-practices/project_validator#202's Gradle convention plugins, A local reusable workflow's caller inputs (with: os:) are not propagated,container:always means linux, and include-only matrices with per-entry runs-on/container are misread cpp-best-practices/project_validator#211).continue-on-error: true(GitHub) andallow_failure: true(GitLab) are never read: an advisory step is credited as a gate cpp-best-practices/project_validator#215 (continue-on-error/allow_failure): marking such steps' commands needs a newCommandfield (every facts snapshot changes) or dropping their commands from what the checks see; which do you prefer?Behaviour changes worth a look
warnings-as-errorsresult (the Rust fixtures' snapshots gain it).src/test/have the test role (facts snapshots only).makein a CMake project's directory with no Makefile builds CMake's targets.All gates green on the tip (unit tests, ruff, mypy, lizard, jscpd, 99% coverage, strict self-validation).
Mutation job fixed (it was passing without running)
The mutmut job on this PR and on upstream cpp-best-practices#291 reported success without running a mutant. The tests import the
corpuspackage, which was not inalso_copy, so the run failed while mapping tests to functions. The step pipesmutmut runintoteeunder GitHub's default shell (nopipefail), sotee's exit status hid the failure.tests/test_mutation_setup.pynow checks both. The fix addscorpus/toalso_copyand setsshell: bashon the step. cpp-best-practices#291 needs thealso_copychange too.C/C++ static analysis requires one analyzer
At the user's request, clang-tidy or cppcheck now passes
static-analysis, as one analyzer does in every other language. The hint offers either, and the docs say so. Running both will earn depth in the planned score (cpp-best-practices#295) instead of being needed to pass. Nothing marks a tool as required any more, but the mechanism (Tool.required) is left in place.Test-suite improvements from the full mutation run (cpp-best-practices#308)
Each issue's tests come first. Where a test exposed a bug, the bug got a failing test, then a fix.
Bugs found and fixed:
--allow-unknowndid not relax an unknownci-lintanswer: the check never passed it (Pin small interfaces: allow_unknown, the CLI surface and parse-error positions cpp-best-practices/project_validator#306).-w(and MSVC/w,/W0) passedcompiler-warningsandwarnings-as-errorsalthough it silences every warning (Tests check what a recogniser accepts, rarely what it must reject (429 surviving mutants) cpp-best-practices/project_validator#301).dotnet test/run/pack --no-buildcounted as a build that runs the project's analyzers (Tests check what a recogniser accepts, rarely what it must reject (429 surviving mutants) cpp-best-practices/project_validator#301).add_test()andadd_test(NAME)registered an empty test (Tests check what a recogniser accepts, rarely what it must reject (429 surviving mutants) cpp-best-practices/project_validator#301).A tsconfig extending a local base (
./tsconfig.base.json,../tsconfig.base.json) never found the base: strict mode set there did not count (Limits and boundaries are never tested at the edge (243 surviving mutants) cpp-best-practices/project_validator#303).Limits that stopped silently or misleadingly (Limits and boundaries are never tested at the edge (243 surviving mutants) cpp-best-practices/project_validator#303):
extendspast 10 levels claimed the template was "not defined";!referencepast 10 levels dropped its commands;Each is now an unknown that names the limit.
JSON trailing content was reported at the end of the value, not where the content starts (Pin small interfaces: allow_unknown, the CLI surface and parse-error positions cpp-best-practices/project_validator#306).
dart format --output=none(option joined to its value) counted as a rewriting run (Alternative spellings and action-input defaults are untested (297 surviving mutants) cpp-best-practices/project_validator#302).Tests added:
allow_unknown(now a required keyword; theJudgeprotocol too);tests/snapshots/registry.jsonsnapshots every tool's name, languages and hint text, and every check's description.answered_check;ulimit -v);Still open:
Results in policy tests.🤖 Generated with Claude Code
https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt