Repository navigation
Open
Labels
area: test-suiteThe validator's own tests, mutation testing and fuzzingThe validator's own tests, mutation testing and fuzzingtrackingRoadmap or umbrella issue tracking several pieces of workRoadmap or umbrella issue tracking several pieces of work
Description
Limits and boundaries are never tested at the edge
From the full mutmut run of fixes at 1c2f869 (27,670 of 32,420 mutants checked; see #308).
243 surviving mutants in 125 functions move a boundary without failing:
- a comparison (
<→<=); - a constant (
+ 1→+ 2,[-5:]→[-6:]); - a starting depth (
depth = 0→1); - a recursion or nesting limit.
No test sits exactly at a limit or one past it:
| Limit | Value |
|---|---|
ci/gitlab.py _DEPTH |
10 (includes, extends, !reference) |
build_systems/cmake/evaluation.py MOST_NESTED |
24 |
ci/github_actions.py _CALL_DEPTH |
4 |
ecosystems/javascript/analysis.py tsconfig extends |
hard-coded depth < 4 |
ecosystems/data.py _DEEP |
– |
Functions with the most survivors
| Function | Survivors | Example mutant |
|---|---|---|
ci.gitlab._referenced |
13 | -def _referenced(value: object, pipeline: Mapping[str, object], depth: int = 0) -> object: || +def _referenc |
ecosystems.data._deep |
11 | -if text.count("[") + text.count("{") <= _DEEP: || +if text.count("[") - text.count("{") <= _DEEP: |
ecosystems.javascript.analysis._strict_settings |
11 | -def _strict_settings(path: str, facts: Facts, depth: int = 0) -> list[Setting]: || +def _strict_settings(pa |
ci.gitlab._included |
8 | -def _included(path: str, document: Mapping[str, object], load: Load | None, depth: int = 0) -> _Pipeline: | |
ci.gitlab._extended |
7 | -def _extended(name: str, pipeline: Mapping[str, object], depth: int = 0) -> Mapping[str, object] | None: |\ |
checks.large_files._size |
6 | -if size < 1024 * 1024: || +if size <= 1024 * 1024: |
ecosystems.beam._bracketed |
6 | -depth = 0 || +depth = 1 |
build_systems.cmake.commands._option |
5 | -if len(arguments) >= 2: || +if len(arguments) > 2: |
facts.outputs._glob_ignores |
5 | -for depth in range(1, len(components) + 1): || +for depth in range(len(components) + 1): |
ecosystems.perl._runs_test_action |
5 | -action = argv[2] if argv[:2] == ("perl", "Build") and len(argv) > 2 else argv[1] if len(argv) > 1 else "" |\ |
build_systems.cmake.evaluation.Evaluation.parse_arguments |
4 | -_parse_arguments(frame.scope, texts[2:6], arguments[_number(texts[1]) :]) || +_parse_arguments(frame.scope, |
build_systems.cmake.conditions._compared |
4 | -return {"EQUAL": difference == 0, "LESS": difference < 0, "GREATER": difference > 0}.get( || +return {"EQUA |
facts.conditions._Diagram.variable |
4 | -return self.nodes[node][0] if node > _TRUE else 1 << 30 || +return self.nodes[node][0] if node >= _TRUE els |
ecosystems.rust.cargo_argv |
4 | -while len(words) > 1 and words[0] in _CARGO_VALUED: || +while len(words) >= 1 and words[0] in _CARGO_VALUED |
build_systems.cmake.evaluation.Evaluation.list_command |
3 | -if len(words) < 2: || +if len(words) <= 2: |
build_systems.cmake.evaluation._arguments |
3 | -if word.argument >= 0 and index and words[index - 1].argument == word.argument: || +if word.argument > 0 an |
build_systems.cmake.commands._registered_test |
3 | -name = words[1] if len(words) > 1 else "" || +name = words[1] if len(words) >= 1 else "" |
build_systems.cmake.commands._set |
3 | -if "CACHE" in texts and texts.index("CACHE") >= 1: || +if "CACHE" in texts and texts.index("CACHE") > 1: |
ci.github_actions._runner_os |
3 | -runner = EXPRESSION.sub(lambda match: rendered(match[0][3:-2], known) or match[0], runner) || +runner = EXP |
pretty.print_reports |
2 | -name_width = max((len(report.check.name) for report in reports), default=0) || +name_width = max((len(repor |
build_systems.make._logical_lines |
2 | -start, parts = 0, [] || +start, parts = 1, [] |
build_systems.meson.evaluation.Evaluation.run |
2 | -self._statements += 1 || +self._statements += 2 |
build_systems.meson.operations._known_element |
2 | -return all(when == ALWAYS for _, when in container.items) and -len(container.items) <= key < len(container.it |
build_systems.cmake.evaluation.Evaluation._evaluate_all |
2 | -lists = sorted((path for path in self.files if PurePosixPath(path).name == LISTS_FILE), key=_depth) || +lis |
build_systems.cmake.evaluation.Evaluation.run |
2 | -self._commands += 1 || +self._commands += 2 |
build_systems.cmake.evaluation.Evaluation._call |
2 | -if arguments in definition.seen or definition.name in frame.stack or len(frame.stack) > MOST_NESTED: || +if |
build_systems.cmake.evaluation.Evaluation.string_command |
2 | -if len(words) >= 2 and words[0].text in ("APPEND", "PREPEND"): || +if len(words) > 2 and words[0].text in ( |
build_systems.cmake.evaluation.Evaluation.separate_arguments |
2 | -name, texts = words[0].text, words[2:] if len(words) > 2 and words[1].text.endswith("_COMMAND") else words[1: |
build_systems.cmake.evaluation.Evaluation.include |
2 | -if len(frame.stack) > MOST_NESTED or (path in self.reached and self._guarded(path)): || +if len(frame.stack |
build_systems.cmake.evaluation._loop_items |
2 | -if texts[:1] == ["RANGE"] or texts[:2] == ["IN", "ZIP_LISTS"]: || +if texts[:1] == ["RANGE"] or texts[:3] = |
build_systems.cmake.values.expanded |
2 | -if end < 0: || +if end <= 0: |
build_systems.cmake.commands._add_custom_command |
2 | -ends = [at for at, text in enumerate(texts[1:], 1) if text in _CUSTOM_COMMAND_KEYWORDS] or [len(texts)] || |
build_systems.cmake.commands._add_dependencies |
2 | -collector.dependencies.setdefault(words[0].text, []).extend(word.text for word in words[1:]) || +collector. |
build_systems.cmake.generator_expressions._parts |
2 | -while (start := text.find(_START, position)) >= 0 and (end := _end(text, start + 2)) >= 0: || +while (start |
checks.large_files._lfs_patterns |
2 | -if len(fields) > 1 and not fields[0].startswith("#") and "filter=lfs" in fields[1:]: || +if len(fields) >= |
Proposal
-
A shared boundary helper builds a chain of N and of N+1 for each limited reader:
- nested GitLab includes;
- tsconfig
extends; - CMake functions calling functions;
- reusable workflows calling reusable workflows.
It asserts that N is followed and N+1 produces the documented unknown, naming the limit.
-
One edge test per off-by-one survivor in the parsers: the
stderr[-5:]tail,argv[1:2]/argv[2]positions, andrange(1, …)starting points. -
Fold into Unify parsing: shared infrastructure, trees with spans, evaluation kept separate #300's single limits policy once it exists, so every limit is tested the same way.
🤖 Generated with Claude Code
Metadata
Metadata
Assignees
Labels
area: test-suiteThe validator's own tests, mutation testing and fuzzingThe validator's own tests, mutation testing and fuzzingtrackingRoadmap or umbrella issue tracking several pieces of workRoadmap or umbrella issue tracking several pieces of work
Activity
Test that tsc is strict through a local base configuration
Follow a tsconfig's local base by its normalised path
Test every chain-following reader exactly at its limit
Say why a chain stopped at its limit: GitLab templates and references…
leftibot commentedon Nov 18, 2026
Progress in leftibot#80:
tests/test_limits.pyhas one helper that builds a chain at the limit and one link past it, for:extendsand!reference(10);It found:
./tsconfig.base.json,../tsconfig.base.json) never found the base, because the joined path kept its./or... Strict mode set in a base did not count. Fixed.extendspast the limit claimed the template was "not defined". It now says the templates nest too deep.!referencepast the limit dropped the job's commands silently. It is now an unknown.MOST_NESTEDvanished. They now record "calls, includes or subdirectories nest more than 24 deep".Remaining: folding these limits into #300's single limits policy.
🤖 Generated with Claude Code
https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt