Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 16 additions & 9 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,17 @@

Build and test: `pip install -e . && pytest`

## Changing the `--ui live` dashboard
## Changing the `--ui live` TUI

Unit tests check strings in the frame. They can't tell you whether the layout reads well, so
look at the dashboard before you commit any change to `strain/dashboard.py`, `strain/console.py`
or `strain/keys.py`:
The TUI is a Textual app in `strain/tui.py`, drawn from the `LiveState` in `strain/live_state.py`
that the runner's console hooks update from the runner's thread. Textual is the optional `ui` extra:
without it, `--ui live` prints plain output. Keep it that way; don't add a second, hand-drawn TUI.

1. `pip install -e '.[ui-snapshot]'` (adds `pyte` and `pillow`).
The Pilot tests in `tests/test_tui.py` press keys and click headlessly. They can't tell you whether the
layout reads well, so look at the TUI before you commit any change to `strain/tui.py`,
`strain/live_state.py` or `strain/console.py`:

1. `pip install -e '.[ui-snapshot]'` (adds `textual`, `pyte` and `pillow`).
2. Configure a copy of the fixture as a git repository with a coverage build (strain refuses a folder
without `CMakeCache.txt`), using the flags from `configure_project` in `tests/test_end_to_end.py`:
```sh
Expand All @@ -18,20 +22,23 @@ or `strain/keys.py`:
"-DCMAKE_CXX_FLAGS=--coverage -fsanitize=address,undefined -fno-omit-frame-pointer -D_GLIBCXX_ASSERTIONS -ftrivial-auto-var-init=pattern" \
"-DCMAKE_EXE_LINKER_FLAGS=--coverage -fsanitize=address,undefined"
cmake --build $demo/build
python tools/ui_snapshot.py $demo/project --size 120x36 --keys down,tab,q --out ui-snapshots \
python tools/ui_snapshot.py $demo/project --size 120x36 --keys 2,down,3,4,q --out ui-snapshots \
-- --build-dir $PWD/$demo/build --report-dir $PWD/$demo/report
```
- This needs the fixture's toolchain (CMake, a C++ compiler, gcov) and takes about a minute.
- Arguments after `--` go to `strain run`.
- Also try a small size such as `--size 80x24` to check truncation and panel sizing.
- Keys can be mouse events, such as `click:12:5`; `--delay 9` waits for the toasts to fade first.
3. Read the `ui-snapshots/*.txt` grids to check alignment, and view the `*.png` frames to check
colours and reverse video. `pyte` drops SGR 2 (faint), so `DIM` text looks like normal text in the PNGs.
colours and reverse video. `pyte` drops SGR 2 (faint), so `dim` text looks like normal text in the PNGs,
and the PNG font has no braille, so the spinner shows as a box.
4. Check these points, which have regressed before:
- The header totals match the survivors view, so each mutant is counted once across tests.
- The progress bar and the title name the same test as the queue.
- The header names the same test as the queue.
- Each survivor detail row is padded to `DETAIL_LABEL_WIDTH`.
- No panel leaves large blank areas while another panel is truncated.
- At 120x36, every table fits its panel without a horizontal scrollbar.
5. Attach before and after frames, or the relevant `.txt` excerpt, to the pull request.
Don't commit `ui-snapshots/`.

Keep iterations short: render the dashboard once, fix what you see, and render it once more to confirm.
Keep iterations short: render the TUI once, fix what you see, and render it once more to confirm.
38 changes: 35 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

strain changes your code in small ways (`<` becomes `<=`, `&&` becomes `||`) and checks whether your tests notice. Every mutant that survives points at a test that runs the code but doesn't really check it.

![The strain run --ui live dashboard in an 80x24 terminal, listing three survivors and showing the selected one as a diff with a hint](docs/images/dashboard-80x24.png)
![The strain run --ui live TUI in a 120x36 terminal: the Survivors tab lists three survivors and shows the selected one with its coverage, a hint and a highlighted diff](docs/images/tui-120x36.png)

*`strain run --ui live` on `tests/fixtures/sample_project` after the run, with the second survivor selected.*

Expand Down Expand Up @@ -37,14 +37,14 @@ Notes for whoever fills this in (human or Claude Code):
- Optional:
- `cppcheck`: the best mutation engine
- `tree-sitter`: the fallback engine (pip)
- `rich`: the live dashboard (pip)
- `textual`: the live TUI, `--ui live` (pip)

## Install

```sh
# TODO(claude): replace once packaging exists
pip install strain # core
pip install "strain[ui]" # with the live dashboard
pip install "strain[ui]" # with the live TUI
```

## Quick start
Expand Down Expand Up @@ -72,6 +72,8 @@ strain run --build-dir build-strain -R parser # only tests matching "parser"
strain run --build-dir build-strain --since main # only lines changed since main
```

Add `--ui live` (with `strain[ui]` installed) to watch the run in a full-screen TUI, described [below](#live-tui).

### 3. Read the results

<!-- TODO(claude): paste a real console summary once one exists -->
Expand All @@ -85,6 +87,36 @@ Action items
3. parser.cpp:97 `+` → `-` survived (not_detected): assert the computed offset
```

## Live TUI

`strain run --ui live` shows the run in a full-screen TUI built on [Textual](https://textual.textualize.io/). Without
Textual installed, it warns and prints plain output instead. Use the keyboard or the mouse.

| View | Shows |
|------|-------|
| `1` Run | The mutated code as a highlighted diff, the mutant queue with each result and time, findings and the build log |
| `2` Survivors | Each survivor with its coverage, the tests it survived, a hint and its source, and the score of each file |
| `3` Mutants | Every mutant so far, filtered by state, with the same details |
| `4` Stats | The [safety nets](#safety-nets) the configuration let fail, why each matters and how to fix it; scores by file, operator and test; the slowest mutants |
| `5` Log | The full console output |

Above the views, the header shows the score with its history, the totals, the elapsed time and an estimate of the time
left. When a safety net fails but its level is `warn` or `ignore`, a `! N safety nets off` badge appears there; click it
for the details. It is red when one of those nets errors by default, because then a mutant that breaks the code can
survive, and each survivor's details carry the same caution. Under it, a bar has one cell per mutant, colored by its result: hover to name a mutant, click to show it in the
queue.

| Key | Does |
|-----|------|
| `1`–`5`, `tab`, `shift+tab` | Switch views; or click a tab |
| `↑` `↓`, `PgUp` `PgDn` | Move through a list; in Run, look at another mutant while the run goes on |
| `enter` | In Run, open the selected mutant in Mutants |
| `f` | In Run, follow the running mutant again; in Mutants, show the next state filter |
| `p` | Pause before the next mutant starts, with the sources unmutated; press again to resume |
| `ctrl+c` | Stop the run: strain restores the sources and writes the reports, as with `--resume` |
| `q` | Leave once the run has finished, then print the plain summary |
| `?`, `ctrl+p` | Show every key; open the command palette, which also changes the theme |

## How it works

For each selected test:
Expand Down
Binary file removed docs/images/dashboard-80x24.png
Binary file not shown.
Binary file added docs/images/tui-120x36.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ dependencies = []

[project.optional-dependencies]
tree-sitter = ["tree-sitter>=0.26.0", "tree-sitter-cpp>=0.23.4"]
ui-snapshot = ["pyte>=0.8.2", "pillow>=13.0.0"]
ui = ["textual>=8.2.8"]
ui-snapshot = ["textual>=8.2.8", "pyte>=0.8.2", "pillow>=13.0.0"]

[project.scripts]
strain = "strain.cli:main"
Expand Down
2 changes: 2 additions & 0 deletions requirements-ci.in
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@ lizard
pytest
ruff>=0.17.4
setuptools>=64
# The live TUI, so its tests run rather than skip.
textual>=8.2.8
37 changes: 37 additions & 0 deletions requirements-ci.txt
Original file line number Diff line number Diff line change
Expand Up @@ -129,10 +129,29 @@ iniconfig==2.3.1 \
--hash=sha256:67f4b9c50da0dedf52af349e7749a80a9057a5031199791b906c3bb3ae878960 \
--hash=sha256:9121e2c1fdb355232495be3194c8dfe87ccc2d5dee45947b78e68f499790d7a7
# via pytest
linkify-it-py==2.2.0 \
--hash=sha256:3adc40eb5af300b2605fcfdb968c24e1d780a90f1f2221af7c15e5111e94d443 \
--hash=sha256:907acd2d17ac1fbb9ddb62c8957ccbd6158cac602231a15c3b0cd1e215f03cee
# via markdown-it-py
lizard==1.25.1 \
--hash=sha256:3513ce263d17ed580f8efa202658b02c684d5da2b39ef40c7663971be822a9d6 \
--hash=sha256:789e7103875cb15aac273e1701cd72d4b9d5fa521790e258fe071dbf79dbc4ee
# via -r requirements-ci.in
markdown-it-py==4.3.0 \
--hash=sha256:7aa1511a46c39138de78d9eba92ce47e63124d8003fc98b3a351909e04239efa \
--hash=sha256:91a58fa5b72c71a40cf7177250fd767480d7a2bb3ac426ccfb76fb90305fdaaa
# via
# mdit-py-plugins
# rich
# textual
mdit-py-plugins==0.7.0 \
--hash=sha256:26d2179227aaf4d1dc7acffe7a738cbd15cade6b5a61cff0a2cbf9fed93efede \
--hash=sha256:7f8e30403f3dd8bfe3afcdcbc6e61066d3f2e714167507b9c26eb385ec399353
# via textual
mdurl==0.1.2 \
--hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \
--hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba
# via markdown-it-py
packaging==26.3 \
--hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 \
--hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c
Expand All @@ -141,6 +160,10 @@ pathspec==1.1.1 \
--hash=sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a \
--hash=sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189
# via lizard
platformdirs==4.14.5 \
--hash=sha256:32e9921db7385aa8169bee4fe9501923e5715ba9fb7ea3b9c69b0d8e0639f7a1 \
--hash=sha256:8e778c6fa421516f4d3369449f8df3b0e1b72c0e0005fe88e3ec772d0a14e750
# via textual
pluggy==1.7.0 \
--hash=sha256:7dd7b0d8832ba3cb632c306926ded123429211b83641b35dc5c41ad2d34f9bec \
--hash=sha256:d1eaa46ebb595891b860ab086b4d09c8588af65ebd4361b8e8f4bb8920b90ba8
Expand All @@ -151,10 +174,16 @@ pygments==2.21.0 \
# via
# lizard
# pytest
# rich
# textual
pytest==9.2.0 \
--hash=sha256:07acdcf88ec174f907dc01af70bf52d49b014bbab03faeeade31c96b18c2e588 \
--hash=sha256:97c4f3b1f3c7414aa36465e09b8220f1d03627993bd51b1282e193753c80557b
# via -r requirements-ci.in
rich==15.0.0 \
--hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \
--hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36
# via textual
ruff==0.17.4 \
--hash=sha256:0d1f4848fd2ec0499be4046db9b7dfcc952666885f00233cc7033515282556a8 \
--hash=sha256:13e781c3c5f663e7fba737eaaeb70dd943dd55171daf316afb25cd5268cfe0c7 \
Expand All @@ -175,6 +204,14 @@ ruff==0.17.4 \
--hash=sha256:f32d178b50e826e00f6c6c1be3f56333068acb9b67fe167143527fa9b81e890d \
--hash=sha256:f99e95a1515161138be3cbd1b06c06e739d8feb38ade35564556999fffc3686e
# via -r requirements-ci.in
textual==8.2.8 \
--hash=sha256:267375fd402dc8d981457212efa71f0e3365fd17bba144ba9bb3ed7563cb374a \
--hash=sha256:3f106a9fbc73e39dd266c9712432087de78a6d644084c7c241d6a25c3169115b
# via -r requirements-ci.in
typing-extensions==4.16.0 \
--hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 \
--hash=sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5
# via textual

# The following packages are considered to be unsafe in a requirements file:
setuptools==84.2.1 \
Expand Down
2 changes: 1 addition & 1 deletion strain/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ def _add_backend_arguments(parser: argparse.ArgumentParser) -> None:
choices=list(UiMode),
default=UiMode.AUTO,
help=(
"console output: auto (live progress and color on a terminal), live (full-screen dashboard), "
"console output: auto (live progress and color on a terminal), live (full-screen TUI; needs strain[ui]), "
"log (plain lines) or json (events)"
),
)
Expand Down
26 changes: 24 additions & 2 deletions strain/console.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
"""Human-readable console output: phases, steps, a live status line and colored mutant states."""

import enum
import importlib.util
import os
from collections.abc import Mapping, Sequence
from collections.abc import Callable, Mapping, Sequence
from typing import TextIO

from .coverage import LineCoverage
from .mutators import Mutant
from .results import MutantResult, MutantState
from .safety_nets import RelaxedSafetyNet, SafetyNet

RESET = "\033[0m"
BOLD = "\033[1m"
Expand Down Expand Up @@ -127,7 +129,11 @@ def warn(self, message: str) -> None:
self.finish()
print(f"strain: {paint('warning:', BOLD + YELLOW, self.err_color)} {message}", file=self.err, flush=True)

# Progress hooks for the live dashboard; plain console output shows the same progress as lines.
def run(self, work: Callable[[], int]) -> int:
"""Runs the whole of strain's `work`; the live TUI overrides this to run it beside the TUI."""
return work()

# Progress hooks for the live TUI; plain console output shows the same progress as lines.
def test_started(self, name: str, number: int, total: int) -> None:
pass

Expand All @@ -146,8 +152,24 @@ def mutant_evaluated(self, result: MutantResult, source: str, line: LineCoverage
def build_started(self, command: str) -> None:
pass

def safety_nets_checked(self, relaxed: Sequence[RelaxedSafetyNet], holding: Sequence[SafetyNet]) -> None:
"""The safety nets that fail but did not stop the run, and those that hold; the lines show them already."""

def build_output(self, line: str) -> None:
pass

def run_finished(self) -> None:
pass


def create_console(out: TextIO, err: TextIO, mode: UiMode) -> Console:
"""The Textual TUI for `--ui live` on a terminal; plain console output everywhere else, or without Textual."""
if mode is not UiMode.LIVE or not is_terminal(out):
return Console(out, err, mode)
if importlib.util.find_spec("textual") is None:
console = Console(out, err, UiMode.AUTO)
console.warn("--ui live needs Textual: pip install 'strain[ui]'; showing plain output instead")
return console
from .tui import TuiConsole

return TuiConsole(out, err)
Loading
Loading