ci: make the review something you ask for, and tell it where the bugs are - #138
Conversation
The first version of the briefing carried the conventions and nothing about failure modes, so a review arrived knowing what it must not break and not where to look. The five classes it now names are read off the mutation registry rather than invented - that file records every behaviour broken on purpose to prove a test catches it, and the entries cluster hard: lifecycle races around targeting, view state drifting from fact, a warning that stops matching the state it describes, a column registry drifting from the table it renders, and empty or just-changed states. First among them is the one no gate can see: a guard that cannot fail. An assertion that would also pass over an empty set, a collector that returns nothing, a walk rooted where a runner has no directory. The briefing gives the question to ask of every added assertion - what input makes this red - and says outright that a test which would still pass with the behaviour removed is a finding. Two blind spots are named because a diff hides them. The fake tkinter models one widget class with no style validation and no geometry, so changes to styles, geometry or column mapping are not covered by the tests that look like they cover them. And the suite runs on two platforms while a developer checks one, so a symbol named after one system can raise on the other. Paid for in tokens on every run, so it is not only longer: the meta paragraph and the CI list are cut back, and a "what not to propose" section is added, because the fastest way to make a review worthless is to let it repeat the linter. No counts are written into the prose - an unguarded number goes stale. Also drops the mention of where the maintainer's own notes live. The reason a reviewer needs is that the file is not part of this repository, and that is all it now says. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| - **The fake tkinter has blind spots.** The suite drives a stand-in for Tk with one widget | ||
| class, no style validation and no geometry. Changes to styles, geometry or column mapping | ||
| are therefore **not** covered by the tests that appear to cover them, and deserve a closer | ||
| read than their green suite suggests. |
There was a problem hiding this comment.
This describes the previous fake, not the one in the tree — and because this file becomes CLAUDE.md for every review run, it will steer future reviewers away from the areas that are in fact best covered.
- "one widget class" —
tests/fake_tk.pydefines seven with real behaviour:W,Text,Root,Notebook,Paned,Menu,Treeview. Its own docstring says this in the past tense:BeanNetworkTester/tests/fake_tk.py
Lines 4 to 9 in 728729f
- "no geometry" —
pack()recordspack_infoand even raises whenbefore=/after=names a non-sibling;grid()recordsgrid_info;Root.geometry()is modelled. Tests assert on it directly:BeanNetworkTester/tests/test_gui_layout.py
Lines 163 to 171 in 728729f
- "column mapping" —
Treeview.column()resolves a"#N"spec againstdisplayed_columns()exactly as Tk does, added specifically so the hidden-column header-tooltip bug would stop being invisible:— which is the same bug this PR names as class 4 two paragraphs above.BeanNetworkTester/tests/fake_tk.py
Lines 557 to 579 in 728729f
Only "no style validation" holds: install() maps ttk.Style to a bare W, so style names and options are never checked and theme_use/map/layout are no-ops. The other true gap is that winfo_width/winfo_height/winfo_geometry return constants, so nothing measures real wrapping, clipping or overflow.
Suggested rewrite that keeps the intent and names the blind spot that is actually there:
- The fake tkinter models behaviour, not pixels.
tests/fake_tk.pymodels the widgets
the app depends on and records geometry and column calls, so layout and column mapping
are covered (tests/test_gui_layout.py). What it cannot see needs a real renderer:
ttk.Styleis a bare widget, so style names and options are never validated and
theme_use/map/layoutare no-ops; andwinfo_width/winfo_height/winfo_geometry
are constants, so nothing measures real wrapping, clipping or overflow. Changes to
gui/theme.py, or to anything that depends on measured text, deserve a closer read than
their green suite suggests.
There was a problem hiding this comment.
Correct, and verified against the source before changing anything: the docstring of tests/fake_tk.py says so itself, in the past tense. Fixed in 081cbcb, with the two blind spots that are real named instead - ttk.Style mapped to a bare widget, and winfo_width/height/geometry returning constants.
Steering a review away from the best-covered part of the GUI would have been worse than saying nothing, and this file becomes project memory for every later run, so it would have compounded. Thank you.
The same check turned up a second wrong claim in the header of that file, fixed in the same commit.
The review on this pull request caught it, with file and line citations, and it was right: the paragraph described the PREVIOUS fake. `tests/fake_tk.py` says so itself, in the past tense, at the top of its own docstring. Verified against the source before changing anything. It defines eleven classes, records `pack_info` and `grid_info`, raises when `before=`/`after=` names a non-sibling, and `Treeview.column` resolves a `"#N"` spec against the columns currently shown - which was added precisely so the hidden-column header-tooltip bug would stop being invisible, the same bug this file names two paragraphs above. Layout and column mapping are covered, and `tests/test_gui_layout.py` asserts on `pack_info` directly. Two blind spots are real and they are the two now named: `ttk.Style` is mapped to a bare widget, so style names and options are never validated; and `winfo_width`, `winfo_height` and `winfo_geometry` return constants, so nothing measures wrapping, clipping or overflow. Steering a reviewer away from the best-covered part of the GUI is worse than saying nothing, and this file becomes the project memory of every future run - so the error would have compounded. Also fixes a second claim in the same file, found while checking the first: the header said every rule below was already stated in CONTRIBUTING.md or the READMEs. After the failure classes were added that was no longer true. It now says what is true - every line comes from files already public here, the registry included. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It started as a review on every pull request. One run cost $2.92 - Opus 5, 13 turns, 6 minutes 42 seconds of model time - on a pull request carrying a single Markdown file. The cost tracks how far the reviewer explores, not how large the diff is: that run read the fake tkinter, the layout suite and their docstrings to check one claim. At this project's rate that is a standing monthly bill against a subscription with better uses. So the trigger is inverted rather than tuned. The workflow now waits for `@claude` in a comment and runs no other way: interactive mode, which is what dropping the `prompt` input selects, so it answers the question asked instead of running a fixed task. Reach for it on a change big enough that a second reader earns its keep. The turn count is deliberately not capped. When it is asked for, the deep read is the whole point; `timeout-minutes` is the backstop. Still read-only, still the maintainer only, and the rule digest is still copied into place - that step earns itself: on the run measured above the reviewer was pointed at the briefing and found that one of its own paragraphs described a stand-in that had since been rewritten, with file and line citations. The guard gets stronger rather than weaker. It used to say "must not review on every push"; it now says "must not run by itself at all" - no `pull_request` trigger, no schedule - because half a trigger is the same bill. Mutation entry replaced to match, and caught. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things, one decision: the automatic review is withdrawn, and the briefing it uses
is rewritten to say where this repository's bugs actually come from.
The review becomes something you ask for
🔴 One run cost $2.92. Measured here: Opus 5, 13 turns, 6 minutes 42 seconds of model
time, on a pull request carrying a single Markdown file. The cost tracks how far the
reviewer explores, not how large the diff is - that run read the fake tkinter, the layout
suite and their docstrings to check one claim. At this project's rate that is a standing
monthly bill against a subscription with better uses.
So the trigger is inverted rather than tuned. The workflow now waits for
@claudein acomment and runs no other way - interactive mode, which is what dropping the
promptinput selects, so it answers the question asked instead of running a fixed task. It is
the emergency exit rather than the front door.
The turn count is deliberately not capped: when it is asked for, the deep read is the
whole point.
timeout-minutesis the backstop. Still read-only, still the maintaineronly.
The guard gets stronger rather than weaker. It used to say "must not review on every
push"; it now says "must not run by itself at all" - no
pull_requesttrigger, noschedule - because half a trigger is the same bill.
The briefing says where the bugs are
The first version carried the conventions and nothing about failure modes, so a review
arrived knowing what it must not break and not where to look. The five classes it now
names are read off
tests/test_mutation_registry.pyrather than invented - lifecycleraces around targeting, view state drifting from fact, a warning that stops matching the
state it describes, a column registry drifting from the table it renders, and empty or
just-changed states.
🔴 First among them is the one no gate can see: a guard that cannot fail. An assertion
that would also pass over an empty set, a collector that returns nothing, a walk rooted
where a runner has no directory. The briefing gives the question to ask of every added
assertion - what input makes this red - and says outright that a test which would still
pass with the behaviour removed is a finding.
Kept short on purpose, since it is read on every run: the meta paragraph and the CI list
are cut back, and a what not to propose section is new, because the fastest way to
make a review worthless is to let it repeat the linter. No counts are written into the
prose - an unguarded number goes stale.
The briefing earned its keep before it was merged
The review ran on this branch and found a real defect in the briefing itself: one
paragraph described a stand-in that had since been rewritten.
tests/fake_tk.pysays soin its own docstring, in the past tense. Verified against the source before changing
anything - it models eleven classes, records
pack_infoandgrid_info, andTreeview.columnresolves a"#N"spec against the columns currently shown, which wasadded precisely so the hidden-column header-tooltip bug would stop being invisible. So
layout and column mapping are covered; the two blind spots that are real are named
instead.
Steering a reviewer away from the best-covered part of the GUI would have been worse than
saying nothing, and this file becomes project memory for every later run. A second wrong
claim in the same file, found while checking the first, is fixed in the same commit.
🤖 Generated with Claude Code