Skip to content

ci: make the review something you ask for, and tell it where the bugs are - #138

Merged
donislawdev merged 3 commits into
masterfrom
docs/review-briefing
Aug 19, 2026
Merged

ci: make the review something you ask for, and tell it where the bugs are#138
donislawdev merged 3 commits into
masterfrom
docs/review-briefing

Conversation

@donislawdev

@donislawdev donislawdev commented Aug 19, 2026

Copy link
Copy Markdown
Owner

Two things, one decision: the automatic review is withdrawn, and the briefing it uses
is rewritten to say where this repository's bugs actually come from.

The review becomes something you ask for

🔴 One run cost $2.92. Measured here: Opus 5, 13 turns, 6 minutes 42 seconds of model
time, on a pull request carrying a single Markdown file. The cost tracks how far the
reviewer explores, not how large the diff is - that run read the fake tkinter, the layout
suite and their docstrings to check one claim. At this project's rate that is a standing
monthly bill against a subscription with better uses.

So the trigger is inverted rather than tuned. The workflow now waits for @claude in a
comment and runs no other way - interactive mode, which is what dropping the prompt
input selects, so it answers the question asked instead of running a fixed task. It is
the emergency exit rather than the front door.

The turn count is deliberately not capped: when it is asked for, the deep read is the
whole point. timeout-minutes is the backstop. Still read-only, still the maintainer
only.

The guard gets stronger rather than weaker. It used to say "must not review on every
push"; it now says "must not run by itself at all" - no pull_request trigger, no
schedule - because half a trigger is the same bill.

The briefing says where the bugs are

The first version carried the conventions and nothing about failure modes, so a review
arrived knowing what it must not break and not where to look. The five classes it now
names are read off tests/test_mutation_registry.py rather than invented - lifecycle
races around targeting, view state drifting from fact, a warning that stops matching the
state it describes, a column registry drifting from the table it renders, and empty or
just-changed states.

🔴 First among them is the one no gate can see: a guard that cannot fail. An assertion
that would also pass over an empty set, a collector that returns nothing, a walk rooted
where a runner has no directory. The briefing gives the question to ask of every added
assertion - what input makes this red - and says outright that a test which would still
pass with the behaviour removed is a finding.

Kept short on purpose, since it is read on every run: the meta paragraph and the CI list
are cut back, and a what not to propose section is new, because the fastest way to
make a review worthless is to let it repeat the linter. No counts are written into the
prose - an unguarded number goes stale.

The briefing earned its keep before it was merged

The review ran on this branch and found a real defect in the briefing itself: one
paragraph described a stand-in that had since been rewritten. tests/fake_tk.py says so
in its own docstring, in the past tense. Verified against the source before changing
anything - it models eleven classes, records pack_info and grid_info, and
Treeview.column resolves a "#N" spec against the columns currently shown, which was
added precisely so the hidden-column header-tooltip bug would stop being invisible. So
layout and column mapping are covered; the two blind spots that are real are named
instead.

Steering a reviewer away from the best-covered part of the GUI would have been worse than
saying nothing, and this file becomes project memory for every later run. A second wrong
claim in the same file, found while checking the first, is fixed in the same commit.

🤖 Generated with Claude Code

The first version of the briefing carried the conventions and nothing about failure
modes, so a review arrived knowing what it must not break and not where to look. The
five classes it now names are read off the mutation registry rather than invented -
that file records every behaviour broken on purpose to prove a test catches it, and
the entries cluster hard: lifecycle races around targeting, view state drifting from
fact, a warning that stops matching the state it describes, a column registry drifting
from the table it renders, and empty or just-changed states.

First among them is the one no gate can see: a guard that cannot fail. An assertion
that would also pass over an empty set, a collector that returns nothing, a walk
rooted where a runner has no directory. The briefing gives the question to ask of
every added assertion - what input makes this red - and says outright that a test
which would still pass with the behaviour removed is a finding.

Two blind spots are named because a diff hides them. The fake tkinter models one
widget class with no style validation and no geometry, so changes to styles, geometry
or column mapping are not covered by the tests that look like they cover them. And the
suite runs on two platforms while a developer checks one, so a symbol named after one
system can raise on the other.

Paid for in tokens on every run, so it is not only longer: the meta paragraph and the
CI list are cut back, and a "what not to propose" section is added, because the fastest
way to make a review worthless is to let it repeat the linter. No counts are written
into the prose - an unguarded number goes stale.

Also drops the mention of where the maintainer's own notes live. The reason a reviewer
needs is that the file is not part of this repository, and that is all it now says.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Comment thread .github/claude-review-rules.md Outdated
Comment on lines +87 to +90
- **The fake tkinter has blind spots.** The suite drives a stand-in for Tk with one widget
class, no style validation and no geometry. Changes to styles, geometry or column mapping
are therefore **not** covered by the tests that appear to cover them, and deserve a closer
read than their green suite suggests.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This describes the previous fake, not the one in the tree — and because this file becomes CLAUDE.md for every review run, it will steer future reviewers away from the areas that are in fact best covered.

  • "one widget class"tests/fake_tk.py defines seven with real behaviour: W, Text, Root, Notebook, Paned, Menu, Treeview. Its own docstring says this in the past tense:
    either. The previous fake was a single ``__getattr__`` no-op widget, so no test
    could ever see a layout problem (``winfo_width`` was hard-coded to 400 and
    ``pack``/``grid`` did nothing). This one records geometry calls and models the
    handful of widgets whose behaviour the app actually depends on - which is what
    makes ``test_gui_layout`` / ``test_gui_state`` possible.
  • "no geometry"pack() records pack_info and even raises when before=/after= names a non-sibling; grid() records grid_info; Root.geometry() is modelled. Tests assert on it directly:
    bar = app.btn_start.master
    assert bar.pack_info["side"] == "top"
    bottom = bar.master
    assert bottom.pack_info["side"] == "bottom" # anchored, not negotiable
    assert bottom.pack_info.get("expand") in (None, False, 0)
    assert app.log_box.master is app.log_wrap and app.log_wrap.master is bottom
    # the notebook only gets what is left over
    assert app.nb.master.pack_info["expand"] is True
    """)
  • "column mapping"Treeview.column() resolves a "#N" spec against displayed_columns() exactly as Tk does, added specifically so the hidden-column header-tooltip bug would stop being invisible:
    def column(self, col, option=None, **kw):
    """Configure a column, or read one option back.
    🔴 A "#N" spec is a DISPLAY position and is resolved against the columns
    currently shown, exactly as Tk does (measured 2026-08-18). Without this
    the fake could not tell the two rules apart, and the header-tooltip bug -
    every column after a hidden one describing its neighbour - was invisible
    to the whole suite while being obvious on screen.
    """
    if option == "id" and str(col).startswith("#"):
    try:
    index = int(str(col).lstrip("#")) - 1
    except ValueError:
    return None
    shown = self.displayed_columns()
    return shown[index] if 0 <= index < len(shown) else None
    entry = self.cols.setdefault(col, {})
    if option is not None and not kw:
    return entry.get(option)
    entry.update(kw)
    return None
    — which is the same bug this PR names as class 4 two paragraphs above.

Only "no style validation" holds: install() maps ttk.Style to a bare W, so style names and options are never checked and theme_use/map/layout are no-ops. The other true gap is that winfo_width/winfo_height/winfo_geometry return constants, so nothing measures real wrapping, clipping or overflow.

Suggested rewrite that keeps the intent and names the blind spot that is actually there:

  • The fake tkinter models behaviour, not pixels. tests/fake_tk.py models the widgets
    the app depends on and records geometry and column calls, so layout and column mapping
    are covered (tests/test_gui_layout.py). What it cannot see needs a real renderer:
    ttk.Style is a bare widget, so style names and options are never validated and
    theme_use/map/layout are no-ops; and winfo_width/winfo_height/winfo_geometry
    are constants, so nothing measures real wrapping, clipping or overflow. Changes to
    gui/theme.py, or to anything that depends on measured text, deserve a closer read than
    their green suite suggests.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and verified against the source before changing anything: the docstring of tests/fake_tk.py says so itself, in the past tense. Fixed in 081cbcb, with the two blind spots that are real named instead - ttk.Style mapped to a bare widget, and winfo_width/height/geometry returning constants.

Steering a review away from the best-covered part of the GUI would have been worse than saying nothing, and this file becomes project memory for every later run, so it would have compounded. Thank you.

The same check turned up a second wrong claim in the header of that file, fixed in the same commit.

donislawdev and others added 2 commits August 19, 2026 17:32
The review on this pull request caught it, with file and line citations, and it was
right: the paragraph described the PREVIOUS fake. `tests/fake_tk.py` says so itself, in
the past tense, at the top of its own docstring.

Verified against the source before changing anything. It defines eleven classes, records
`pack_info` and `grid_info`, raises when `before=`/`after=` names a non-sibling, and
`Treeview.column` resolves a `"#N"` spec against the columns currently shown - which was
added precisely so the hidden-column header-tooltip bug would stop being invisible, the
same bug this file names two paragraphs above. Layout and column mapping are covered, and
`tests/test_gui_layout.py` asserts on `pack_info` directly.

Two blind spots are real and they are the two now named: `ttk.Style` is mapped to a bare
widget, so style names and options are never validated; and `winfo_width`, `winfo_height`
and `winfo_geometry` return constants, so nothing measures wrapping, clipping or overflow.

Steering a reviewer away from the best-covered part of the GUI is worse than saying
nothing, and this file becomes the project memory of every future run - so the error
would have compounded.

Also fixes a second claim in the same file, found while checking the first: the header
said every rule below was already stated in CONTRIBUTING.md or the READMEs. After the
failure classes were added that was no longer true. It now says what is true - every line
comes from files already public here, the registry included.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It started as a review on every pull request. One run cost $2.92 - Opus 5, 13 turns,
6 minutes 42 seconds of model time - on a pull request carrying a single Markdown
file. The cost tracks how far the reviewer explores, not how large the diff is: that
run read the fake tkinter, the layout suite and their docstrings to check one claim.
At this project's rate that is a standing monthly bill against a subscription with
better uses.

So the trigger is inverted rather than tuned. The workflow now waits for `@claude` in
a comment and runs no other way: interactive mode, which is what dropping the `prompt`
input selects, so it answers the question asked instead of running a fixed task. Reach
for it on a change big enough that a second reader earns its keep.

The turn count is deliberately not capped. When it is asked for, the deep read is the
whole point; `timeout-minutes` is the backstop.

Still read-only, still the maintainer only, and the rule digest is still copied into
place - that step earns itself: on the run measured above the reviewer was pointed at
the briefing and found that one of its own paragraphs described a stand-in that had
since been rewritten, with file and line citations.

The guard gets stronger rather than weaker. It used to say "must not review on every
push"; it now says "must not run by itself at all" - no `pull_request` trigger, no
schedule - because half a trigger is the same bill. Mutation entry replaced to match,
and caught.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@donislawdev donislawdev changed the title docs(review): tell the reviewer where this repository's bugs come from ci: make the review something you ask for, and tell it where the bugs are Aug 19, 2026
@donislawdev
donislawdev merged commit e9c6f18 into master Aug 19, 2026
13 of 14 checks passed
@donislawdev
donislawdev deleted the docs/review-briefing branch August 19, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant