You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(agents): gates for per-mark label density and the plot language's number format - #12184
Block H of the agent chat: two findings of the live tests of 2026-10-14/15 become measured gates, with the rules behind them in the adapter's, reviewer's and scorer's prompts.
G11, label density (VQ-03). The catalogue draws some elements once per mark of its few example marks (bar-grouped: a value label, a dot and a shadow on each of 12 bars); the adapter kept them on 52 bars of 26 regions. New harness.density_report (probe field density, a separate helper called from the savefig wrapper next to polar_report, outside probe()) counts per axes the bars and points, the texts at a data coordinate anchored on a bar's end or a point (heatmap cell text and axes-fraction callouts are not value labels), dots on bar ends and shadowed bars, and the median bar-label width against the space per bar. The gate fires on more than 25 value labels, on bar labels wider than 90 % of the space per bar (from 6 labels on), or on more than 25 dots or shadows.
G12, number format (VQ-06). Numbers followed English whatever the plot language ("6.0 M", "5.30" in German plots); locale stays banned. language.NUMBER_FORMATS is a server-side table of decimal mark, thousands mark and large-number words for the 12 languages of ENGLISH_NAMES (de: ,. Tsd./Mio./Mrd.), rendered into the adapter's static instruction with a "Numbers" rule (FuncFormatter or an f-string whose marks are swapped through a placeholder). G12 reads the probe's labels field from fix(agents): ground replies in the render, bindings and data source #12183 (block F, as agreed: no text field of its own) plus a new probe field tick_axes (harness.tick_axes, per axis: number, date, category or code-set labels), and flags annotations and tick labels of numeric axes whose decimal mark is the other language's (mathtext read as drawn); dates, years, version-like strings, titles, axis titles and legend entries never count, and no label text reaches the line.
One source for the exemption. The house-style rule of scorer.md (which policy.adapter_rubric copies to the adapter) and reviewer.md now say that an element whose count grows with the data is not house style. Not moved into prompts/default-style-guide.md: plot language is an agent-only concept, and a density rule there would change the catalogue pipeline and need a review-retest arm; see owner decisions below.
Fixtures. Four hand-written eval cases generalise the live cases: bar-grouped-matplotlib-many-categories-de, bar-basic-seaborn-decimals-de with its English control -en, and line-multi-matplotlib-millions-de (follow-up "Zeig die y-Achse in Millionen, zum Beispiel 6,5 Mio."). Unit tests run the density report on real matplotlib figures (the live 52-bar case fires, the catalogue's own 12 bars, heatmap cells, a single callout and a long labelled line are checked) and G12 on de and en label sets.
Deploy order: G11 needs the renderer image rebuilt (new harness field); G12 needs #12183's labels. Without them both gates stay silent (an absent field gives no line), so merging before the renderer deploy is safe.
Measurements
scripts/agents-experiment.sh, 7 cases x 1 run per arm, deployed renderer, adapter and reviewer on the Claude CLI, root on Vertex eu Haiku (about $0.03 of Vertex spend for both arms), scores by agents.evals.score (Opus, catalogue rubric). Before = origin/main9c2cf35, after = this branch. The deployed renderer has neither density nor labels, so these arms measure the prompt and rubric side; the gates themselves are measured by the in-process fixtures.
Case
Before: score first → shipped, status, attempts, time
After: score first → shipped, status, attempts, time
before: v1 raw integers, Mio. only after the follow-up; after: 6,0 Mio. from v1
bar-basic-seaborn-decimals-en (control)
87 → 87, ok, 1, 70.3 s
83 → 83, ok, 1, 60.5 s
5.30 in both
bar-grouped-seaborn (control)
84 → 83, ok, 1, 76.7 s
83 → 84, ok, 1, 86.1 s
unchanged
scatter-basic-matplotlib (control)
83 → 84, ok, 2, 128.6 s
81 → 84, ok, 2, 123.0 s
unchanged
stress-bar-100-categories
77 → 49, needs_attention, 2, 218.3 s
67 → 83, ok, 1, 148.3 s
before: shadows on 100 bars
Metric
Before
After
Shipped score, mean
78.3
84.3
Status ok
5 of 7
7 of 7
Runs with a repair attempt
3 of 7
1 of 7
Reviewer defect verdicts
2
0
Gate lines G3 / G7
4 / 4
2 / 2
End to end p50 / p95
128.6 s / 213.0 s
123.0 s / 220.0 s
One run per case: the controls move by up to 4 points either way, so read the per-case deltas on the three target cases and the status counts, not the control noise.
Owner decisions (2026-10-15) and follow-ups
G11 thresholds stay: 25 value labels, 90 % of the space per bar, 25 dots or shadows.
de writes the point as thousands mark and 6,5 Mio.. Follow-up: a region setting (de-CH with the apostrophe, 7'184'003) as its own session setting.
Follow-up: the per-bar density rule also goes into prompts/default-style-guide.md, in a separate PR with a review-retest run (not in this PR).
Tick labels the code sets itself (set_yticklabels([...])) stay unchecked by G12.
Applied, each with a unit-test reproducer: named points no longer count against the cap (only numeric labels without bars); the cap counts labelled bar positions (stacked segments once); centre labels inside stacked segments are counted; the space per label comes from neighbouring labelled bars (offset shadow series, extremes-only labelling); trailing currency and sign-before-currency are numbers; 0.125 is a decimal; a zero-padded 14.10 annotation is a date; bar ends from data coordinates (inverted axes); plot(..., "o") markers count as dots; clipped ax.text outside the view is skipped; the adapter's mark swap goes through a placeholder (exact for fr/pl/ru).
Follow-ups (not in this PR):
A shadow drawn as a second, offset bar container (Fable b05) is not yet a decoration; only path-effect shadows count.
An annotation anchored in axes fraction with its text placed in data coordinates (Fable b04) is not matched to a bar.
A ScalarFormatter tick such as 1.002 (offset axes, Fable c04) reads as a German thousands group and passes.
Plan
N/A (block H of the live-test follow-ups; design in docs/concepts/agent-network.md, new paragraph "Label density and number formats").
Test plan
uv run ruff check . and uv run ruff format --check .
uv run --extra typecheck --extra agents mypy api core agents --pretty
uv run --extra agents --extra test pytest tests/unit tests/integration -q (10,932 passed)
tests/unit/agents/runtime/test_density_locale_gates.py: density report on real figures, G11/G12 lines in the defect grammar, tampered probes, de/en fixtures, the adapter instruction carries the table and rules
Before/after experiment arms (tables above)
After the renderer deploy: one live German many-category session to see G11 and G12 fire on the real probe
…ber format
Two findings of the live tests of 2026-10-14/15 become measured gates:
- G11 (VQ-03): the harness's density_report counts per axes the bars and
points, the data-anchored value labels on them, dots on bar ends and
shadowed bars, and the space a bar label has. More than 25 labels, bar
labels wider than 90 % of the space per bar, or more than 25 dots or
shadows give a measured line. The scorer's, reviewer's and adapter's
house-style exemption no longer covers elements whose count grows with
the data.
- G12 (VQ-06): a numeric tick label or annotation whose decimal mark is
not the plot language's (5.30 and 6.0 M in a German plot), read from the
probe's `labels` field. language.NUMBER_FORMATS is the server's table of
decimal and thousands marks and large-number words, carried in the
adapter's static instruction with the rule to format through a
FuncFormatter or an f-string (locale stays banned).
Four hand-written eval cases generalise the live cases (de and an en
control).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
`harness.drawn_labels` (#12183) names tick labels by axis (`x tick label`,
`y tick label`, `z tick label`, `colorbar tick label`); G12 now checks every
kind ending in "tick label" plus annotations and texts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
- density_report: bars, points and annotation anchors outside the axes view
(a zoomed plot) and non-finite bar geometry (a NaN value) are left out,
so they neither count nor void the probe's JSON; scatter marks are
matched to bar ends in chunks of 2,000, so the pairwise arrays stay
bounded however many marks a plot draws.
- New probe field `tick_axes` (harness.tick_axes): per axes, whether each
axis's tick labels are numbers, dates, categories or labels the code set
itself. G12 checks tick labels of numeric axes only, so a German `%d.%m`
date (14.10) or a category the data names (3.11) is never a finding;
without the field only annotations count.
- G12 reads a mathtext label as drawn (`$\mathdefault{5.30}$` is 5.30).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
Copilot round 1: all five applied in 761144b. Scatter marks are matched to bar ends in bounded chunks (memory); NaN bar geometry is skipped (probe JSON stays valid); bars and anchors outside the view are skipped unless annotation_clip=False (zoomed plots); a new probe field tick_axes makes G12 check only tick labels of numeric axes (no dates like 14.10, no data categories like 3.11); mathtext is read as drawn. Each has a regression test in tests/unit/agents/runtime/test_density_locale_gates.py.
Scientific notation bypasses G12 even when its mantissa uses the wrong decimal mark. Both 5.30e+06 and $\mathdefault{5.30\times10^{6}}$ fail this pattern, so numeric ticks or annotations containing them produce no finding in a German plot. Extend number parsing and mathtext normalization to recognize exponents and check the mantissa. Add foreign-decimal and correctly localized controls while preserving date and version exclusions.
Inverted value axes place bar labels on the wrong edge
agents/anyplot/render/harness.py:1236
The data-space sign selects the wrong display-space edge when the value axis is inverted. After invert_yaxis(), a positive vertical bar ends at extent.y0, but this selects extent.y1, its baseline; horizontal bars have the analogous failure after invert_xaxis(). Labels and dots then fail to match their bars, so G11 can miss a crowded chart. Transform the rectangle's endpoint into display coordinates and test both inverted orientations.
All path effects are incorrectly counted as shadows
agents/anyplot/render/harness.py:1239
A nonempty path-effect list does not necessarily draw a shadow: withStroke(...) draws an outline, and Normal() adds no decoration. On 26 otherwise undecorated bars, either is counted as 26 shadows, triggering G11 and an unnecessary repair or needs_attention result. Count shadow effects rather than all path effects, and add outline-only and Normal() controls.
- A bar counts when its rectangle meets the view, so bars zoomed on their
value axis (ylim 900 to 1100 over bars ending at 1000) keep their labels.
- A date formatter marks the axis as dates before the converter check, so
date2num coordinates under DateFormatter("%d.%m") are never numbers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
Copilot round 2: both applied in 4ed334d (bars count when their rectangle meets the view, so a value-axis zoom keeps them; a date formatter marks an axis as dates before the converter check, so date2num coordinates are never numbers), each with a regression test. Per the one-extra-round rule no further review is requested.
The merged harness writes block F's `labels` next to `tick_axes`; a German
bar chart with "5.30" labels fires G12 and the comma-formatted one does not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
… gates
G11:
- Named points no longer count against the 25-label cap: without bars only
numeric labels count (`numeric_labels`); scatter-annotated asks for 10 to 50.
- With bars the cap counts labelled bar positions (`labelled_bars`), so a
stacked bar's segment labels count once.
- Labels inside a bar segment (`bar_label(label_type="center")`, a callable
xycoords) are counted; they were missed.
- The space per bar label comes from the labelled bars' nearest neighbours,
so offset shadow bars or a chart that labels only its extremes no longer
shrink it.
- Bar ends come from the bars' data coordinates (inverted axes), markers of
`plot(..., "o")` count as dots, a clipped `ax.text` outside the view is
skipped.
G12:
- A trailing currency (`5.30 €`) and a sign before the currency (`-$5.30`)
are numbers; `0.125` is never a thousands group; a zero-padded `14.10`
annotation reads as a day and month.
Adapter prompt: the marks are swapped through a placeholder
(`.replace(",", "_").replace(".", DECIMAL).replace("_", THOUSANDS)`), exact
for space-grouped languages. Each reproducer is a unit test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
The per-mark label rule #12184 gave the agent adapter now lives once in
prompts/default-style-guide.md ("Per-mark Labels and Decorations"), with
a proportional VQ-03 paragraph in quality-criteria.md and a 5d bullet in
ai-quality-review.md, so the catalogue pipeline applies it too. The
adapter prompt points at the shared rule instead of carrying a copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
…12189)
## Summary
- **One source for the per-mark label rule.** #12184 gave the agent
adapter a rule for elements the catalogue draws once per bar or point
(value labels, dots on bar ends, shadows): keep them only while each
fits its mark, which rarely holds past about 25 marks; otherwise drop
them or label the extremes. Per the owner's decision of 2026-10-15, the
rule now lives in `prompts/default-style-guide.md` ("Per-mark Labels and
Decorations"), so the catalogue pipeline applies it too.
- **Rubric, proportional and soft.** `prompts/quality-criteria.md` VQ-03
gains a "Per-mark labels and decorations count as elements" paragraph
(deduct VQ-03 in proportion to how much of the plot they bury, VQ-02
where they overlap; labels that fit, or a few labelled extremes, are no
deduction), and the 5d proportional sizing check of
`ai-quality-review.md` gets a matching bullet. No new hard threshold:
"about 25 marks" is a heuristic like the data-density heuristic's "< 50
points"; the agent's G11 gate keeps its own measured thresholds in code.
- **Agent prompts.** `agents/anyplot/prompts/adapter.md` keeps its
agent-specific framing ("the same style" covers fixed-count elements)
and points at the shared rule instead of repeating it; the adapter reads
the style guide and the VQ-03 rubric verbatim. The scorer's and
reviewer's narrowed house-style exemption stays as is (it is the
agent-only exception, not the rule). A unit test holds that the "about
25 marks" wording appears once in the adapter's instruction.
`docs/concepts/agent-network.md` names the new source.
## Review retest
The last all-Opus set v1 arm (37233543394, rules `c64630aa7`) predates
the claude-code-action pin bump (#11975/#12099) and the rubric change of
#12102, so a fresh baseline was run on `main`'s rules (`e2a63d6`, run
[38702205512](https://github.com/MarkusNeusinger/anyplot/actions/runs/38702205512)).
To keep the spend near one core arm, both arms ran the 15 fresh core
items plus `f-bar-stacked-labeled-d3` and `f-heatmap-basic-seaborn`
(labelled plots, as false-alarm probes), 3 runs each, all on
`claude-opus-5-5`, without the bubble regen pairs, which carry no
per-mark labels.
### Review retest — set v1
f-bubble-basic-ggplot2,f-area-elevation-profile-seaborn,f-bar-horizontal-makie,f-bode-basic-altair,f-bar-diverging-plotnine,f-pie-basic-highcharts,f-bar-pareto-letsplot,f-waterfall-basic-muix,f-line-win-probability-chartjs,f-bubble-basic-d3,f-network-basic-bokeh,f-bar-diverging-likert-matplotlib,f-acf-pacf-plotly,f-line-stock-comparison-pygal,f-gantt-basic-echarts,f-bar-stacked-labeled-d3,f-heatmap-basic-seaborn
(17 fresh × 3)
Fresh: claude-opus-5-5 · rules e2a63d6 → 35caaaf · harness v1
| Metric | Baseline | Candidate | Δ (95% CI) |
|---|---|---|---|
| Total mean / pooled within-item SD (typed) | 79.2 / 1.4 | 79.3 / 1.5 |
0.0 (-0.8…+0.9) / 0.1 (-0.5…+0.8) |
| Items with split verdict at 90 | 0/17 | 0/17 | |
| Noisiest criteria (SD) | VQ-01 0.5 · DE-02 0.5 · LM-02 0.4 | VQ-01 0.5
· VQ-03 0.5 · VQ-05 0.5 | |
| Weakness-topic Jaccard | 0.61 | 0.62 | |
| Silent deductions (technical items, fresh) | 0/178 (0%) | 0/174 (0%) |
|
| Named-defect miss rate | 17/69 | 15/69 | |
| False alarms on permitted features | 0/9 | 1/9 | |
| Sessions / API-equivalent cost | 51/51 / $37 | 51/51 / $41 | |
[run](https://github.com/MarkusNeusinger/anyplot/actions/runs/38705105133)
· set v1 lock sha256 a6831a315a11
**Reading.** Neutral, as intended for a set with no crowded labels. No
candidate weakness cites per-mark label density; the labelled plots
(`f-bar-stacked-labeled-d3`, `f-bar-horizontal-makie` value labels,
`f-bar-diverging-plotnine` value labels, `f-heatmap-basic-seaborn` cell
text) keep their VQ-03 scores (6/6/6, 6/6/6, 6/6/6, 5–6). VQ-03's mean
stays at 5.4. VQ-03's within-item SD (0.31 → 0.46) moves on
`f-bode-basic-altair` (one 4) and `f-line-stock-comparison-pygal` (4 to
6), whose VQ-03 comments name a bright dark-theme grid, thin lines and
small legend swatches, not labels. The one false alarm is
`f-bubble-basic-ggplot2` A1, run 1: an SC-01 line about the off-spec
top-3 callout layer says the callouts "sit on top of the dense bubble
clusters", which the probe's overlap regex catches; it is not a per-mark
label deduction. The set has no crowded-label item, so the retest shows
that the rule does not take points from plots that pass it, but it
cannot measure detection. G11's fixtures in #12184 cover the crowded
case on the agent side.
## Plan
N/A
## Test plan
- [x] `uv run pytest tests/unit -n 4` (11,037 passed)
- [x] `ruff check`, `ruff format --check`, `tools.changelog check --base
origin/main`
- [x] Review retest candidate vs baseline: no per-mark label deductions
on the labelled plots, no rise in silent deductions, total within ±0.9
points
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Block H of the agent chat: two findings of the live tests of 2026-10-14/15 become measured gates, with the rules behind them in the adapter's, reviewer's and scorer's prompts.
harness.density_report(probe fielddensity, a separate helper called from the savefig wrapper next topolar_report, outsideprobe()) counts per axes the bars and points, the texts at a data coordinate anchored on a bar's end or a point (heatmap cell text and axes-fraction callouts are not value labels), dots on bar ends and shadowed bars, and the median bar-label width against the space per bar. The gate fires on more than 25 value labels, on bar labels wider than 90 % of the space per bar (from 6 labels on), or on more than 25 dots or shadows.localestays banned.language.NUMBER_FORMATSis a server-side table of decimal mark, thousands mark and large-number words for the 12 languages ofENGLISH_NAMES(de:,.Tsd./Mio./Mrd.), rendered into the adapter's static instruction with a "Numbers" rule (FuncFormatter or an f-string whose marks are swapped through a placeholder). G12 reads the probe'slabelsfield from fix(agents): ground replies in the render, bindings and data source #12183 (block F, as agreed: no text field of its own) plus a new probe fieldtick_axes(harness.tick_axes, per axis: number, date, category or code-set labels), and flags annotations and tick labels of numeric axes whose decimal mark is the other language's (mathtext read as drawn); dates, years, version-like strings, titles, axis titles and legend entries never count, and no label text reaches the line.scorer.md(whichpolicy.adapter_rubriccopies to the adapter) andreviewer.mdnow say that an element whose count grows with the data is not house style. Not moved intoprompts/default-style-guide.md: plot language is an agent-only concept, and a density rule there would change the catalogue pipeline and need a review-retest arm; see owner decisions below.bar-grouped-matplotlib-many-categories-de,bar-basic-seaborn-decimals-dewith its English control-en, andline-multi-matplotlib-millions-de(follow-up "Zeig die y-Achse in Millionen, zum Beispiel 6,5 Mio."). Unit tests run the density report on real matplotlib figures (the live 52-bar case fires, the catalogue's own 12 bars, heatmap cells, a single callout and a long labelled line are checked) and G12 on de and en label sets.Deploy order: G11 needs the renderer image rebuilt (new harness field); G12 needs #12183's
labels. Without them both gates stay silent (an absent field gives no line), so merging before the renderer deploy is safe.Measurements
scripts/agents-experiment.sh, 7 cases x 1 run per arm, deployed renderer, adapter and reviewer on the Claude CLI, root on Vertex eu Haiku (about $0.03 of Vertex spend for both arms), scores byagents.evals.score(Opus, catalogue rubric). Before =origin/main9c2cf35, after = this branch. The deployed renderer has neitherdensitynorlabels, so these arms measure the prompt and rubric side; the gates themselves are measured by the in-process fixtures.12.5005.30, ticks5; after:5,30, ticks5,0Mio.only after the follow-up; after:6,0 Mio.from v15.30in bothOne run per case: the controls move by up to 4 points either way, so read the per-case deltas on the three target cases and the status counts, not the control noise.
Owner decisions (2026-10-15) and follow-ups
dewrites the point as thousands mark and6,5 Mio.. Follow-up: a region setting (de-CH with the apostrophe, 7'184'003) as its own session setting.prompts/default-style-guide.md, in a separate PR with a review-retest run (not in this PR).set_yticklabels([...])) stay unchecked by G12.Fable + Gemini review (folded in, 480cc62)
Applied, each with a unit-test reproducer: named points no longer count against the cap (only numeric labels without bars); the cap counts labelled bar positions (stacked segments once); centre labels inside stacked segments are counted; the space per label comes from neighbouring labelled bars (offset shadow series, extremes-only labelling); trailing currency and sign-before-currency are numbers;
0.125is a decimal; a zero-padded14.10annotation is a date; bar ends from data coordinates (inverted axes);plot(..., "o")markers count as dots; clippedax.textoutside the view is skipped; the adapter's mark swap goes through a placeholder (exact for fr/pl/ru).Follow-ups (not in this PR):
1.002(offset axes, Fable c04) reads as a German thousands group and passes.Plan
N/A (block H of the live-test follow-ups; design in
docs/concepts/agent-network.md, new paragraph "Label density and number formats").Test plan
uv run ruff check .anduv run ruff format --check .uv run --extra typecheck --extra agents mypy api core agents --prettyuv run --extra agents --extra test pytest tests/unit tests/integration -q(10,932 passed)tests/unit/agents/runtime/test_density_locale_gates.py: density report on real figures, G11/G12 lines in the defect grammar, tampered probes, de/en fixtures, the adapter instruction carries the table and rules