You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(core): adaptability checker for Python implementations - #12244
core/adaptability.py, the stdlib-only checker of the adaptable-code convention, implements rules 1 to 6 with the amendments of 2026-10-23. It returns a record {version, status, findings, block, constants, color_role, shape} and writes at most one CQ-01 (code) defect line per rule class, with advisory lines marked (advisory). python -m core.adaptability report plots/ [--sample N] prints the baseline below.
core/spec_roles.py is the agent's role parser, moved, not copied. The agent and its tests import it from core. It adds data_bullets (pinned to sync_to_postgres.py on every spec), the shape rule with SHAPE_OVERRIDES and MAP_SPECS, and the colour-driving role.
automation/scripts/adaptability_check.py is the single-file mirror for the workflow runner. core/adaptability_mirror.py generates it, and a test pins the file to the build, keeps it stdlib-only, runs it with python -I where core cannot be imported, and compares its records with the module's on a catalogue sample.
core/perturbation.py builds the smoke run's tables and the mutated source. It has no nullable-dtype perturbation. core/render_identity.py computes the share of differing pixels against a bound, with a small CLI.
No workflow, prompt, rubric or score changes. The pipeline does not run any of this until PR 2.
Plan
docs/concepts/adaptable-code.md, "Rollout" item 1, with the decisions of 2026-10-23 from the brief. Only the status line of that file changes here.
Catalogue baseline
uv run python -m core.adaptability report plots/ on main at f2b4629:
Library
Files
Pass
Fail
Excluded
matplotlib
325
8
304
13
seaborn
325
2
310
13
plotly
325
7
305
13
altair
325
0
312
13
plotnine
321
0
308
13
bokeh
325
3
309
13
pygal
325
2
310
13
letsplot
325
0
312
13
all
2596
22
2470
104
Files with a finding, per rule class:
Rule
Class
matplotlib
seaborn
plotly
altair
plotnine
bokeh
pygal
letsplot
All
1
layout
299
281
300
295
269
308
310
288
2350
2
columns
281
281
281
281
277
281
281
281
2244
3
interface
191
120
117
25
136
80
33
110
812
4
column-literal
12
90
12
32
170
53
0
132
501
4
sample-literal (advisory)
11
25
13
12
21
11
3
14
110
4
data-coordinate (advisory)
120
114
117
86
127
91
77
98
830
4
statistic-text (advisory)
8
9
8
7
6
10
12
4
64
4
df-comparison
0
9
0
1
3
1
0
1
15
4
count-literal (advisory)
5
4
7
1
7
1
2
5
32
5
palette
47
51
46
44
47
44
37
43
359
6
constants (advisory)
0
0
0
0
0
0
0
0
0
Specs per shape: 278 table, 19 generated, 9 network, 6 matrix, 13 excluded. That matches the panel's count of 278 tables. All 22 passing files belong to generated specs, where the design asks only for the markers, a clean block and the palette (see "Findings").
Rule 1 fails mostly on missing markers, as the table below shows. Altair is the extreme case: only 43 of its 325 files have a # Plot marker.
Layout finding
Files
# Data marker with no # Plot after it
1,223
A block that does not end by binding df
780
No # Data marker at all
325
A plotting, styling or theme token inside the block
309
Precision
A finding is judged true when the literal or structure ties the code to the catalogue's own data, so that other data would break it or show stale text. Each sample draws random findings, at most one per file, from the catalogue with a fixed seed (--sample N --seed S). The per-class verdicts use the samples taken after the last change to each class.
Class
Rule
Judged after the fixes
True
Precision
Verdict
layout
1
30 (seed 7)
30
100 %
binding
columns
2
30 (seed 4)
30
100 %
binding
interface
3
30 (seed 7)
30
100 %
binding
column-literal
4
35 (seeds 5, 7)
33
94 %
binding
df-comparison
4
15 (every file)
15
100 %
binding
palette
5
30 (seed 5)
28
93 %
binding
sample-literal
4
35 (seeds 4, 7)
30
86 %
advisory
count-literal
4
35 (seeds 3, 7)
26
74 %
advisory
data-coordinate
4
35 (seeds 2, 7)
22
63 %
advisory
statistic-text
4
35 (seeds 3, 7)
12
34 %
advisory
constants
6
none: no catalogue file has a # Constants zone
advisory by decision
The verdict is per class, not per rule. Rule 4 as a whole judges 22 of 30 on the stratified sample below, but two of its classes clear 90 % and stay binding. These fixes came from earlier samples:
df-comparison went from 14 of 25 to 15 of 15 after inequalities stopped counting. A threshold such as df[RSI] >= 70 or if w > 7 holds for any data.
palette went from 23 of 30 to 28 of 30. A missing palette now counts only for a required colour role. A strict zip over a palette sliced to len(groups) is allowed. entity is no longer a colour role, since slope charts colour by direction.
data-coordinate improved from 21 of 30 through four changes. Files with hidden axes are drawings. Padding beside a computed bound is ignored. range= in alt.Scale and scale_size is visual. PIL's draw.text((x, y)) is pixels. The class still stays at 63 %: annotation positions in layout units and domain constants (pH 0 to 14, a rebased-100 line, 45° phase ticks) cannot be separated statically.
sample-literal stopped counting record field names and one-character values. Its misses are domain vocabulary that the block also spells out: ECG lead names, Venn zones, spectral classes, QR module types.
Rule 1, layout: 30 lines, seed 7 (30 TP)
TP band-basic/letsplot:49 # Data with no # Plot after it
TP candlestick-basic/pygal no # Data marker
TP curve-bias-variance-tradeoff/altair:32 no # Plot
TP curve-dose-response/bokeh:77 block ends with fold_diff = ec50_b / ec50_a
TP curve-dose-response/pygal:58 no # Plot
TP drawdown-basic/matplotlib:78 block ends with a for loop over drawdown
TP ecdf-basic/bokeh:50 block ends with a ColumnDataSource
TP gauge-basic/seaborn:37 no # Plot
TP heatmap-calendar/bokeh:80 LinearColorMapper (styling) inside the block
TP heatmap-clustered/bokeh:48 no # Plot
TP heatmap-periodic-table/altair no # Data marker
TP hexbin-basic/pygal:48 no # Plot
TP histogram-epidemic/plotnine:92 block ends with a label string
TP histogram-kde/plotly:27 no # Plot
TP indicator-ichimoku/altair:36 no # Plot
TP ks-test-comparison/letsplot:96 theme token INK_MUTED inside the block
TP line-annotated-events/matplotlib:31 block ends with an events list
TP line-multi/bokeh:29 no # Plot
TP network-basic/bokeh:143 block ends with a layout loop
TP network-weighted/altair:24 no # Plot
TP phase-diagram/seaborn:61 block ends with sns.set_theme(...)
TP point-and-figure-basic/plotnine:123 block ends with a second frame (trend_lines)
TP radar-multi/seaborn:52 no # Plot
TP scatter-embedding/pygal:47 block ends with centroids
TP scatter-pitch-events/seaborn:78 theme token IMPRINT_PALETTE inside the block
TP shap-waterfall/letsplot:27 no # Plot
TP smith-chart-basic/pygal:76 no # Plot
TP streamgraph-basic/altair:24 no # Plot
TP venn-basic/matplotlib:29 block ends with a_and_b_and_c = 10
TP violin-swarm/bokeh:31 no # Plot
Rule 2, columns: 30 lines, seed 4 (30 TP; no catalogue file has role constants yet)
Rule 4, literals below the block: 30 lines, seed 7, five per class (22 TP)
TP column-literal datamatrix-basic/plotnine:80 aes(x="x", ...)
TP column-literal dot-matrix-proportional/plotnine:62 aes(..., y="y")
TP column-literal heatmap-rainflow/plotnine:93 aes(x="Mean Stress (MPa)")
TP column-literal span-basic/plotnine:68 aes(y="price")
TP column-literal stock-event-flags/letsplot:155 aes(y="flag_y") on a frame the block builds
TP sample-literal bar-spine/plotly:55 for outcome in ["Active", "On Trial", "Cancelled"]
TP sample-literal histogram-epidemic/plotnine:155 {"Probable": ...} color map
TP sample-literal scatter-pitch-events/bokeh:172 row["event_type"] == "pass"
TP sample-literal scatter-pitch-events/seaborn:156 df["Outcome"] == "Successful"
TP sample-literal sunburst-basic/bokeh:194 inner_radius="inner_radius", a field the block builds
FP data-coordinate bode-basic/matplotlib:136 phase ticks every 45° (domain ticks)
TP data-coordinate histogram-kde/letsplot:63 geom_rect band at density 0.28
FP data-coordinate line-stock-comparison/plotnine:84 geom_hline(yintercept=100), the rebased baseline
TP data-coordinate phase-diagram-pt/plotnine:183 annotate("text", x=218, y=5e7, label="SOLID")
FP data-coordinate titration-curve/plotnine:143 pH breaks 0..14 (domain scale)
TP statistic-text curve-oc/pygal:123 "n=100, c=3" plan typed as text
FP statistic-text ks-test-comparison/altair:76 "p < 0.001" fallback beside a computed f-string
FP statistic-text ks-test-comparison/bokeh:129 same
FP statistic-text ma-differential-expression/pygal:189 "Upregulated (p<0.05)", a threshold
FP statistic-text skewt-logp-atmospheric/pygal:149 "Mixing Ratio r=10g/kg", chart furniture
TP df-comparison errorbar-basic/altair:84 df["category"] == "Treatment D"
TP df-comparison funnel-meta-analysis/seaborn:141 row["study"] == "Martinez 2020"
TP df-comparison indicator-ema/plotnine:62 df["series"] == "Close Price"
TP df-comparison line-stepwise/letsplot:63 df["hour"].isin([6, 9, 12, 15, 18, 21])
TP df-comparison scatter-hr-diagram/seaborn:149 df["Region"] == "Main Sequence"
FP count-literal ecg-twelve-lead/plotly:164 range(4) rows of the 12-lead layout
TP count-literal line-basic/plotnine:53 breaks=range(1, 13), the sample's months
TP count-literal line-filled/plotnine:52 breaks=range(1, 13)
TP count-literal ridgeline-basic/plotnine:137 breaks=range(12)
TP count-literal scatter-constellation-diagram/matplotlib:54 range(4), the sample's 16-QAM grid
Rule 5, palette: 30 lines, seed 5 (28 TP)
TP andrews-curves/plotnine:34 IMPRINT 3 of 8
TP area-stacked-confidence/plotly no IMPRINT list, category colors the stack
TP area-stacked-percent/matplotlib:21 5 of 8
TP bar-3d-categorical/altair:25 5 of 8
TP bar-3d-categorical/letsplot:23 3 of 8
TP bar-grouped/seaborn:28 3 of 8
TP bar-stacked-labeled/pygal:28 4 of 8
TP bar-stacked/bokeh:33 4 of 8
TP bar-stacked/pygal:27 5 of 8
TP biplot-pca/pygal:32 7 of 8
TP box-notched/matplotlib:81 zip(boxes, IMPRINT, strict=True)
TP bubble-packed/altair:64 IMPRINT_PALETTE[:5]
TP column-stratigraphic/seaborn:81 zip(age_order, IMPRINT_PALETTE[:5], strict=True)
TP frequency-polygon-basic/plotly no IMPRINT list, group colors the lines
TP gauge-activity-rings/plotnine:34 3 of 8
FP horizon-basic/altair horizon bands use one diverging scheme, not a color per series
TP line-multi/letsplot:24 3 of 8
TP line-tanabe-sugano/bokeh:141 zip(IMPRINT_PALETTE, terms, strict=True)
TP lollipop-grouped/bokeh:65 zip(series_names, IMPRINT, strict=True)
TP marimekko-basic/matplotlib:65 zip(products, IMPRINT, ..., strict=True)
TP polar-scatter/letsplot:25 3 of 8
TP scatter-embedding/bokeh:33 6 of 8
TP stereonet-equal-area/altair:26 dict(zip(FEATURES, IMPRINT, strict=True))
TP streamgraph-basic/bokeh:29 COLORS 6 of 8
TP swimmer-clinical-timeline/altair:30 ARM_COLORS 2 of 8
TP timeline-basic/plotly:22 4 of 8
TP treemap-basic/matplotlib:24 5 of 8
TP treemap-basic/plotly:116 dict(zip(unique_cats_ordered, IMPRINT, strict=True))
TP violin-grouped-swarm/matplotlib:21 COLORS 2 of 8
FP violin-split/seaborn a binary split always has two colors
Rule 6 has no catalogue evidence: no file has a # Constants zone yet. Unit tests cover it, and it stays advisory.
Dry run of the perturbation harness
The dry run used the two converted examples, tests/unit/core/fixtures/adaptability/, with matplotlib 3.11.2 and seaborn 0.13.2 in the local venv. For each table it ran head and block, built the table with core.perturbation, swapped the block with smoke_source and rendered both themes. The canvas was checked with core.canvas. The bar-grouped fixture is the design's example, pivoted without fill_value=0.
Table
bar-grouped (matplotlib)
line-basic (seaborn)
Catalogue data
pass, 3200×1800 both themes
pass, 3200×1800 both themes
renamed (one column with spaces, a unit and a dot; categories renamed)
pass
pass
more_categories (5 quarters, 4 lines)
pass
(no categorical column)
fewer_categories (3 quarters, 2 lines)
pass
(no categorical column)
single_group
pass
(no colour role)
fewer_rows
pass (6 of 12 rows)
pass (6 of 60)
more_rows (10×)
pass (120 rows)
pass (600 rows)
scaled (×1,000 + 100,000)
pass
pass
Nine groups, out of palette by design
IndexError: list index out of range
The whole run took 39 s. The stale-label check on text artists belongs to the PR 2 runner and was not run.
Decisions
Open questions of the design (the owner can override any of them):
# Constants allowlist: scalars, thresholds, ordered domain scales and semantic maps may live there. Rule 6 flags a list, dict or string there that shares a value with the block's sample values, or two data numbers, and is always advisory. An ordered scale that the block also spells out cannot be told from hoisted categories statically, so the smoke run's renamed table is the real test.
Columns that are not spec roles: they get # role: extra. Any number of them is allowed, they are never checked against the roles, and the agent binds them by name or falls back to the adapter.
Scenario text that cannot be computed: an event label goes in # Constants with its key, such as a date, and is drawn only when the key lies inside df. The checker does not flag it unless it repeats a sample value, which is advisory.
df.to_string(buf=...): ban the buf keyword in the agent's validator and keep to_string(). The checker gets no rule, because no catalogue file uses it. The validator change belongs to the agent's readiness PR.
R loader: base read.csv(..., check.names = FALSE) with today's R image. readr only once the image ships it.
Persisting the smoke result: yes. impl-merge writes the merged version's smoke result forward next to the static record, with no backfill. PR 2 stages it, and PR 3 lets the mechanical path require it.
Renaming constants after substitution: no deterministic rename in phase 1. QUARTER = "Region" stays readable, and a slug rename risks colliding with other names.
Implementation choices, decided here and documented in the modules:
status is pass, fail or excluded. A map spec is never checked, so neither pass nor fail would be true.
Markers are column-0 comments, # Data or # Plot alone or followed by : or a dash and a hint. The catalogue writes # Data - ... 873 times and # Data — ... 713 times.
The role comment is # role: <role> with the role name alone, on any line of the assignment. extra is the one non-spec role.
A block "binds df" by df = ..., by a column or attribute assignment on df, or by an in-place call on it.
Rule 4's string rules compare whole strings only, so prose that contains a sample value is fine. They skip style keywords, prose words ("Total", "Other"), style words ("center", "dash"), property-like dict keys, one-character values, the title (any string with "anyplot.ai") and output calls.
Fixed domains are 0 to 1 and -1 to 1 with 20 % headroom, and 0 to 100 and 0 to 360 with 10 %. Neutral values are -1, 0 and 1. CRITICAL_VALUES now lives in core.adaptability, and readiness.py imports it, as it imports MAP_SPECS from core.spec_roles.
The setup allowlist is per library (SETUP_CALLS). There is one unit test per entry for the head and one for the block.
The shape rule runs in this order: map specs, overrides, a nodes or edges role, a role named *matrix*, text-only roles as generated, and the table as default. There are 16 overrides, each with a reason. A test asserts that every override changes its spec's shape.
The colour role is the first categorical or text role whose name is in COLOR_ROLE_NAMES or whose description names a colour encoding. It is tuned for precision, so a pie's category is not detected.
A colour role whose group count the spec fixes (boolean, or "binary", "two", "pair", "split", "both", "versus" in its description) keeps its groups in every perturbation. Only an unconstrained colour role gets the single-group table, and more_rows never touches the colour column.
Network specs get no perturbations yet. Their block may bind a nodes table whose ids must follow the perturbed edges, which belongs with network eligibility (from the Copilot review).
smoke_source takes the perturbation and renames every column constant by its old column name, # role: extra included (rewrite_columns).
The mirror is generated, not hand-copied: uv run python -m core.adaptability_mirror, with --check for CI-style use.
Findings for the owner
Statistic-text precision is far below 90 % and no static fix is in sight, at 34 %. Typed thresholds and domain facts look exactly like typed statistics. It stays advisory and is a candidate for removal if the shadow phase confirms that.
Spec-side roles: about 20 table specs list scalar parameters as required roles, so rule 2 asks for column constants that no tidy table has. Examples are sample_rate and n_mels (spectrogram), lsl, usl and target (histogram-capability), min_value and max_value (gauge-basic), auc (roc-curve), annual totals and elevation (climograph), base_value and final_value (shap-waterfall), and triple_point (phase-diagram-pt). This does not block the parser. It is input for the spec audit the design schedules first.
Generated specs pass rule 1 to 5 cheaply: 22 files pass today because the design asks only for the markers, a clean block and the palette there. A rule that the block holds parameters only would need a definition of "parameter" first.
The matrix shape is lenient by design until the misfiled matrix specs are retyped (heatmap-correlation, heatmap-clustered, learning-curve-basic, pdp-basic): rule 2 asks nothing of a matrix.
First review (Gemini 3.8 Flash, routine lane): the run hit its input budget and its answer was cut at the output limit (exit 4), so only part of it arrived. It raised one finding, which I verified and fixed in a6cf527: annotated assignments such as QUARTER: str = "Quarter" # role: category and an annotated IMPRINT were invisible to the checker, so it reported a missing constant and a missing palette. The catalogue baseline in the PR body is unchanged by the fix.
This PR adds the foundation for checking whether Python catalogue implementations can work with replacement data. It shares the agent’s role parser with the checker and introduces smoke-test helpers without enabling them in the pipeline.
Changes:
Adds static adaptability checks and a generated standalone mirror.
Adds data perturbations, source substitution, and render comparison helpers.
Moves role parsing into core, updates consumers, and adds tests and documentation.
File
Description
tests/unit/core/test_spec_roles.py
Tests shared role parsing, shapes, and colour roles.
tests/unit/core/test_render_identity.py
Tests pixel comparisons and CLI results.
tests/unit/core/test_perturbation.py
Tests table mutations and smoke-source generation.
tests/unit/core/test_adaptability.py
Covers checker rules, substitutions, and catalogue reports.
Exclude fixed colour columns from numeric category scaling
core/perturbation.py:217
Excluding a fixed colour-group column from varied does not protect it from this numeric-scaling loop. When an axis category is expanded, copied rows with integer group code 1 become group 1.1 while the original groups 0 and 1 remain. A compliant two-group plot therefore receives an invalid third group; with eight numeric groups, this can also exceed the palette limit. Pass the colour column into _more_categories and exclude it from numeric scaling, as _more_rows does. Add a regression combining an axis category with an integer-coded fixed two-group role.
- `scale *= 2` reads `scale` before rebinding it, though the target's
context is Store; rule 3 now counts that read.
- The matrix branch's renamed perturbation carries its column renames, so
a `# role: extra` constant naming a matrix column follows the new label.
From the second Copilot round on #12244.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
Implements "PR 1, the checker" of docs/concepts/adaptable-code.md with the
decisions of 2026-10-23:
- core/adaptability.py: stdlib-only checker of the six-zone convention
(rules 1-6 as RULE_CLASSES), a record {version, status, findings, block,
constants, color_role, shape} and at most one CQ-01 (code) defect line per
rule class; classes below 90 % precision on judged samples are advisory
(data-coordinate, statistic-text, count-literal, sample-literal, and
constants from the start). A `report` CLI prints the catalogue baseline.
- core/spec_roles.py: the agent's role parser moved here, plus data_bullets,
the data shape rule with SHAPE_OVERRIDES and MAP_SPECS, and the colour role;
the agent imports it from core.
- automation/scripts/adaptability_check.py: the generated single-file mirror
(core/adaptability_mirror.py builds it; a test pins it to the build).
- core/perturbation.py: the smoke run's tables and mutated source.
- core/render_identity.py: share of differing pixels between two renders.
No workflow, prompt or score changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Gemini Flash first review found that `QUARTER: str = "Quarter" # role: category`
and an annotated IMPRINT were invisible to the checker, which then reported
a missing constant and a missing palette. One helper now reads targets and
value from both assignment forms at every module-level collector.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Rule 3 tracks the names rebound on every path: branches of an `if`
intersect, a loop body may run zero times, a `try` body binds nothing
for its handlers, `match` cases bind nothing after the match, and `del`
unbinds.
- Substitution keeps a role comment that sits inside a multi-line value
(`SERIES = [ # role: y`) by moving it to the statement's last line; a
further comment after the role is not part of it.
- `rewrite_columns` renames every column constant by its old column name,
`# role: extra` included, and `smoke_source` takes the perturbation and
follows its `renames`.
- A colour role whose group count the spec fixes keeps its groups in the
category perturbations; `more_rows` leaves the colour column alone, so
numeric group codes stay groups.
- Network specs get no perturbations until network eligibility: their
block may bind a `nodes` table that the smoke source cannot carry yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
- `scale *= 2` reads `scale` before rebinding it, though the target's
context is Store; rule 3 now counts that read.
- The matrix branch's renamed perturbation carries its column renames, so
a `# role: extra` constant naming a matrix column follows the new label.
From the second Copilot round on #12244.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
…imit (#12260)
## Summary
- **Root cause of the
`TestRender::test_a_launcher_that_outlives_its_kill_blocks_renders_until_it_exits`
flake** (two CI failures on 2026-10-23, runs 39932413037 and
39940278486). The fake launcher execs the harness, so the process the
test keeps alive *is* the harness and carries the run's `RLIMIT_CPU`,
one second for a one-second job (`wire.run_cpu_seconds`). The harness
start (matplotlib import, a fresh font cache per test) costs about 0.33
CPU s locally and roughly 2.4 times that on the CI runner (the plain
render test takes 0.54 s here and 1.31 s there). Whenever it spent the
full second, the kernel's SIGKILL ended the "stuck" launcher about 50 ms
after the timeout, inside the 0.2 s grace window, so `stuck` read 0. The
failing run's `wall_s=1.054` is that death: a survivor keeps `_stop`
waiting the full grace, about 1.22 s.
- **The executor is correct; only the test raced.** A real `sandbox do`
runs on the host, outside the limit the harness sets inside the sandbox,
and `stuck` counts only launchers still alive. The test now gives the
run the host's full CPU cap and asserts that cap on the survivor with
`resource.prlimit`, so it fails every time without the fix instead of
once in a while.
- Takes ruff's PEP 758 `except` form in two lines from #12244 that
main's Lint job flags since the 3.14 move. These are the same hunks as
#12259, so whichever PR lands first, the other merges cleanly.
Ruled out: the fake binary's `delete --force` only logs and exits 0. The
sleeper writes nothing, so it gets no SIGPIPE. Asyncio cancellation of
the readers does not close the transport, and `PYTHONASYNCIODEBUG=1`
with `-X dev` reports no "close running child" or unclosed-transport
warning. Coverage does not follow subprocesses (pytest-cov 7, no `patch
= subprocess`). Python 3.14 makes the harness start only about 5 %
heavier than 3.13 (0.33 against 0.32 CPU s, same harness, same
matplotlib), so it narrowed the margin but did not cause the flake.
## Plan
N/A
## Test plan
- [x] Deterministic reproduction of the CI signature (`assert (True and
0 == 1)`, `wall_s` about 1.02, launcher exit -9): the old test body with
code that spends more than one CPU second before it sleeps.
- [x] Adjusted test with the CPU-cap line removed: 31 of 31 failures, 16
idle and 15 under 48 busy loops on 32 threads.
- [x] Adjusted test as committed: 50 of 50 passes, 25 idle and 25 under
48 busy loops.
- [x] `uv run pytest tests/unit tests/integration -n 4 --dist
worksteal`: 12,715 passed, 3 skipped.
- [x] `tests/unit/agents/renderer` under `python -X dev` with
`PYTHONASYNCIODEBUG=1`: 179 passed, no asyncio warnings.
- [x] `ruff check .`, `ruff format --check .`, `mypy api core agents`
and `tools.changelog check --base origin/main` are clean.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
## Summary
- PR 2 of the adaptable-code rollout
([design](https://github.com/MarkusNeusinger/anyplot/blob/main/docs/concepts/adaptable-code.md),
"Rollout" item 2; PR 1 was #12244): the Data block convention in the
generator and in all 15 library prompts, a Python self-check (Step 3c)
in generation and repair, a record-only smoke run and predecessor render
in `impl-generate.yml` and `impl-repair.yml`, a record-only adaptability
check in `impl-review.yml`, and the identity-based adapt route plus the
adapt regression in the regen gate.
- Score-neutral: the review prompt and the rubric are unchanged, every
new step is record-only (never a raw `exit 1`), and the review-free path
stays off behind the repository variable `ADAPT_REVIEW_FREE`.
- Checks are Python only (the 8 Python libraries). R, Julia and
JavaScript get the convention in their first-generation prompts and keep
today's regeneration mindset; every record carries `language`, so their
checkers plug in later without format changes (the "all 15 libraries"
goal).
## Plan
Design: `docs/concepts/adaptable-code.md` ("The convention", "Other
libraries", "The checks", "Pipeline changes", "The adapt route",
"Rollout" item 2). Four parts were built against one interface contract
and merged here with `--no-ff`; the merge commits keep each part's
history.
### A, prompts (`eb3d074fa`)
- New `prompts/adaptable-code.md` holds the convention once: the six
zones, the `# Data`/`# Plot` markers and `# role:` grammar, the
plot-code rules, the data shapes, the bar-grouped example, and the R,
Julia and JavaScript forms. `plot-generator.md` and Step 1 of
`impl-generate-claude.md` reference it.
- `plot-generator.md`: the template, Structure and Data Generation
Strategy follow the six zones; Data storytelling gets rule-chosen
callout targets; `bbox_inches='tight'` is gone.
`default-style-guide.md`: "list all eight hues and use the first n", no
`bbox_inches='tight'`.
- All 15 library prompts get a "Data block" section; the `[:N]` palette
slices are gone from matplotlib and seaborn; Chart.js no longer recycles
colours with `i % length`.
- `impl-generate-claude.md`: Step 3c (`python -m core.smoke_run
selfcheck`, at most three attempts, never blocking), the commit line
`Adaptability self-check: pass|fail (...), attempt N`, the five
regen-mindset amendments phrased "For Python (`LANGUAGE=python`)", a
"Converting a Python file" bullet that aims at an identical render.
`impl-repair-claude.md`: self-check lines are real defects; Step 5b
reruns the self-check.
### B, generation side (`416065feb`, `3425ac00b`)
- `core/smoke_run.py` (`run`, `pair`, `selfcheck`): one process per
perturbed table and theme, process groups killed on timeout, errors
mapped back to the original line, a stale-label hook for matplotlib and
seaborn, and `pair`, which renders the new source and then the
predecessor in one environment and compares the raw PNGs.
- `core/perturbation.py`: `missing_number` (NaN in every `float64`
column, binding) and `missing_nullable` (`<NA>` in `Int64`, `boolean`
and `string` columns, advisory).
- `impl-generate.yml` "Adaptability smoke run (record-only)" and the
private `adapt/` staging in "Upload to GCS Staging"; `impl-repair.yml`
reruns the smoke run and replaces or removes `adapt/new-smoke.json`.
### C, review side (`4b710e2c0`)
- `impl-review.yml` "Adaptability check (record-only)": the workflow-ref
checker mirror on the PR file and, on a regeneration, its predecessor;
`automation/scripts/adapt_record.py` builds
`/tmp/anyplot-adapt-{new,prev}.json`; `regen_gate.py adapt-route` before
the review; the review is skipped only when `ADAPT_REVIEW_FREE` is
`true` and every condition holds.
- Review-free wiring, the `regen:adapt` count label, carried metadata
(`review.carried`), and the staging download narrowed to `plot-*` so
`adapt/` never reaches the blind reviewer. `impl-merge.yml` promotes
only `plot-*`. `regen_writeback.py` removes `review.carried` on a
write-back.
### D, gate (`0438a60a4`)
- `adapt_assess` with the contract's 20 ordered conditions;
`adapt_identity` replaces the keeps `no_visible_improvement`,
`no_defect_improvement` and `below_tolerance`; `--review-skipped` merges
a counted route before the score checks; `adapt_regression` turns any
merge into keep. The record gains an `adapt` block (`RECORD_VERSION`
stays 1), the outputs and notice gain `adapt_counted` and `adapt_why`.
The gate stays stdlib-only and reads identity from impl-generate's
`pair.json`.
### Integration (`db865896b`)
- `impl-review.yml` gets one unconditional "Set up Python" step
(`python-version-file: .python-version`) before the first regen-tools
call; part C's mirror-only step is gone and every tool runs on plain
`python3`. Since #12247 the gate, `review_provenance.py` and the mirror
use PEP 758 `except A, B:`, and the runner image's Python is not pinned.
- The Step 3c log line gains `smoke_why`, so the adapt record counts a
skipped first attempt as a pass only for a shape skip, as `selfcheck`'s
own exit 0 does.
- Docs: the pipeline diagram and a table of the adaptability steps
(`docs/workflows/overview.md`), `adapt_record.py`
(`automation/scripts/README.md`), `core/smoke_run.py`
(`docs/reference/repository.md`).
### Review fixes (`05ec4a82b`, `602dd40c5`, `4fd7f7c57`)
- Copilot: an adapt record without valid `static` and `smoke` blocks is
`record_invalid` (fail closed); the Chart.js and ECharts Data block
examples run as written; the "ninth group fails loudly" rule names the
idioms it holds for (lookup by position or group) in
`prompts/adaptable-code.md` and `prompts/default-style-guide.md`.
- Gemini and part D's own note: the route needs a `pair.json` rendered
from the very sources the records checked (`code_sha256` on both sides),
else the new reason token `pair_stale`.
### Second opinion (Gemini 4 Argon + Fable, `1e75be40a`..`a423e7b69`)
- The gate rejects a record whose `status` does not follow from its
`static` and `smoke` blocks (`record_invalid`), and `pair.json` must
name its `library`.
- A predecessor whose static check passed against a new source whose
static check fails is an adapt regression even without a fresh
predecessor smoke run.
- `decide --adapt-route-held`, the record field
`adapt.route_pre_review`, the output `adapt_overruled` and the label
`regen:adapt-overruled` make the review's overrules of the route
countable (O1).
- The post-score re-dispatch also covers the review-free path, with the
stored score.
- `adapt/` is staged with `gsutil rsync -r` so `new/` and `prev/` keep
their names.
- `execution_env` drops `GITHUB_*`, `ACTIONS_*`, `RUNNER_*` and `GH_*`
before running an implementation.
- An empty perturbation set has its own token
`no_perturbations:<shape>`, read the same way by `selfcheck` and the
record.
- One palette name, `IMPRINT_PALETTE`, in the convention and its
example; the `IMPRINT` alias is stated once.
- Known gap, unchanged: the early verdict label lands before "Update
metadata and implementation header"; a metadata push failure after it is
left to the watchdog (pre-existing on the normal path).
## Test plan
- [x] `uv run ruff check .` and `uv run ruff format --check .`: clean.
- [x] `uv run --extra typecheck --extra agents mypy api core agents`: no
issues in 151 source files.
- [x] `uv run pytest tests/unit tests/integration -n 4`: 13158 passed, 3
skipped (seaborn is not installed locally). That is 443 more tests than
`main`: `test_smoke_run.py` 34, `test_adapt_generate_steps.py` 36,
`test_adapt_review_steps.py` 85, `test_adapt_record.py` 66, plus
additions to `test_regen_gate.py`, `test_perturbation.py`,
`test_regen_writeback.py` and `test_prompts.py`.
- [x] `uv run python -m tools.changelog check --base origin/main`:
passes.
- [x] actionlint 1.7.12 with shellcheck 0.11.0 on the four changed
workflows: 87 findings on `main` and on this branch, the same set (only
line offsets in impl-generate's upload step moved).
- [x] End to end on one real pair outside the repository: bar-grouped
matplotlib, a predecessor with a three-hue palette (static rule 5 fails)
against the converted fixture. Part B's `run`, `pair` and `selfcheck`
fed part C's mirror and `adapt_record.py`, which fed part D's
`adapt-route` and `decide --review-skipped`: `route=true`, merge
`adapt_identity`, identity 0.0000 in both themes, a valid gate record.
The new side's smoke run passed all 7 binding tables; `missing_nullable`
failed as advisory (`float() argument ... 'NAType'` in `ax.bar`).
- [ ] After the merge: the first Python regeneration's `adapt_smoke`,
`adapt_check`, `adapt_route`, `adapt_review` and `regen_gate ...
adapt_*` notices, and the `adapt/` folder of its `regen-pair` artifact.
## Known gaps
Workflow changes have no branch run: claude-code-action refuses to run
on a branch whose PR changes its own workflows, so the new steps are
first exercised on `main`. Locally they are covered by unit tests that
run the exact step scripts against throwaway repositories with fake
`gsutil` and `gh`, under `bash -eo pipefail`.
- **First real runs.** The MUI X pilot of 8 forced regenerations is the
first real run of the new prompts, and the lead watches it. MUI X is
JavaScript and `regen:forced`, so it exercises the convention text, not
the Python steps; the first Python regeneration after the merge is the
first run of the smoke run, the record-only check and the adapt route.
The daily regeneration stays paused until this merges ("Rollout" step
3).
- **Not verified until a real run:** `gsutil rm -r` on a missing prefix,
`rsync -r` of `adapt/` to staging, `cp -r` back in impl-review, and
whether bucket-level IAM keeps `adapt/` private; impl-merge's
non-recursive `cp "${STAGING}/plot-*"`; the runtime of the Chrome-driven
libraries (bokeh, plotly) inside the 20-minute step timeout (estimate 1
to 3 minutes); the seaborn, plotly, altair, plotnine, bokeh, pygal and
letsplot paths against the runner (matplotlib ran locally);
`setup-python` with `.python-version` in impl-review;
`vars.ADAPT_REVIEW_FREE` resolution; the review-free path end to end.
- **Runner exposure.** The smoke executions run model-written code again
after "Create library metadata file" has the workflow token in
`.git/config`, the same exposure as the generator's own Step 3 renders.
The runner is not an isolation boundary; the protections are the
workflow-ref copy of `core/`, the new-then-predecessor render order, and
the owner-controlled switch.
- **Missing-value coverage.** The loader cast turns integer columns into
`Int64`, so an integer-valued sample (the design's own example) gets no
binding NaN table, only the advisory `<NA>` one.
## Open decisions for the owner
- **O1, review-free switch.** It stays off. The evidence is the
route-held PRs. `regen:adapt` marks a merge on which the adapt route
held before and through the review. `regen:adapt-overruled` marks a PR
whose route held before the review and whose review floor then turned it
off, on a merge or a keep; its gate record has `adapt.route_pre_review:
true`. Turn the switch on (`gh variable set ADAPT_REVIEW_FREE --body
true`) only when the overruled share stays near zero over at least 50
route-held PRs:
```
gh pr list --state merged --label regen:adapt --limit 1000 --json number
--jq length
gh pr list --state all --label regen:adapt-overruled --limit 1000 --json
number --jq length
```
The alternative is an automatic flip once 50 route-held PRs show no
overruled one.
- **O2, nullable integers.** `missing_nullable` stays advisory: in part
A's idiom run it failed 5 of the 8 Python libraries, and it fails the
design's reference example. Either the convention allows
`.astype(float)` on drawn nullable-integer columns, or the loader
changes.
- **Review-retest arm.** This PR edits `prompts/default-style-guide.md`
(two code-only sentences) and adds a "Data block" section to every
library prompt, and the reviewer reads both. `CLAUDE.md` asks for a
candidate `review-retest.yml` arm on such edits; the design says none is
needed, because no criterion, rubric line or review-prompt line changes.
No arm was dispatched; dispatch one if you want the measurement before
merging.
- **Local helpers.** `/regen` and `/update` list
`prompts/adaptable-code.md` but have no Step 3c, so a local regeneration
is not self-checked.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
core/adaptability.py, the stdlib-only checker of the adaptable-code convention, implements rules 1 to 6 with the amendments of 2026-10-23. It returns a record{version, status, findings, block, constants, color_role, shape}and writes at most oneCQ-01 (code)defect line per rule class, with advisory lines marked(advisory).python -m core.adaptability report plots/ [--sample N]prints the baseline below.core/spec_roles.pyis the agent's role parser, moved, not copied. The agent and its tests import it fromcore. It addsdata_bullets(pinned tosync_to_postgres.pyon every spec), the shape rule withSHAPE_OVERRIDESandMAP_SPECS, and the colour-driving role.automation/scripts/adaptability_check.pyis the single-file mirror for the workflow runner.core/adaptability_mirror.pygenerates it, and a test pins the file to the build, keeps it stdlib-only, runs it withpython -Iwherecorecannot be imported, and compares its records with the module's on a catalogue sample.core/perturbation.pybuilds the smoke run's tables and the mutated source. It has no nullable-dtype perturbation.core/render_identity.pycomputes the share of differing pixels against a bound, with a small CLI.Plan
docs/concepts/adaptable-code.md, "Rollout" item 1, with the decisions of 2026-10-23 from the brief. Only the status line of that file changes here.Catalogue baseline
uv run python -m core.adaptability report plots/onmainat f2b4629:Files with a finding, per rule class:
Specs per shape: 278 table, 19 generated, 9 network, 6 matrix, 13 excluded. That matches the panel's count of 278 tables. All 22 passing files belong to generated specs, where the design asks only for the markers, a clean block and the palette (see "Findings").
Rule 1 fails mostly on missing markers, as the table below shows. Altair is the extreme case: only 43 of its 325 files have a
# Plotmarker.# Datamarker with no# Plotafter itdf# Datamarker at allPrecision
A finding is judged true when the literal or structure ties the code to the catalogue's own data, so that other data would break it or show stale text. Each sample draws random findings, at most one per file, from the catalogue with a fixed seed (
--sample N --seed S). The per-class verdicts use the samples taken after the last change to each class.# ConstantszoneThe verdict is per class, not per rule. Rule 4 as a whole judges 22 of 30 on the stratified sample below, but two of its classes clear 90 % and stay binding. These fixes came from earlier samples:
df[RSI] >= 70orif w > 7holds for any data.zipover a palette sliced tolen(groups)is allowed.entityis no longer a colour role, since slope charts colour by direction.range=inalt.Scaleandscale_sizeis visual. PIL'sdraw.text((x, y))is pixels. The class still stays at 63 %: annotation positions in layout units and domain constants (pH 0 to 14, a rebased-100 line, 45° phase ticks) cannot be separated statically.Rule 1, layout: 30 lines, seed 7 (30 TP)
Rule 2, columns: 30 lines, seed 4 (30 TP; no catalogue file has role constants yet)
Rule 3, interface: 30 lines, seed 7 (30 TP)
Rule 4, literals below the block: 30 lines, seed 7, five per class (22 TP)
Rule 5, palette: 30 lines, seed 5 (28 TP)
Rule 6 has no catalogue evidence: no file has a
# Constantszone yet. Unit tests cover it, and it stays advisory.Dry run of the perturbation harness
The dry run used the two converted examples,
tests/unit/core/fixtures/adaptability/, with matplotlib 3.11.2 and seaborn 0.13.2 in the local venv. For each table it ran head and block, built the table withcore.perturbation, swapped the block withsmoke_sourceand rendered both themes. The canvas was checked withcore.canvas. The bar-grouped fixture is the design's example, pivoted withoutfill_value=0.IndexError: list index out of rangeThe whole run took 39 s. The stale-label check on text artists belongs to the PR 2 runner and was not run.
Decisions
Open questions of the design (the owner can override any of them):
# Constantsallowlist: scalars, thresholds, ordered domain scales and semantic maps may live there. Rule 6 flags a list, dict or string there that shares a value with the block's sample values, or two data numbers, and is always advisory. An ordered scale that the block also spells out cannot be told from hoisted categories statically, so the smoke run's renamed table is the real test.# role: extra. Any number of them is allowed, they are never checked against the roles, and the agent binds them by name or falls back to the adapter.# Constantswith its key, such as a date, and is drawn only when the key lies insidedf. The checker does not flag it unless it repeats a sample value, which is advisory.df.to_string(buf=...): ban thebufkeyword in the agent's validator and keepto_string(). The checker gets no rule, because no catalogue file uses it. The validator change belongs to the agent's readiness PR.read.csv(..., check.names = FALSE)with today's R image.readronly once the image ships it.QUARTER = "Region"stays readable, and a slug rename risks colliding with other names.Implementation choices, decided here and documented in the modules:
statusispass,failorexcluded. A map spec is never checked, so neither pass nor fail would be true.# Dataor# Plotalone or followed by:or a dash and a hint. The catalogue writes# Data - ...873 times and# Data — ...713 times.# role: <role>with the role name alone, on any line of the assignment.extrais the one non-spec role.df = ..., by a column or attribute assignment ondf, or by an in-place call on it.CRITICAL_VALUESnow lives incore.adaptability, andreadiness.pyimports it, as it importsMAP_SPECSfromcore.spec_roles.SETUP_CALLS). There is one unit test per entry for the head and one for the block.nodesoredgesrole, a role named*matrix*, text-only roles as generated, and the table as default. There are 16 overrides, each with a reason. A test asserts that every override changes its spec's shape.COLOR_ROLE_NAMESor whose description names a colour encoding. It is tuned for precision, so a pie'scategoryis not detected.more_rowsnever touches the colour column.nodestable whose ids must follow the perturbed edges, which belongs with network eligibility (from the Copilot review).smoke_sourcetakes the perturbation and renames every column constant by its old column name,# role: extraincluded (rewrite_columns).uv run python -m core.adaptability_mirror, with--checkfor CI-style use.Findings for the owner
sample_rateandn_mels(spectrogram),lsl,uslandtarget(histogram-capability),min_valueandmax_value(gauge-basic),auc(roc-curve), annual totals andelevation(climograph),base_valueandfinal_value(shap-waterfall), andtriple_point(phase-diagram-pt). This does not block the parser. It is input for the spec audit the design schedules first.Test plan
uv run pytest tests/unit tests/integration -n 4 --dist worksteal: 12,611 passed.uv run ruff check .anduv run ruff format --check ..uv run --extra typecheck mypy api core.uv run python -m tools.changelog check --base origin/main.uv run python -m core.adaptability_mirror --check. The mirror test also runs the file withpython -Ifrom a directory withoutcore.uv run python -m core.adaptability report plots/for the baseline and the precision samples above.tests/unit/agents/data,runtime,evals,code/test_readiness.py) pass.