Stop the search rewarding candidates that change nothing

[rasros]

Seven commits, one thread: the search was spending most of itself on candidates that changed nothing, and rewarding itself for it. Diagnosed from a run whose output had a blue dot, a zigzag tail and a canvas that had wandered off its own viewBox.

The core defect

58% of candidates measured identically to their parent on all four objectives — 14,296 of 24,767 in one run — while none were byte-identical, so the worker's comparison (#85) caught none of them. Identical objectives cannot be ranked against the parent, so such a candidate is admitted wherever the parent sits and reports back as a survivor. reordered elements, the operator that produces them most reliably, had taken 74% of the policy's weight.

Rejecting on measures rather than bytes, and charging the operator for the draw: 0.0% in the verification run (4 of 14,057), reordered elements back to the floor, and the operator mix inverted toward operators that change something.

Three ways a mutation could only do harm

Colours were invented. mutate_stroke assigned random.choice(_NAMED_SVG_COLORS) — 18 CSS names. On a drawing in blacks and greys, fifteen of eighteen can only be wrong; that is the stray blue dot and the brown ring round the eye. Now colour is fudged, never chosen: mutate_color moves a hex value by at most ±8 per channel (was ±60, which random-walks a grey into brown), skips anything that is not hex, and mutate_stroke no longer sets colours at all — it adds an outline in the shape's own fill, or removes one.

Moves were written as transforms. A transform is a second description of where a thing is, laid over the first: it accumulated — 23 stacked translates on one background rect, which is how the canvas drifted — and it left coordinates saying one thing while the drawing did another, so every later mutation read a position that was not where the element appeared. mutate_translate now edits the element's own numbers: position attributes, points pairs, and absolute path coordinates, with an arc's radii, rotation and flags left alone.

Some markup could not be edited at all. Which forms a model reaches for is a property of the model: one model's seeds carried 147 elements in relative path commands, which describe an offset from wherever the pen is and so cannot be moved. normalize_svg now runs on everything the model returns — relative paths become absolute, polygon/polyline become path, presentation properties move out of style, named colours become hex, short hex becomes long. Verified pixel-identical across 20 real seeds from two models, and idempotent.

The epoch criterion this started as

--epoch-eval-patience armed at 5, counted in evaluator checks. The unit matters: a generation is 100 accepted candidates, so a threshold in generations moves with the acceptance rate and with --pool-size, and one below a single check interval fires before a check can intervene at all. --seeds is 5 and fixed, rather than pool_size // 10, which tied the LLM budget to a number chosen for unrelated reasons.

Scripts

Checking them found four breakages. plot_run.py carried @functools.cache on a plotting function — unhashable arguments, so it raised on every invocation, and had done since commit e51beb1. clean_runs.py kept zero nodes and deleted every drawing, because bare-id filenames parse as score = inf and the next line treats inf as invalid. bench_search.py read a column that no longer exists.

The fourth was in src: save_node wrote 0.0 for any measure a node lacked. front_score exists only on the nodes the evaluator saw, so every other node recorded a perfect evaluator score — metrics.py warns about this hazard for objectives and the CSV writer did it anyway. Now blank. That also showed front_score never reaches lineage.csv at all, since a row is written when a candidate is admitted, before the evaluator has seen anything; the bench reads the evaluator curve from stats.csv instead.

Measured

	before	after
candidates measuring as parent	58%	0.0%
reordered elements weight	74%	3% (floor)
stacked translates in output	23	0
relative-path elements in seeds	147	0
node files / run size	21,456 / 346 MB	646 / 12 MB
evaluator score	0.377012	0.327818

The final run also produced the tail feathers for the first time, and the eye highlight inside the disc rather than bitten out of the rim.

Not established

The last run changed the model as well (gpt-5.6-terra), so its score is not attributable to these fixes alone. The colour and normalization commits landed after it, and are unrun. And 35 of its 50 LLM calls bought no measurable gain, so --epochs 10 is mostly wasted spend — worth its own look.