Count progress as reaching the top tier, not as beating a score margin - #92
Merged
Conversation
rasros
force-pushed
the
run-observability
branch
from
August 17, 2026 15:45
af4910a to
daac4aa
Compare
rasros
force-pushed
the
staleness-top-tier
branch
from
August 17, 2026 15:45
9e991f1 to
8cfd257
Compare
rasros
force-pushed
the
run-observability
branch
from
August 17, 2026 15:47
daac4aa to
12316a4
Compare
rasros
force-pushed
the
staleness-top-tier
branch
from
August 17, 2026 15:50
8cfd257 to
0b14bf3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Patience counted "tasks since the best score improved", where the best score was the weighted blend. Now that ranking is a dominance relation over the objective vector, progress can be defined the same way:
Read off the dominance relation, so it needs no blended score, no magnitude and no threshold on one. The same setting means the same thing on any image and survives a change to the objectives or their number. The check moves from per-result to generation close, because a candidate cannot be known to have reached the top tier before it has been ranked against one.
--epoch-min-deltadeletedIt set the margin a score had to beat to count as progress. With no magnitude compared anywhere, it has nothing left to mean — so it goes entirely rather than staying as a flag that reads as configurable and is not.
Why this and not a pool statistic
Four candidates were measured against the improvement rate that actually followed them, on a run whose end was decided by the wall clock rather than by any criterion, so the improvement curve is uncensored. Partial correlations, time trend removed from both sides:
None of them predicts whether improvement is still coming; spread's apparent signal was entirely the shared time trend. Improvements arrive from lucky mutations, not from pool structure, so the pool's internal state cannot forecast them — the only observable that tracks the arrival of improvements is the arrival of improvements, which is what patience measures.
Not established
That the top-tier definition is better than the score-margin one. It is the coherent replacement given rank-based selection, and it measures the same event; whether it ends epochs at better moments wants a run. It also changes when epochs end, which is worth knowing before merging.