P1 — the manual-review label is flapping in production, ~every 90 seconds
#10155, live on orb-v3.7.0-beta.7:
10:08:31 labeled manual-review by loopover-orb[bot]
10:09:31 unlabeled manual-review 10:09:33 labeled manual-review
10:11:18 unlabeled manual-review 10:11:20 labeled manual-review
10:13:01 unlabeled manual-review 10:13:03 labeled manual-review
10:14:43 unlabeled manual-review 10:14:44 labeled manual-review
Four add/remove cycles in eight minutes on one PR, each pair a GitHub write and a subscriber notification.
The audit trail shows both operations in the same pass, two seconds apart:
10:16:26 agent.action.label | close-audit holdout drew this PR — would-close held for human adjudication (#8831)
10:16:24 agent.action.label | manual-review hold resolved — clearing the "manual-review" label the bot applied
Root cause: an add and a remove of the same label in one plan
Three post-plan transforms surface the manual-review hold:
All three carried the identical idempotency check:
const alreadyNeedsReview = labels.manualReview !== null && next.some(
(a) => a.actionClass === "label" && a.label === labels.manualReview && a.labelOp !== "remove");
It looks for an existing add, so by construction it cannot see a planned remove. When the planner has already scheduled a release — which section 1b does whenever nothing it knows about still wants a hold — the transform appends an add next to that remove. The executor performs both. Next pass, same thing.
Why now
Latent until #10116. noManualReviewHoldWanted used to include !mergeableStateUnstable, which suppressed the release on exactly these PRs and accidentally masked the missing case. Removing that term (correctly — it was the latch keeping #10098 stuck) made the release fire, and the contradiction surfaced.
So #10116 did not create this; it revealed it. The label was previously a sticky latch, which is quieter but strictly more broken.
Why the planner cannot fix it from its side
Section 1b's doc calls noManualReviewHoldWanted "the complete condition: every reason that would ADD this label, in one place". It is not, and cannot be — these three run after planning, so their reasons are not knowable there. #8831 in particular landed long after that comment was written. Any fix that extends the planner's list is one more thing to remember on the next transform.
Fix
One shared withManualReviewHoldLabel(planned, labelSettings, action) that drops a planned release of the same label before adding, used by all three. The contradiction becomes unrepresentable at the point the add happens, rather than being a list someone must maintain.
The remove is dropped rather than the add skipped: a transform only reaches there because it has just diverted a merge or a close, so its hold is strictly newer information than the release the planner decided before that diversion.
Blast radius
gate.closeAuditHoldoutPct is 20 (the maximum) in the global Orb config, so ~1 in 5 would-close PRs is eligible for the holdout path that triggers this.
P1 — the manual-review label is flapping in production, ~every 90 seconds
#10155, live on
orb-v3.7.0-beta.7:Four add/remove cycles in eight minutes on one PR, each pair a GitHub write and a subscriber notification.
The audit trail shows both operations in the same pass, two seconds apart:
Root cause: an add and a remove of the same label in one plan
Three post-plan transforms surface the manual-review hold:
downgradeMergeToHold— merge circuit-breakerdowngradeCloseToHold— close circuit-breakerapplyCloseAuditHoldout— the randomized close-audit holdout (calibration: randomized ε-holdout on would-close PRs with propensity logging #8831)All three carried the identical idempotency check:
It looks for an existing add, so by construction it cannot see a planned remove. When the planner has already scheduled a release — which section 1b does whenever nothing it knows about still wants a hold — the transform appends an add next to that remove. The executor performs both. Next pass, same thing.
Why now
Latent until #10116.
noManualReviewHoldWantedused to include!mergeableStateUnstable, which suppressed the release on exactly these PRs and accidentally masked the missing case. Removing that term (correctly — it was the latch keeping #10098 stuck) made the release fire, and the contradiction surfaced.So #10116 did not create this; it revealed it. The label was previously a sticky latch, which is quieter but strictly more broken.
Why the planner cannot fix it from its side
Section 1b's doc calls
noManualReviewHoldWanted"the complete condition: every reason that would ADD this label, in one place". It is not, and cannot be — these three run after planning, so their reasons are not knowable there. #8831 in particular landed long after that comment was written. Any fix that extends the planner's list is one more thing to remember on the next transform.Fix
One shared
withManualReviewHoldLabel(planned, labelSettings, action)that drops a planned release of the same label before adding, used by all three. The contradiction becomes unrepresentable at the point the add happens, rather than being a list someone must maintain.The remove is dropped rather than the add skipped: a transform only reaches there because it has just diverted a merge or a close, so its hold is strictly newer information than the release the planner decided before that diversion.
Blast radius
gate.closeAuditHoldoutPctis 20 (the maximum) in the global Orb config, so ~1 in 5 would-close PRs is eligible for the holdout path that triggers this.