Skip to content

ci: decide auto-merge when fetch-metadata fails, and leave it alone in a run for an older head - #65

Merged
semnil merged 2 commits into
masterfrom
ci/auto-merge-failed-metadata-stale-run
Oct 25, 2026
Merged

semnil merged 2 commits into
masterfrom
ci/auto-merge-failed-metadata-stale-run

Conversation

@semnil

@semnil semnil commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Summary

Fixes two ways .github/workflows/dependabot-auto-merge.yml fails. A review of semnil/koe-zukan#62, which adds the same workflow, found them; the same fix goes to every repository of semnil that has this workflow.

  • When fetch-metadata failed, the deciding step was skipped, so the required auto-merge failed and the pull request could not be merged by hand either. fetch-metadata fails when the first commit of a Dependabot pull request is no longer a verified Dependabot commit, for example after someone else rebases it; this happened on build(deps): Bump pyee from 14.0.0 to 15.0.0 analyze-spectrum#100 (run 39769440764, 2026-10-22). Now the metadata step continues on error and the deciding step reads its outcome: on a failure it warns, keeps auto-merge off (turning off one an earlier head enabled) and passes.
  • A run for an older head that ran after the run for the newer head turned off the auto-merge the newer run had enabled. Now the deciding step reads the head of the pull request with the auto-merge state and changes nothing when the head is no longer the one its event named, and the runs for one pull request share the concurrency group dependabot-auto-merge-<number>. In the auto-merge runs on Dependabot branches of the 13 repositories with this workflow (170 runs, 139 pairs of an older and a newer head, read from the Actions API on 2026-10-25), no older run was still running when the newer one started.

Changes

  • .github/workflows/dependabot-auto-merge.yml: continue-on-error on the metadata step and its outcome passed to the deciding step as METADATA_OUTCOME (e89f437); the concurrency group and the head comparison before turning auto-merge off (74a10b4).
  • .github/tests/dependabot-auto-merge-test.sh: checks the step settings and the concurrency group, answers the head from the fake gh, and adds the cases of a failed metadata step (e89f437) and of a moved or unreadable head or a failing read (74a10b4).

Verification

At 74a10b4:

  • .github/tests/dependabot-auto-merge-test.sh: 51 checks, 0 failed. Against the workflow of aeaaf23: 51 checks, 35 failed.
  • actionlint 1.7.12 on the changed workflow: no findings (on the whole directory it stops at lines of other workflows this pull request does not change). shellcheck on the test: no findings.
  • .github/tests/workflow-checks-test.sh: 34 cases, 0 failed, 0 skipped.
  • Both files are the same blobs as on ci: run the tests on every pull request and auto-merge Dependabot patch and minor updates koe-zukan#62 (61e81b2, bdb728b), where each of 8 mutations of the workflow makes the test fail.

Measured on GitHub

On a private probe repository, semnil/auto-merge-probe (pull request 1, 2026-10-25), the job of .github/workflows/dependabot-auto-merge.yml as semnil/koe-zukan#62 merged it ran with two changes: its steps ran for a pull request semnil opened, so fetch-metadata failed on it, and a 45-second wait sat between the metadata step and the deciding step. Heads A, B and C were pushed at 01:22:17Z, 01:22:31Z and 01:22:43Z.

  • Every run of the job that ran: fetch-metadata failed (PR is not from Dependabot, nothing to do.), its step concluded success, the deciding step ran with METADATA_OUTCOME: failure and warned fetch-metadata did not classify this pull request (failure), so auto-merge stays off, and the job concluded success.
  • The run for A decided after the head had moved to C: it logged the head of the pull request is now 27435858646bfc5628cf11daecb9c745c090f1e0, so the run for that head decides and called no gh pr merge.
  • The run for B waited and was cancelled when the run for C arrived (Canceling since a higher priority waiting request for dependabot-auto-merge-1 exists); the run for C started at 01:23:20Z, after the run for A ended at 01:23:18Z. A control workflow with the same wait and no concurrency group ran its three runs overlapping (01:22:27–01:23:16Z, 01:22:38–01:23:26Z, 01:22:49–01:23:37Z).
  • Not run there: turning off an auto-merge that is on after a failed metadata step. A private repository on GitHub Free takes no auto-merge (allow_auto_merge=true set on the probe reads back false). The gh pr merge --disable-auto call itself has run on GitHub: build(deps): Bump pyee from 14.0.0 to 15.0.0 analyze-spectrum#100 shows auto_merge_disabled by github-actions[bot] at 2026-10-22T11:12:20Z, and the branch that reaches it after a failed metadata step is a case of the test above.

Not measured yet

  • None.

fetch-metadata fails on a Dependabot pull request whose first commit is
no longer a verified Dependabot commit, for example after someone else
rebases it. The deciding step was then skipped, so `auto-merge`, a
required check, failed and the pull request could not be merged by hand
either. The metadata step now continues on error and the deciding step
reads its outcome: on a failure it warns, keeps auto-merge off (turning
off one an earlier head enabled) and passes.

dependabot-auto-merge-test.sh checks the two step settings and three
cases of a failed metadata step.
A run for an older head that ran after the run for the newer head read
the commits of the newer head, found them ineligible for its own head
and turned off the auto-merge the newer run had enabled. The deciding
step now reads the head of the pull request together with the
auto-merge state and changes nothing when the head is no longer the one
its event named, and the runs for one pull request share a concurrency
group, so they run one at a time.

dependabot-auto-merge-test.sh checks the concurrency group and five
cases of a moved or unreadable head or a failing read.
@semnil
semnil merged commit aff14c8 into master Oct 25, 2026
9 checks passed
@semnil
semnil deleted the ci/auto-merge-failed-metadata-stale-run branch October 25, 2026 01:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant