CI: a tag release ends with a GitHub Release built from the version's milestone

.github/ISSUE_TEMPLATE/release.md

@@ -22,7 +22,11 @@ Every step says who does it: script (run the command, read the result), CI (a wo
   ```
 
 - [ ] 2. owner: for a final or post release, the native readiness bar is met and the repository variable `ARCADEDB_NATIVE_READINESS_BAR_MET` is `true` (the preflight line `(g)` shows it, or SKIPs when the login cannot read variables).
-- [ ] 3. owner: the go. Nothing is published before this box is ticked.
+- [ ] 3. owner: the go. Nothing is published before this box is ticked. Before it, preview the GitHub Release notes and check that every pull request of this release has the milestone `<VERSION>` (a pull request without it is not listed in a `devN` or final release; a `postN` lists the commit range instead and needs no milestone):
+
+  ```bash
+  uv run --no-project python -I scripts/release_notes.py <VERSION> --dry-run
+  ```
 
 ## The tag (the only publishing step)
 
@@ -32,7 +36,7 @@ Every step says who does it: script (run the command, read the result), CI (a wo
   git fetch origin main --no-tags && git tag <VERSION> origin/main && git push origin <VERSION>
   ```
 
-- [ ] 5. CI: both release workflows finish and publish. Note the two run ids (the native build takes the longest). To retry a failed upload, re-run only the publish job, never the whole workflow.
+- [ ] 5. CI: both release workflows finish and publish, then `release-notes.yml` creates the GitHub Release and, for a final release, closes the milestone. Note the two run ids (the native build takes the longest). To retry a failed upload, re-run only the publish job, never the whole workflow.
 
   ```bash
   gh run list -R ArcadeData/arcadedb-python --branch <VERSION> --limit 6
@@ -69,4 +73,4 @@ Every step says who does it: script (run the command, read the result), CI (a wo
   If the engine NOTICE changed, also refresh the kept copy with `git -C <arcadedb clone> show <VERSION>:NOTICE > native/wheel/ARCADEDB_ENGINE_NOTICE`.
 
 - [ ] 9. CI, then owner: merge the bump PR when every check is green, and delete the branch and the worktree. Then remove the tests and skip-list lines that waited for the new engine: `git grep -n -i "native build links\|native image links"`.
-- [ ] 10. owner: close this issue with the two run ids and the two PyPI links.
+- [ ] 10. owner: check the GitHub Release (notes, pre-release flag) and, after a final release, that the milestone is closed and what was still open in it moved to the next milestone (the open one due next, or one the workflow created named after the next upstream engine version). If the run failed with a message about the next milestone, create it with a due date and re-run `GitHub Release` (Actions, `workflow_dispatch`, the tag). Close this issue with the two run ids and the two PyPI links.

.github/workflows/release-notes.yml

@@ -0,0 +1,173 @@
+name: GitHub Release
+
+# A tag release ends here. When BOTH release workflows of a tag (release-native.yml for `arcadedb`, release-jvm.yml for `arcadedb-jvm`)
+# have finished successfully, this workflow creates the GitHub Release of the tag with notes written by scripts/release_notes.py from the
+# milestone named after the base version (a `postN` uses the commit range since the previous tag instead), and closes that milestone after
+# a FINAL release (A.B.C, not A.B.C.devN, not A.B.C.postN), after moving what is still open in it to the next milestone.
+#
+# Why `workflow_run` and not a last job in each release workflow, or a poll of PyPI: the two release workflows do not wait for each other
+# (the native build and the JVM build take different times and either can need a retry), and a job inside one of them cannot know the
+# other finished. `workflow_run` fires when each one completes, including after a re-run of only a publish job. Each firing looks at both
+# workflows for the same tag commit, and does nothing unless both concluded `success` (the first one to finish therefore exits quietly,
+# the second one creates the release). A workflow only concludes `success` after its publish job uploaded to PyPI and, for the JVM
+# package, after the install from PyPI on every platform. As a last check the gate asks PyPI for both versions. If both firings run at
+# once, the release is created by one and edited by the other (the concurrency group below also serialises them), so the result is the same.
+#
+# A failed or half done release (for example a final tag whose native readiness bar is not met) creates nothing. When the missing part is
+# fixed and its publish job re-run, this workflow fires again. `workflow_dispatch` is the manual way for a tag that is already published
+# (the gate runs the same checks), for example to create the release of a tag that was published before this workflow existed.
+#
+# `workflow_run` workflows always run the file on the default branch, so a change here takes effect after it is merged.
+
+on:
+  workflow_run:
+    workflows: ["Release native (arcadedb)", "Release JVM (arcadedb-jvm)"] # the `name:` of the two release workflows
+    types: [completed]
+  workflow_dispatch:
+    inputs:
+      tag:
+        description: "The bare version of a tag that is already published, for example 26.11.1"
+        required: true
+        type: string
+
+permissions:
+  contents: read
+
+concurrency:
+  group: github-release-${{ github.event.workflow_run.head_branch || inputs.tag }}
+  cancel-in-progress: false
+
+jobs:
+  github-release:
+    name: GitHub Release
+    # A release workflow that did not succeed, or one that was not started by a tag push, never gets here.
+    if: >-
+      ${{ github.repository == 'ArcadeData/arcadedb-python' &&
+      (github.event_name == 'workflow_dispatch' ||
+      (github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.event == 'push')) }}
+    runs-on: ubuntu-24.04
+    timeout-minutes: 15
+    permissions:
+      contents: write # create the release on the existing tag
+      issues: write # close the milestone, and read the milestone's pull requests
+      pull-requests: read # merge commit of each pull request
+      actions: read # the runs of the two release workflows
+    env:
+      GH_TOKEN: ${{ github.token }}
+      GH_REPO: ${{ github.repository }}
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+
+      # Both packages must be published before anything is created. Exit 0 with ready=false means "the other package is not finished
+      # yet"; exit 1 means the workflows say success but PyPI does not have the version, which a person must look at.
+      - name: Both release workflows succeeded and both packages are on PyPI
+        id: gate
+        env:
+          INPUT_TAG: ${{ inputs.tag }}
+          RUN_TAG: ${{ github.event.workflow_run.head_branch }}
+        run: |
+          set -euo pipefail
+          tag="${INPUT_TAG:-$RUN_TAG}"
+          if ! [[ "$tag" =~ ^[0-9]+\.[0-9]+\.[0-9]+(\.(dev|post)[0-9]+)?$ ]]; then
+            echo "::error::'$tag' is not A.B.C, A.B.C.devN or A.B.C.postN (bare version, no v, no aN/bN/rcN)"
+            exit 1
+          fi
+          base="${tag%%.dev*}"
+          base="${base%%.post*}"
+          if [[ "$tag" == *.dev* ]]; then kind=dev; elif [[ "$tag" == *.post* ]]; then kind=post; else kind=final; fi
+          sha="$(gh api "repos/$GH_REPO/commits/$tag" --jq .sha)"
+          for wf in release-native.yml release-jvm.yml; do
+            conclusion="$(gh api "repos/$GH_REPO/actions/workflows/$wf/runs?head_sha=$sha&event=push&per_page=1" --jq '.workflow_runs[0].conclusion // "none"')"
+            echo "$wf for $tag ($sha): $conclusion"
+            if [ "$conclusion" != success ]; then
+              echo "::notice::$wf has not succeeded for $tag ($conclusion): no GitHub Release yet. This workflow runs again when it finishes."
+              echo "ready=false" >> "$GITHUB_OUTPUT"
+              exit 0
+            fi
+          done
+          for dist in arcadedb arcadedb-jvm; do
+            code="$(curl -sS -o /dev/null -w '%{http_code}' "https://pypi.org/pypi/$dist/$tag/json")"
+            if [ "$code" != 200 ]; then
+              echo "::error::$dist $tag is not on PyPI (HTTP $code) although its release workflow succeeded"
+              exit 1
+            fi
+          done
+          { echo "ready=true"; echo "tag=$tag"; echo "base=$base"; echo "kind=$kind"; } >> "$GITHUB_OUTPUT"
+
+      - name: Write the notes
+        if: steps.gate.outputs.ready == 'true'
+        env:
+          TAG: ${{ steps.gate.outputs.tag }}
+        run: python3 -I scripts/release_notes.py "$TAG" --output "$RUNNER_TEMP/notes.md"
+
+      # A pre-release for A.B.C.devN, a normal release for A.B.C and A.B.C.postN. --verify-tag: the tag must exist, the release never
+      # creates one. When the release exists (the other firing, or an earlier run) its notes and its pre-release flag are brought up to date.
+      - name: Create or update the GitHub Release
+        if: steps.gate.outputs.ready == 'true'
+        env:
+          TAG: ${{ steps.gate.outputs.tag }}
+          KIND: ${{ steps.gate.outputs.kind }}
+        run: |
+          set -euo pipefail
+          if [ "$KIND" = dev ]; then pre=true; else pre=false; fi
+          if gh release view "$TAG" > /dev/null 2>&1; then
+            gh release edit "$TAG" --title "$TAG" --notes-file "$RUNNER_TEMP/notes.md" --prerelease="$pre"
+          elif [ "$pre" = true ]; then
+            gh release create "$TAG" --verify-tag --title "$TAG" --notes-file "$RUNNER_TEMP/notes.md" --prerelease
+          else
+            gh release create "$TAG" --verify-tag --title "$TAG" --notes-file "$RUNNER_TEMP/notes.md" --latest
+          fi
+          gh release view "$TAG" --json url,isPrerelease --jq '"release: " + .url + " (prerelease: " + (.isPrerelease|tostring) + ")"'
+
+      # Only a final release closes its milestone. A dev tag leaves it open (the cycle goes on); a post tag has no milestone of its own
+      # (its notes come from the commit range, not from a milestone). Items still open in the milestone move to the next one first, and
+      # every move is logged. The next one is the open milestone with the earliest due date after this one. When there is none, the
+      # milestone is created, named after the next upstream engine version (due 30 days after this one; the base of the upstream `main` version, as in
+      # development/release.adoc), when that can be read. Otherwise nothing is moved or closed and the job fails, so the maintainer
+      # creates the next milestone and re-runs this workflow (the release step is a no-op edit the second time).
+      - name: Move the open items and close the milestone of a final release
+        if: steps.gate.outputs.ready == 'true' && steps.gate.outputs.kind == 'final'
+        env:
+          BASE: ${{ steps.gate.outputs.base }}
+        run: |
+          set -euo pipefail
+          milestones="$(gh api "repos/$GH_REPO/milestones?state=all&per_page=100")"
+          row="$(jq -r --arg t "$BASE" '[.[] | select(.title == $t)][0] // empty | "\(.number) \(.state) \(.open_issues) \(.due_on // "")"' <<< "$milestones")"
+          if [ -z "$row" ]; then
+            echo "::warning::No milestone named $BASE: nothing to close"
+            exit 0
+          fi
+          read -r number state open_issues due <<< "$row"
+          if [ "$state" = closed ]; then
+            echo "Milestone $BASE (#$number) is already closed"
+            exit 0
+          fi
+          if [ "$open_issues" != 0 ]; then
+            next="$(jq -r --arg n "$number" --arg due "$due" '[.[] | select(.state == "open" and (.number | tostring) != $n and .due_on != null and ($due == "" or .due_on > $due))] | sort_by(.due_on) | .[0].number // empty' <<< "$milestones")"
+            if [ -z "$next" ]; then
+              upstream="$(curl -fsS https://raw.githubusercontent.com/ArcadeData/arcadedb/main/pom.xml | sed -n '/<artifactId>arcadedb-parent<\/artifactId>/,/<\/version>/ s/.*<version>\(.*\)-SNAPSHOT<\/version>.*/\1/p' | head -n 1 || true)"
+              if [[ "$upstream" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ && "$upstream" != "$BASE" ]]; then
+                found="$(jq -r --arg t "$upstream" '[.[] | select(.title == $t)][0] // empty | "\(.number) \(.state)"' <<< "$milestones")"
+                if [ -z "$found" ]; then
+                  # due 30 days after this one (releases are monthly), so a pick-by-due-date finds it
+                  args=(-f title="$upstream")
+                  [ -z "$due" ] || args+=(-f due_on="$(date -u -d "$due + 30 days" +%Y-%m-%dT%H:%M:%SZ)")
+                  next="$(gh api --method POST "repos/$GH_REPO/milestones" "${args[@]}" --jq .number)"
+                  echo "Created milestone $upstream (#$next), named after the next upstream engine version"
+                elif [ "${found#* }" = open ]; then
+                  next="${found%% *}"
+                fi
+              fi
+            fi
+            if [ -z "$next" ]; then
+              echo "::error::Milestone $BASE has $open_issues open items and there is no next open milestone with a due date, and the next upstream version could not be used (read '${upstream:-nothing}'). Nothing was moved or closed. Create the next milestone with a due date, then re-run this workflow."
+              exit 1
+            fi
+            for item in $(gh api --paginate "repos/$GH_REPO/issues?milestone=$number&state=open&per_page=100" --jq '.[].number'); do
+              gh api --method PATCH "repos/$GH_REPO/issues/$item" -F milestone="$next" --jq '"moved #\(.number) \(.title) to milestone \(.milestone.title)"'
+            done
+          fi
+          gh api --method PATCH "repos/$GH_REPO/milestones/$number" -f state=closed --jq '"closed milestone " + .title + " (" + .html_url + ")"'

CLAUDE.md

@@ -73,6 +73,6 @@ Each package has one test and one release workflow: `test-jvm.yml` and `release-
 
 `.github/workflows/test-jvm.yml` builds ONE wheel per platform (linux/amd64, linux/arm64, darwin/arm64, windows/amd64), tagged `py3-none-<platform>` by `setup.py` (`bdist_wheel.get_tag`; `scripts/verify_wheel_tag.py` checks it), and tests that wheel on Python 3.11–3.14; also runs nightly against `arcadedata/arcadedb:latest`. `requires-python` is `>=3.11,<3.15` because the tag no longer keeps pip off 3.15 and JPype has no 3.15 wheel; drop the cap with it. The org shares a runner cap with upstream, so the unit tests run 10 legs on every event (linux/amd64 on all four Pythons, the other three platforms on 3.11 and 3.14, the shape of test-native.yml), and the examples run all four platforms on 3.14 on a pull request (4 legs, 28 checks in all) and 16 legs on push to main, the nightly run, dispatch and the release (40 checks). `scripts/select_matrix.py` holds both lists (`PULL_REQUEST_LEGS`, keyed `jvm` and `examples`), the first job of each workflow runs it, and `tests/test_select_matrix.py` pins the full matrix; docs page `development/ci.adoc`. Keep the job names `Test arcadedb (<platform>, Python <version>)` unchanged. Both workflows accept an `arcadedb-tag` input to pin the image. `test-python-examples.yml` runs `examples/`. A dependency-floor audit locks the dependencies and extras on a throwaway uv project with `--resolution lowest-direct` for each Python classifier and runs `uvx pip-audit` on the export, so raise lower bounds when a floor has an advisory. No tool is installed with pip anywhere in CI or the scripts: one-off tools run through `uvx tool==X`, the wheel is built with `uvx --from "build[uv]==1.6.1" pyproject-build --installer uv`, and the uv install URL is pinned (`0.12.0`).
 
-`.github/workflows/release-jvm.yml` publishes `arcadedb-jvm` on a bare-version tag (`26.11.1.dev0`, `26.11.1`, `26.11.1.post1`; no `v`, never `aN`/`bN`/`rcN`). Its first job runs `scripts/check_version.py` (the devN/postN counting rule, read-only, fails closed); `scripts/release_preflight.py <VERSION>` checks everything before the tag (read-only); `scripts/after_tag_verify.sh` (`PACKAGE=native` for `arcadedb`) checks the finished release; `scripts/bump_engine.py` moves the dev-engine pins after a release. The monthly steps are the "Release day" section of `development/release.adoc` and the issue template `.github/ISSUE_TEMPLATE/release.md`. The rule is stated once, on the docs page `development/release.adoc`.
+`.github/workflows/release-jvm.yml` publishes `arcadedb-jvm` on a bare-version tag (`26.11.1.dev0`, `26.11.1`, `26.11.1.post1`; no `v`, never `aN`/`bN`/`rcN`). Its first job runs `scripts/check_version.py` (the devN/postN counting rule, read-only, fails closed); `scripts/release_preflight.py <VERSION>` checks everything before the tag (read-only); `scripts/after_tag_verify.sh` (`PACKAGE=native` for `arcadedb`) checks the finished release; `scripts/bump_engine.py` moves the dev-engine pins after a release. The monthly steps are the "Release day" section of `development/release.adoc` and the issue template `.github/ISSUE_TEMPLATE/release.md`. `.github/workflows/release-notes.yml` (`workflow_run` on both release workflows, `workflow_dispatch` for a published tag) creates the GitHub Release once both packages are published, with notes from `scripts/release_notes.py` (merged pull requests of the milestone named after the base version for a final and a `devN` release, the commit range since the previous tag for `postN`, grouped by label; `--dry-run` previews), a pre-release for `devN`, and after a final release moves the open items to the next milestone (created from the next upstream engine version when none exists) and closes the milestone. The rule is stated once, on the docs page `development/release.adoc`.
 
 `test-native.yml` builds the GraalVM native image and runs the parity suite against the installed wheel (selective: scripts/select_native_matrix.py, 5 to 30 minutes per platform, do not add runs lightly). `.github/workflows/release-native.yml` publishes `arcadedb` on the same tag: it calls `test-native.yml` with `release: true` (4 platforms x Python 3.11 to 3.14), then the readiness gate (the repository variable `ARCADEDB_NATIVE_READINESS_BAR_MET`, for a final or post version), then one publish job through the environment `pypi`.