robostack-bot: monthly upgrade-pixi PR for the tooling's dependencies

.github/ISSUE_TEMPLATE/3_robostack_bot.yaml

@@ -19,14 +19,15 @@ body:
         | `update-vinca` | maintainers | Move the vinca pin in `pixi.toml` to the newest commit of its branch (or the ref given under Packages), lock and check. The PR lists the vinca commits. |
         | `dependency-report` | maintainers | Check the dependencies against the pins and update the distribution's dependency-report issue (closed when nothing conflicts). Runs weekly. |
         | `update-build-timings` | maintainers | Record the build times of the last week's build-branch runs in `shared/build_timings/`, which the build workflows use to pack each stage's recipes into jobs by time. Runs weekly. |
+        | `upgrade-pixi` | maintainers | Upgrade the tooling's dependencies in `pixi.toml` / `pixi.lock` (`pixi upgrade`, except the deliberate caps such as rattler-build); the PR lists the changes. Runs monthly. |
 
         On a pull request, `@robostack-bot rebuild-dependents` (maintainers) commits the rebuilds its ABI check suggests to the pull request's branch.
   - type: dropdown
     id: command
     attributes:
       label: Command
       options: [add-package, update-rosdistro-snapshot, find-stale-packages, update-conda-forge-pinning, rebuild, update-vinca,
-                dependency-report, update-build-timings]
+                dependency-report, update-build-timings, upgrade-pixi]
     validations:
       required: true
   - type: input

.github/workflows/bot.yaml

@@ -10,13 +10,15 @@
 #   distribution ("<distro>: dependency report", label dependency-report), closed when
 #   nothing conflicts; and a PR with the build times the week's build-branch runs
 #   recorded (shared/build_timings/, for packing the build jobs by time).
+# - monthly (on the 1st): a PR upgrading the tooling's dependencies in pixi.toml / pixi.lock
+#   (pixi upgrade, except the deliberate caps in tools/maintenance.py PIXI_UPGRADE_KEEP).
 # - on demand (owners, members and collaborators):
 #     Actions > robostack-bot > Run workflow, an issue from the "robostack-bot command"
 #     template, or a comment: @robostack-bot <command> [<distro>]
 #   commands: update-rosdistro-snapshot [<distro>...|all] | find-stale-packages [<distro>...|all] |
 #             update-conda-forge-pinning [<distro>...|all] | add-package <pkg>... [<distro>...] |
 #             rebuild <pkg>... [<distro>...|all] [--with-dependents] | update-vinca [<ref>] |
-#             dependency-report [<distro>...|all] | update-build-timings |
+#             dependency-report [<distro>...|all] | update-build-timings | upgrade-pixi |
 #             rebuild-dependents (a comment on a pull request: commits the rebuilds its ABI
 #             check suggests to its branch; not possible for pull requests from forks)
 # - package requests (the "Package request" issue form or `@robostack-bot add-package`,
@@ -31,12 +33,13 @@ on:
   schedule:
     - cron: "17 3 * * *"
     - cron: "43 4 * * 1"  # weekly dependency report (the plan job matches this string)
+    - cron: "29 5 1 * *"  # monthly pixi upgrade (the plan job matches this string)
   workflow_dispatch:
     inputs:
       command:
         type: choice
         options: [add-package, update-rosdistro-snapshot, find-stale-packages, update-conda-forge-pinning, rebuild,
-                  update-vinca, dependency-report, update-build-timings]
+                  update-vinca, dependency-report, update-build-timings, upgrade-pixi]
       distro:
         description: Distribution (for the per-distribution commands; empty = all)
         required: false
@@ -97,6 +100,10 @@ jobs:
                 echo "jobs=$jobs" | tee -a "$GITHUB_OUTPUT"
                 exit 0
               fi
+              if [ "$SCHEDULE" = "29 5 1 * *" ]; then
+                echo 'jobs=[{"command":"upgrade-pixi","distro":"","args":"","preview":false}]' | tee -a "$GITHUB_OUTPUT"
+                exit 0
+              fi
               # rosdistro syncs, and the conda-forge pinning: one job for all distributions
               # (they share it; one with its own pinning is updated in the same pull request)
               jobs=$(pixi run -q rs rosdistro-syncs | python3 -c 'import json,sys; print(json.dumps(json.load(sys.stdin) + [{"command":"update-conda-forge-pinning","distro":"","args":"","preview":False}]))') ;;
@@ -166,6 +173,11 @@ jobs:
             update-vinca) pixi run rs update-vinca $ARGS --summary "$RUNNER_TEMP/summary.md" ;;
             # the build-branch runs' build-times artifacts (actions: read)
             update-build-timings) pixi run rs update-build-timings --summary "$RUNNER_TEMP/summary.md" ;;
+            # pixi upgrade itself not inside `pixi run`: it reinstalls the environment
+            upgrade-pixi)
+              # shellcheck disable=SC2046  # a list of --exclude <name> arguments
+              pixi upgrade --json $(pixi run -q rs upgrade-pixi --exclude-args) > "$RUNNER_TEMP/upgrade.json" &&
+                pixi run rs upgrade-pixi --changes "$RUNNER_TEMP/upgrade.json" --summary "$RUNNER_TEMP/summary.md" ;;
             # creates / updates the distribution's dependency-report issue
             dependency-report) pixi run rs "$DISTRO" dependency-report --summary "$RUNNER_TEMP/summary.md" ;;
           esac

README.md

@@ -150,6 +150,7 @@ waiting for the macOS runners; remove them for a full build before merging.
 | `update-vinca [<ref>]` | maintainers | Moves the vinca pin in `pixi.toml` to the newest commit of the pinned repository's branch (`robostack-integration` for Tobias-Fischer/vinca, else its default branch) or of `<ref>`, runs `pixi lock`, re-renders the pinning and runs `rs check`. The PR links the comparison and lists the vinca commits. |
 | `dependency-report [<distro>... \| all]` | maintainers | Runs the dependency check (`check-deps`, linux-64) and keeps one issue per distribution, *`<distro>`: dependency report* (label `dependency-report`), up to date with the conflicts, notes and conda-forge migration status; the issue is closed when nothing conflicts. Runs weekly (Mondays). |
 | `update-build-timings` | maintainers | Merges the build times the last week's build-branch runs recorded (`rs <distro> build-ci`, kept as each job's artifact) into `shared/build_timings/<platform>.json` and opens a PR. The build workflows pack each stage's recipes into jobs by these times. Runs weekly (Mondays). |
+| `upgrade-pixi` | maintainers | Runs `pixi upgrade` on the tooling's dependencies in `pixi.toml` / `pixi.lock`, except the deliberate caps in `PIXI_UPGRADE_KEEP` (`tools/maintenance.py`: rattler-build, cmake, setuptools, and vinca, which `update-vinca` moves), runs `rs check` in the upgraded environment and opens a PR with the changes as tables (`pixi-diff-to-markdown`). Runs monthly (on the 1st). |
 
 There are three ways to trigger a command:
 - a *Package request* or *robostack-bot command* issue;

tools/maintenance.py

@@ -10,6 +10,8 @@
     pixi run rs rebuild PKG... [DISTRO...] [--with-dependents]  # per-package build-number bumps
     pixi run rs rebuild-dependents PR                 # commit the ABI check's suggested rebuilds to the PR
     pixi run rs update-vinca [REF]                    # move the vinca pin to the newest commit of its branch
+    pixi upgrade --json $(pixi run -q rs upgrade-pixi --exclude-args) > up.json
+    pixi run rs upgrade-pixi --changes up.json        # summarise a `pixi upgrade` of the tooling
     pixi run rs <distro> dependency-report            # dependency check, kept in a dependency-report issue
     pixi run rs parse-command --body TEXT --association ROLE   # for @robostack-bot comments
 
@@ -44,6 +46,7 @@
     "update-vinca": "move the vinca pin to the newest commit of its branch (opens a PR)",
     "dependency-report": "check the dependencies against the pins, in the distribution's dependency-report issue",
     "update-build-timings": "record the build times of the last week's build-branch runs, for batching by time (opens a PR)",
+    "upgrade-pixi": "upgrade the tooling's dependencies in pixi.toml / pixi.lock (opens a PR)",
 }
 PER_DISTRO_COMMANDS = ("update-rosdistro-snapshot", "find-stale-packages", "update-conda-forge-pinning",
                        "dependency-report")
@@ -1175,6 +1178,37 @@ def update_build_timings() -> Result:
                   + ". `rs <distro> gha` packs each stage's recipes into jobs by them.", changed=True)
 
 
+# `pixi upgrade` widens every version spec in pixi.toml to the newest release; these keep
+# their spec (and lock) and are moved by hand, or by update-vinca
+PIXI_UPGRADE_KEEP = {
+    "rattler-build": "the builds depend on its behaviour (variants, build ids, --skip-existing); moved deliberately",
+    "cmake": "capped below 4: CMake 4 rejects the cmake_minimum_required below 3.5 of older packages",
+    "setuptools": "capped below 82, which removed pkg_resources",
+    "vinca": "a git commit, moved by `@robostack-bot update-vinca`",
+}
+
+
+def upgrade_pixi(changes: Path) -> Result:
+    """Summarise a `pixi upgrade --json` of pixi.toml / pixi.lock (run beforehand, outside
+    `pixi run`: the upgrade reinstalls the environment), then run the repository checks
+    in the upgraded environment."""
+    diff = subprocess.run(["git", "status", "--porcelain", "--", "pixi.toml", "pixi.lock"], cwd=rs.ROOT,
+                          capture_output=True, text=True, check=True).stdout.strip()
+    if not diff:
+        return Result("pixi dependencies are up to date", "`pixi upgrade` found nothing newer.")
+    md = subprocess.run(["pixi", "exec", "pixi-diff-to-markdown"], input=changes.read_text(), cwd=rs.ROOT,
+                        capture_output=True, text=True)
+    lines = [md.stdout.strip() if md.returncode == 0 else
+             f"❌ `pixi-diff-to-markdown` failed:\n\n```\n{tail(md.stdout + md.stderr, 20)}\n```", "",
+             "Kept as they are (`PIXI_UPGRADE_KEEP` in `tools/maintenance.py`):", ""]
+    lines += [f"- `{name}`: {why}" for name, why in PIXI_UPGRADE_KEEP.items()]
+    print("pixi run rs check", flush=True)
+    checked = subprocess.run(["pixi", "run", "rs", "check"], cwd=rs.ROOT, capture_output=True, text=True)
+    lines += ["", "`pixi run rs check`: " + ("✅ ok" if checked.returncode == 0 else
+                                            f"❌ problems:\n\n```\n{tail(checked.stdout + checked.stderr, 40)}\n```")]
+    return Result("Upgrade pixi dependencies", "\n".join(lines), changed=True)
+
+
 def _default_ci_yaml() -> str:
     return (rs.TOOLS / "ci_default.yaml").read_text()
 
@@ -1373,6 +1407,10 @@ def main(command: str, argv: list[str], distro: str | None = None) -> int:
         parser.add_argument("pull_request")
     if command == "update-vinca":
         parser.add_argument("ref", nargs="?", help="vinca branch, tag or commit (default: the pinned branch)")
+    if command == "upgrade-pixi":
+        group = parser.add_mutually_exclusive_group(required=True)
+        group.add_argument("--changes", type=Path, help="the output of `pixi upgrade --json`")
+        group.add_argument("--exclude-args", action="store_true", help="print the --exclude arguments for pixi upgrade")
     args = parser.parse_args(argv)
     if command == "parse-command":
         print(json.dumps(parse_command(args.body, args.association, args.pull_request)))
@@ -1390,6 +1428,11 @@ def main(command: str, argv: list[str], distro: str | None = None) -> int:
         return report(dependency_report(distro), args.summary)
     if command == "update-build-timings":
         return report(update_build_timings(), args.summary)
+    if command == "upgrade-pixi":
+        if args.exclude_args:
+            print(" ".join(f"--exclude {name}" for name in PIXI_UPGRADE_KEEP))
+            return 0
+        return report(upgrade_pixi(args.changes), args.summary)
     if command == "add-package":
         known = set(rs.distros())
         distros = [n for n in args.names if n in known]

tools/robostack.py

@@ -692,7 +692,8 @@ def changed_distros(base: str) -> list[str]:
 
 
 REPO_COMMANDS = ("check", "update-pinning", "new-distro", "parse-command", "add-package", "rosdistro-syncs",
-                 "rebuild", "rebuild-dependents", "update-vinca", "update-build-timings")
+                 "rebuild", "rebuild-dependents", "update-vinca", "update-build-timings",
+                 "upgrade-pixi")
 
 
 def main() -> int:

tools/tests/test_bot_commands.py

@@ -1,5 +1,5 @@
 """robostack-bot: command parsing and the pure helpers of rebuild, rebuild-dependents,
-update-vinca and dependency-report (no network)."""
+update-vinca, upgrade-pixi and dependency-report (no network)."""
 
 import json
 
@@ -317,3 +317,15 @@ def test_uploads_retry_five_times_waiting_longer(monkeypatch):
     monkeypatch.setattr(rs.time, "sleep", waits.append)
     assert rs.retry(["upload"], rs.ROOT, "anaconda.org") == 0
     assert waits == [30, 60, 120, 240]
+
+
+def test_upgrade_pixi_parsing_and_kept_caps(capsys):
+    assert jobs("@robostack-bot upgrade-pixi") == [{"command": "upgrade-pixi", "distro": "", "args": "", "preview": False}]
+    assert jobs("@robostack-bot upgrade-pixi", association="NONE") == []  # maintainers only
+    # the caps pixi upgrade must leave alone are all in pixi.toml
+    pixi = (rs.ROOT / "pixi.toml").read_text()
+    for name in maintenance.PIXI_UPGRADE_KEEP:
+        assert f"\n{name} = " in pixi
+    assert maintenance.main("upgrade-pixi", ["--exclude-args"]) == 0
+    assert capsys.readouterr().out.split() == [
+        arg for name in maintenance.PIXI_UPGRADE_KEEP for arg in ("--exclude", name)]