Reduce pull request runner occupancy without weakening main protection

[Astro-Han]

GitHub Actions runners are a shared ASF resource, and concurrent runner slots are scarcer than minutes. This is a measured audit of what pull requests currently occupy and what can be given back without lowering the bar for main.

Method

• Job, step and queue timings from recent runs of each pull-request workflow: 120 ci runs (32 with complete three-job timing), 300 release-windows-check runs covering 2026-08-29 to 2026-09-01.
• Path filters replayed against the last 200 first-parent commits on main, per filter entry, recording how often each entry was the sole reason a workflow started.

Current cost

Pull requests occupy roughly 2900 runner slots per day:

Workflow	Runs/day	Runners per run	Slots/day
cli-package-validation	~80	14	~1120
ci	~400	2.875 avg	~1150
windows-recovery	~264	1 (Windows)	~264
release-windows-check	~108	1 (Windows, 25m29s)	~108

Findings

1. ci.yml makes every pull request queue three times to produce one verdict

plan, heavy and test are three jobs, so a pull request is allocated a runner, queued, and released three times. plan executes for 19 seconds; test executes for 4 seconds and only runs a bash case.

Measured queue, over 32 runs with complete timing:

Stage	Mean	p50	p90
plan queue	5m57s	0m58s	20m21s
heavy queue (after plan)	4m46s	0m16s	18m39s
test queue (after heavy)	7m57s	1m04s	19m13s
Total	18m40s	2m18s	—

The distribution is bimodal: when the pool is free each stage waits about a minute, and when it is starved each waits about twenty. In run 33485186715, plan queued 30 minutes to execute 16 seconds of work, and heavy then queued another 18 minutes.

Change: merge the three into one unconditional job. Affected-surface planning becomes its first step; every later step keeps the per-step if: gating ci.yml:130-370 already uses. That job is then a stable required context on its own, so .asf.yaml requires it directly.

Runner allocations per ci run:

	Today	Merged
heavy selected (87.5%)	3	1
Documentation-only (12.5%)	2	1
Weighted mean	2.875	1

Saves: ~750 runner allocations/day, and ~12m44s of queue per run — about 85 runner-hours/day of pure waiting.

Costs nothing in runner minutes. The install-free checks run on a runner the job allocates anyway. A documentation-only pull request goes from two runners (~25s total) to one (~30s), so it gets cheaper too — it does not currently occupy zero runners. Fail-fast is preserved: the install-free checks still precede Install dependencies, and a failure there costs one allocation instead of today's two.

Timeout headroom is not at risk. The longest heavy observed is 25m37s (run 33480676324), and that run had every lane on — Desktop e2e 678s, Storybook smoke 110s, Playwright install and Storybook build 40s, CLI pack and smoke 130s, Runtime Host 141s, workspace tests 154s, alignment audit 60s — with only the ASF source lane skipped. Plus plan's 29s worst case that is ~26 minutes against timeout-minutes: 45, leaving 42% slack. No run in the 28-PR and 3-push sample came near 35 minutes.

Gives up: plan's outputs become step outputs rather than job outputs, and one long log replaces three. Re-running only heavy is no longer possible, but GitHub re-runs the whole job either way and the extra work is 19 seconds.

The CLI Rust packaging steps stay on this lane. Kache restore is 5s, the build 46s and the smoke 84s — 8.6% of a 26-minute job, not a long tail, and the only pre-merge evidence that the CLI still packages. The genuine long tail is Desktop e2e at 44%, and sharding it would raise runner count.

2. cli-package-validation opens 14 runners; 5 merge losslessly

The cross-platform jobs cannot fold into the single ubuntu lane without deleting the evidence they exist for. The waste is in job splitting:

• The three state-root-qualification matrix jobs become one. Two of them qualify a transition between hardcoded, already-published tarballs; nothing in a pull request can change that outcome except scripts/qualify-released-cli-state-root.mjs. Serial cost ~3m30s on a job that is not on the critical path.
• Linux x64 / Node 22.19 and Linux x64 / Node 24 become one job running both versions.
• release-predecessor (15s, resolves one npm version) folds into build, which already waits ~16 minutes on peer-native.

Saves: 14 → 9 runners, ~400 slots/day. Gives up: nothing.

3. windows-recovery triggers on 60% of merges, against its own stated rule

The rule is already a forward closure, but computed at workspace granularity and enforced as such by scripts/ci-test-plan.test.mjs: it derives {runtime, runtime-host, storage} from the dist paths the steps run, closes over the workspace dependency graph, and asserts every resulting workspace contributes src/**, tsconfig.json and package.json to the filter. That is why packages/runtime-host/src/** is in, and why peer-mesh, WebRTC upgrades and OAuth Connection work all pull a Windows runner. The change is to compute the same closure at file granularity, which means replacing that contract, not just editing the YAML.

Entry	Hits / 200	Sole driver
packages/runtime-host/src/**	74	33
packages/core/src/**	44	6
packages/runtime/src/**	39	6
packages/storage/src/**	38	4

Examples dragged in with no recovery surface: #4405, #3924, #4274, #4216.

Only 3 of the last 100 pull-request runs failed: two dependabot bumps (package-lock.json) and #3731 (packages/runtime/src/sandbox/). A concern-scoped filter replays at 51/200 instead of 121/200 and still catches all three.

Change: generate the paths from the file-granularity forward import closure of the dist test files the job already names in its steps, with a --check gate in the install-free lane beside windows:inventory and check:app-shell-hooks, replacing the workspace-granularity assertion in scripts/ci-test-plan.test.mjs. push: main (unfiltered) and the nightly stay as they are. Drop packages/core/src/** as part of the same change.

Saves: ~150 Windows runners and ~450 Windows minutes/day. Gives up: an indirectly-affected recovery authority outside the generated closure is caught by the unfiltered push: main run minutes after merge instead of before it. windows_recovery is already deliberately not a required context.

windows-baseline.yml does not overlap: it covers PTY input, PowerShell UTF-8, storage path/lock gates and residual processes, and its own comment defers crash and owner-death evidence to this lane.

4. release-windows-check spends most of its Windows minutes on one line and one flaky download

This is a single windows-2025 job measured at 25m29s, triggering on 55/200 merges — the largest Windows minute consumer in the repository. Two separate problems.

One file drives a quarter of the triggers, for one line. The packaging script entries are never sole drivers; the desktop main-process entries are lopsided:

Entry	Hits / 200	Sole driver
apps/desktop/src/main/runtime-host-boot.ts	19	13
apps/desktop/src/main/main-window.ts	1	0
apps/desktop/src/main/app-update-service.ts	1	0

The only reason the comment gives for runtime-host-boot.ts is that it hands MAKA_UPDATE_TEST_FEED to the updater. That is runtime-host-boot.ts:792 — one line in a 1964-line file — a symbol-level concern expressed as a file-level filter, costing 13/200 merges of 25-minute Windows packaging.

The lane has never failed on the steps that concern protects. Across 300 pull-request runs, 36 failures:

Failing step	Count
Download and verify the pinned Windows upgrade baseline	14
Install dependencies	10
Package the Windows installer and ZIP	9
Exercise pinned-version upgrade and uninstall	3
Verify the Windows release	0
Build the version-bumped autoupdate installer	0
Verify automatic update end to end	0
Prove deterministic mid-install failure rollback	0

Every failure is at packaging or earlier. The whole autoupdate path — the reason runtime-host-boot.ts is in the filter — has never gone red.

The dominant failure mode is not a product signal. The 14 upgrade-baseline failures (39% of all failures) land on branches that cannot have caused them: ci/dedupe-asf-npm-policy-check, refactor/model-catalog-host-authority, fix/ui-build-astryx-0.5.0, feat/plugin-platform-foundation. That step has scripts/prepare-windows-upgrade-baseline.mjs download a pinned historical release, so it fails on infrastructure, not on the diff. Each false red also costs a re-push and a second 25-minute Windows runner — roughly 5 per day.

Change:

• Move runtime-host-boot.ts out of the pull-request filter and add an install-free contract test asserting the boot file still forwards MAKA_UPDATE_TEST_FEED. Leave main-window.ts and app-update-service.ts in place — they are never sole drivers, so moving them would save nothing and only remove coverage.
• Move Download and verify the pinned Windows upgrade baseline, Exercise pinned-version upgrade and uninstall and Prove deterministic mid-install failure rollback off the pull-request path. Their input is a pinned historical release, not the diff.
• Add a schedule trigger so the full packaged Windows path, including those three steps, runs nightly rather than first on release day.

Saves: ~26 runs/day, and the remaining job drops from 25m29s to roughly 12 minutes — about 1700 Windows minutes/day, plus the ~5 daily re-runs the flaky baseline step causes.

Gives up: a boot-wiring change that breaks the packaged launch, and an installer upgrade/rollback regression, are found nightly instead of before merge. Nothing else covers packaged boot — windows-baseline nightly runs smoke:windows:dist against unpackaged dist.

5. App icon drift is gated on the wrong surface

ci.yml runs the app icon drift tests whenever code == 'true' (52s, regenerating PNGs through Python). Its real inputs are apps/desktop/assets/app-icons/**, scripts/generate-app-icons.py, scripts/verify-packaged-app.mjs and APP_ICONS in packages/core/src/settings.ts. Narrowing it to those costs no coverage.

Deliberately not proposed

• Trimming checks inside the heavy lane. The renderer architecture ratchet, app icon drift, both knip runs, astryx:theme --check and the surface inventory total 1m32s of a 22m38s job on an already-allocated runner. Deleting all of them saves zero runners and gives up four regression classes nothing else covers.
• Sharding Desktop e2e. It is 11m11s, 44–49% of the heavy lane, but sharding increases runner count.
• Moving CLI Rust packaging to nightly. 2m15s with the Kache cache warm, and the only pre-merge CLI packaging evidence.
• Cutting tests. Test cost has zero elasticity against runner count: every test runs on a runner allocated for another reason. The install-free policy tests are brittle, but they complete in 22 seconds and guard release invariants (id-token: write absent from the preflight, no NPM_TOKEN in publish, header audit before npm ci) that nothing else guards.

Effect

Pull-request runner slots fall from ~2900 to ~1650 per day (-43%), and roughly 85 runner-hours of daily queue disappear with the two extra ci allocations. Windows slots fall from ~590 to ~415 (-30%) and Windows minutes from ~4750 to ~2530 (-47%). Required checks still report unconditionally on every pull request, and the non-author committer approval is untouched.

Plan

One pull request. None of these five pieces has a merge constraint that forces it onto its own schedule or route: cli-package-validation, windows-recovery and release-windows-check are all paths-filtered and therefore cannot be required contexts, and every piece lands against the same main. Ordering between them is commit order, not a pull request boundary.

The merged ci.yml job keeps the name test. .asf.yaml then needs no change at all: branch protection keeps requiring test, which is now the single unconditional job that plans and validates. Renaming it instead would deadlock — .asf.yaml only takes effect once merged to main, so a branch whose workflow no longer produces test could never satisfy the protection it is still under. Nothing outside ci.yml and scripts/ci-test-plan.test.mjs references the plan or heavy job names, and no workflow listens to this one via workflow_run.

Commits, in dependency order:

• ci: merge planning and validation into the required test job
• ci: gate app icon drift on the artwork it verifies
• ci: fold redundant CLI package validation jobs
• ci: derive the Windows recovery filter from its executed tests
• ci: scope the packaged Windows gate to its own inputs

[Astro-Han]

take

[Astro-Han]

Implemented in #4461. Two corrections to this issue, both from measurement.

cli-package-validation is 14 → 10, not 9

−1 predecessor, −2 state-root, −1 smoke. Arithmetic slip above.

Finding 3 (windows-recovery) was wrong about both the cost and the fix

The proposed fix does not pay for itself. I built the generator. The forward import closure of the 14 recovery dist tests is 709 files — runtime-host 245/443, runtime 231/492, core 123/257, storage 110/244 — because these tests boot a real Runtime Host and are therefore integration tests over most of those workspaces. The generated filter matches 106 of 200 commits against the current filter's 118. That is ~10%, not the ~58% projected above from a hand-scoped list, and it costs a new generator plus a contract test. The hand-written filter was not inaccurate; it was a correct description of a genuinely wide closure.

The cost table above also overstates this lane. It ranked Windows lanes by slot count without weighting by duration. Measured over 300 runs:

	exec	queue (p50 / p90)
windows-recovery	2m49s	4s / 39s
release-windows-check	25m29s	—

windows-recovery is ~12 Windows-hours/day against release-windows-check's ~46 from a fifth of the runs, and it does not queue. Where the minutes actually go, npm ci and build:test are 58% of it; the eight verification steps total 56 seconds.

The real lever is the trigger, not the list. Across 300 runs, 241 of them pull requests, this lane produced 12 failures and not one was unique — every pull-request red sat on a commit whose test run had already failed, usually at the identical step:

windows_recovery failing step	CI on the same head SHA
Install dependencies ×6	Install dependencies ×6
Build test artifacts ×4	Lint ×1, superseded/cancelled ×3
Verify Runtime Host owner-death recovery ×1	Run affected standard workspace tests
Verify managed dependency alternate streams ×1	success — but this run was push: main, not a pull request

The recovery suites are ordinary TypeScript, so test runs them on Linux on every pull request and fails first. The lane's one unique catch in that window was a Windows-only NTFS alternate stream regression, found by the unfiltered main push — the trigger that exists for exactly that. windows_recovery is also deliberately not a required context in .asf.yaml, so it never stood between a regression and main.

Change made instead: the pull-request filter now names only what a Windows runner can prove and test cannot — how npm ci resolves and what the dependency patches and the Electron installer produce there, what the clean step removes there, and scripts/windows-runtime-host-local-ipc-trust.ps1, which has no other caller. Every workspace source directory came out. The install and build steps stay unconditional, because proving those suites still build and run on Windows is what the lane is for once it does run. push: main (unfiltered) and the nightly are unchanged.

118 → 14 of 200 merges, about 230 fewer Windows runs a day. The contract test in scripts/ci-test-plan.test.mjs is inverted rather than deleted: it still derives the executed workspaces from the dist paths the steps run and closes over the workspace dependency graph, but now asserts none of that closure appears in the filter.

Gives up: a Windows-only recovery regression is found minutes after merge by the unfiltered main push, or by the nightly, instead of before merge. That has happened once in 300 runs, and the main push is what found it.

Revised effect

Pull-request runner slots fall from ~2900 to ~1400/day. Windows minutes fall further than the ~47% projected above, since the trigger-rate cut is 88% on this lane rather than the 58% assumed.

[Astro-Han]

Closing: #4461 landed the single-job ci allocation and the cli-package-validation reduction. The two corrections above (14 → 10, and windows-recovery not being worth a derived filter) are the final word on the remaining findings; nothing else here is pending.