Repository navigation
Conversation
A release commit changes only the version's files, but it matches Interop's paths (package.json, package-lock.json), so Interop ran the advisory servers' suites again on code it had just run them on, and, sharing one concurrency group on main, canceled that run if it was still going. Its job is now skipped, and its run has a group of its own, so the run on the code it releases always finishes. Refs: #199
A change met CI up to four times on its way to the site, and every run had the same say: 4.1.1 stayed released but undeployed until the engines' journeys, flaky on the release commit, were re-run by hand three times, although its code had passed in the merge queue and on main already. Each of those runs also spent its slowest jobs, the timing-dependent ones, on code they had already passed. A new gate job (npm run ci:gate) decides what each run runs. A pull request, the merge queue, a run by hand and a push to main that no merge_group run passed (an advisory's merge through the ruleset bypass, say) run every job, and every job blocks. A push the queue passed, and a release commit whose parent CI passed on main, skip the timing-dependent jobs (pod, contract, the journeys in Firefox, WebKit and on a phone) instead of running them again; ci-ok lets them be skipped then and only then. The gate fails a release commit that changes anything but the files release.config.mjs commits. The report the site publishes at /journeys/ keeps every browser's journeys, as the release commit's run takes the engines' from the run that ran them on its code, which the gate names, and the queue's engines keep their traces and blob for it. When that blob has expired, the gate runs every job instead. The caches the skipped jobs saved on main, each server's images and the engines' browsers, are looked up by a warm job and only the missing ones made and saved. Each job in ci.yml is labeled deterministic or timing-dependent, as in docs/testing.md, which also says to count requests rather than time them; docs/releasing.md says what each stage runs. Closes: #199
The Chromium journeys were kept on every push to main and on the release commit, as the deterministic jobs were, although they wait on a real server behind TLS as the other engines' do, and although that tree had just passed them in the merge queue. They are the run's longest job, about 8 minutes, spent twice more on main for every merged pull request. Both journeys jobs are now timing-dependent, and skipped where the gate skips the others. The merge queue keeps both jobs' traces and blobs, and main's report takes both from the run the gate names, which must still keep them. The warm jobs also refill the journeys' server image and Chromium's browsers, which the Chromium job saved on main. A skipped warm-images job no longer shows its name's unexpanded matrix. Refs: #199
The journeys are the run's longest jobs, about 8.5 minutes each, and a pull request and the merge queue wait for them. Each browser's journeys now run in two shards side by side (npx playwright test --shard), so the run waits for about half as long. Each shard pays the job's setup again, about a minute, for the shorter wait. Each shard keeps a blob of its own, which journeys-report merges and requires, and the gate checks that the run it takes them from keeps its journeys-blob-* artifacts, whatever the number of shards. Refs: #199
1 task
14 of 15 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
A change met CI up to four times on its way to the site: its pull request, the merge queue, the push to
main, and the release commit. Every run had the same say, and every run ran the slowest jobs again: the timing-dependent ones (pod,contractand the user journeys). In recent runs they took about 30 of each run's 36–41 job-minutes. 4.1.1 stayed released but not deployed until the engines' journeys, flaky on the release commit, had been re-run by hand three times. Its code had already passed in the merge queue and onmain.This makes each stage's job explicit, as the issue proposes (written down in
docs/releasing.md#what-each-stage-checks):gatejob (npm run ci:gate,scripts/ciGate.ts) decides what each run runs. Every job that runs blocks.mainthat nomerge_grouprun passed, such as an advisory's merge through the ruleset bypass. So a release built on such a commit waits for the full run.mainthat the queue passed, and on a release commit whose parent passed CI onmain. These arepod,contract, and the journeys in every browser. The same code already passed them.ci-okaccepts a skippedpod,contractorjourneysthen, and only then.release.config.mjscommits, or that has more than one parent./journeys/report keeps every browser. The release commit's run takes the journeys' blobs from the run that ran them on its code, which the gate names (tested-by). That is usually the parent's merge queue run, which now keeps traces and blobs for every journeys job. If those blobs have expired (GitHub keeps them 7 days, so a release made late by hand), the gate runs every job instead, and the deploy is never left stuck.journeys (chromium, 1/2)tojourneys (engines, 2/2)). Pull requests and the queue wait about half as long for them. Each shard repeats about a minute of setup.warmjobs keepmain's caches filled. The skipped jobs used to save each server's images, the journeys' server image and the journeys' browsers.warmlooks them up, andwarm-imagesandwarm-browsersmake and save only what is missing. Likecaches, they never fail the run.ci.ymland indocs/testing.md.docs/testing.mdalso adds the rule to count requests rather than time them.The timing-dependent jobs now run twice per merged pull request (in its pull request and in the queue), not four times. A push to
mainthe queue passed now takes about 4 minutes instead of about 9.Some of this can only be confirmed on GitHub: the run passing with skipped jobs, downloading the journeys' blobs from another run, and the cache lookup. Please watch the first push to
mainand the first release after merging. On that first push, every journeys job should be skipped andjourneys-reportshould pull all four blobs from the queue run. No repository settings need to change.Closes: #199
Checklist
Refs:, notCloses:. Leave this unchecked otherwise.npm run checkpasses (lint, Markdown and spelling included),npm run lint:reusetoo if a file was added, andnpm run crosschecktoo ifns/ordecks/changed. (npm run lint:workflowspasses too; crosscheck n/a:ns/anddecks/unchanged)docs/is listed in docs/docs.ttl. (no new page)packages/ui/src/i18n/). (n/a: no message changed)ns/was agreed in an issue first; no publishedns/shapes/*/vN.ttlordecks/*/vN.ttlwas edited; generated files were regenerated, not edited. (n/a: nothing inns/ordecks/changed)!and aBREAKING CHANGE:footer).npm run check:breakingsees a line ofapi/that goes, but not a feature removed, nor a reader that reads less than before.npm run lint:reuse), or a new deck its line inREUSE.toml; third-party content is credited.npm run dev) and tried the change, and considered the user journeys. (n/a: CI only; the journeys' report onmainis covered above)type(scope): subject,!and aBREAKING CHANGE:footer if it breaks something), with an imperative subject and a body that explains why (CONTRIBUTING.md).