Repository navigation
ci(unit-tests): a KTLO or experimental leg that fails at job level, such as a lost runner, still fails Required checks passed #1738
Description
Describe the bug
A KTLO or experimental leg can still block a PR. Those legs carry only step-level continue-on-error (unit-tests.yml:158, :162, :209, :235, :243). If the job itself fails rather than a step, nothing contains the failure. test then reports failure, and Required checks passed fails.
Reproduction
On #1734 (ba67ef70e, run 38599394443), job 115975614204 Python graalpy-25.4 (experimental) lost its runner 18 minutes into Run unit tests (cap 25):
17:17:32Z ##[error]The runner has received a shutdown signal. ...
17:17:37Z ##[error]The operation was canceled.
Job 115994929719 Required checks passed then printed test: failure and result was 'failure'; accepted here: success, and exited 1. Every non-experimental leg passed.
Expected behavior
A KTLO or experimental leg never fails the required context, whatever ends it: a step failure, a step cap, a job cap or a lost runner. The fix needs a source for how needs.<job>.result treats a matrix leg under job-level continue-on-error.
Additional context
This edits the same file as #1732, so it is blocked until #1732's PR merges. The failed jobs on #1734 have been re-run.
Activity
JarryShaw commentedon Oct 14, 2026
Narrowed. On #1732 the owner chose to move the experimental legs into a job of their own that nothing gating waits on, and that work is in progress. Once it merges, a lost runner on an experimental leg can no longer block. What remains here is the KTLO half: KTLO legs stay in
test, so a job-level failure on one still failsRequired checks passed. This stays blocked until #1732 merges.JarryShaw commentedon Oct 14, 2026
#1742 has merged. The experimental half is fixed: those legs now run in
experimental.yml, which nothing gates on. What remains is the KTLO half. KTLO legs still live intest, whichrequired-checks(unit-tests.yml:1885) andunittest-orderingbothneeds:. A job-level failure on one, such as a lost runner or the job cap, therefore still failsRequired checks passed.needs:, as #1742 didtest, but add job-levelcontinue-on-errorneeds.test.resultthen readssuccess. That needs a source and a test run, and the check would still show redRecommendation: A. It is the same shape as the experimental fix, and KTLO legs are non-blocking by design (#1633).
JarryShaw commentedon Oct 14, 2026
A
JarryShaw commentedon Oct 14, 2026
Recorded: A. The KTLO legs of
test,integrationandpypcap-paritymove into jobs of their own, which neitherrequired-checksnorunittest-orderingneeds:. They keep their check names, their warning-level annotations and their KTLO summaries. A worker is on it.ci(python): move the KTLO legs into jobs nothing gating waits on (#1738…