Repository navigation
Conversation
3 tasks
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The notification watch list omits the new workflow lint, leaving its merge-queue failures unreported.
1 open issue
What changed in this PR
Improves GitHub Actions observability with Slack failure alerts and explicit job timeouts.
Changes:
- Adds reusable Slack notifications for watched workflows failing on
mainor in the merge queue. - Sets job timeouts across existing workflows.
- Enforces timeouts through shared workflow linting and enables that lint locally.
| File | Description |
|---|---|
.github/workflows/update-provider-locks.yml |
Adds a 10-minute timeout. |
.github/workflows/trello-poster.yml |
Adds a 10-minute timeout. |
.github/workflows/terraform-ci.yml |
Adds 20-minute job timeouts. |
.github/workflows/team-review-requests.yml |
Adds a 10-minute timeout. |
.github/workflows/shell-ci.yml |
Adds a 10-minute timeout. |
.github/workflows/ruby-ci.yml |
Adds 15-minute job timeouts. |
.github/workflows/reusable-review_apps_on_pr_close.yml |
Limits cleanup to 20 minutes. |
.github/workflows/reusable-review_apps_on_pr_change.yml |
Limits updates to 30 minutes. |
.github/workflows/reusable-notify-failure.yml |
Adds reusable Slack message delivery. |
.github/workflows/reusable-lint-workflows.yml |
Enforces job timeouts and adds its own. |
.github/workflows/reusable-build-image.yml |
Limits builds to 30 minutes. |
.github/workflows/require-pinned-github-actions.yml |
Adds a 10-minute timeout. |
.github/workflows/pre-commit.yml |
Adds 10-minute job timeouts. |
.github/workflows/pipeline-visualiser.yml |
Adds 20-minute job timeouts. |
.github/workflows/notify-failure.yml |
Watches failures and supports test notifications. |
.github/workflows/lint-workflows.yml |
Runs shared lint for pull requests and merge groups. |
.github/workflows/guardrail_unique_job_names.yml |
Adds a 10-minute timeout. |
.github/workflows/guardrail_tf_version_upgrade.yml |
Adds a 10-minute timeout. |
.github/workflows/guardrail_matching_pentester_email_addresses.yml |
Adds a 10-minute timeout. |
.github/workflows/guardrail_matching_pentester_cidr_blocks.yml |
Adds a 10-minute timeout. |
.github/workflows/forms-cli.yml |
Adds a 15-minute timeout. |
Review effort: Balanced
whi-tw
force-pushed
the
observe-workflow-failures
branch
from
October 20, 2026 12:48
2ab87d6 to
75725af
Compare
whi-tw
force-pushed
the
observe-workflow-failures
branch
from
October 20, 2026 12:56
75725af to
07b3c9b
Compare
whi-tw
force-pushed
the
observe-workflow-failures
branch
from
October 20, 2026 13:02
07b3c9b to
c3d9974
Compare
Failures on main or in the merge queue happen where nobody is watching the run, so a broken main can go unnoticed. Failures on pull requests are left alone because the author already sees them. The sending logic is a reusable workflow so the other forms-* repos can call it from their own workflow_run trigger.
Without timeout-minutes a hung job runs for up to six hours before GitHub fails it, holding a runner and delaying the failure signal. Limits are deliberately generous (10 to 30 minutes) against p95 durations of roughly two minutes or less, so they only catch hangs.
Adds a check to the shared workflow lint so the other forms-* repos get it too, and calls that lint from this repo, which did not run it. The lint runs in the merge queue, so add it to the failure notifications.
whi-tw
force-pushed
the
observe-workflow-failures
branch
from
October 20, 2026 13:16
c3d9974 to
b5f7b6a
Compare
|
Warning You are changing the Terraform version. Before you merge this PR, you must apply the |
whi-tw
requested review from
theseanything
and
a balanced review from Copilot
October 20, 2026 13:20
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What problem does this pull request solve?
Trello card: https://trello.com/c/jPeqHHPu/772-improve-the-observability-of-our-github-workflows
If a workflow fails on
mainor in the merge queue, nobody is told unless they happen to look. This adds a reusable workflow that posts to#govuk-forms-alertwhen that happens. Failures on pull requests are not reported, because the author already sees them.notify-failure.ymllists the workflows it watches by name, so a new workflow that runs onmainor in the merge queue needs adding to that list.It also sets
timeout-minuteson every job, so a hung job fails fast rather than running for up to six hours, and adds a check to the shared workflow lint that fails if a job has none.forms-deploywas not running that lint on itself, so it now calls it.The limits are deliberately generous (10 to 30 minutes) against p95 durations of roughly two minutes or less.
Before merging
SLACK_WEBHOOK_GOVUK_FORMS_ALERTsecret exists and is available to the forms-* repos.workflow_dispatch) to check a test message arrives.@main, so their workflow changes will fail it until they have timeouts.Things to consider when reviewing
workflow_runapproach, with a list of watched workflows, acceptable compared with adding a notify job to each workflow?