Refresh required check status for fork pull requests

.github/scripts/pull-request-dashboard/RATIONALE.md

@@ -26,6 +26,13 @@ the implementation understandable and operationally cheap.
   repository's current open PR list.
 - The dashboard issue is discovered dynamically by title and label, so target
   repositories do not need to store issue numbers in config.
+- Refresh events that carry no pull request number report the head commit
+  instead, and the workflow resolves it to an open pull request. GitHub omits
+  the pull request association from check and status events whose head branch
+  lives in a fork, which is where nearly every contribution comes from, so
+  without this the CI columns of those PRs would only refresh on the hourly
+  backfill. The webhook bridge cannot resolve the commit itself, because its
+  GitHub App is installed only on `shared-workflows`.
 
 ## Workflow Concurrency
 
@@ -180,6 +187,16 @@ the implementation understandable and operationally cheap.
   holds the merge, so it is reported as failing rather than skipped. Tools with
   no such check are not reported, because GitHub expects results only from the
   tool configurations that actually ran.
+- That `NEUTRAL` is only reported as failing once every check at the head has
+  finished, including optional ones. The code scanning app publishes the tool
+  check as `NEUTRAL` before the analysis is uploaded and then replaces it in
+  place, so an analysis still running is reported as pending instead of pinning
+  the PR to its author. The replacement leaves the enclosing check suite
+  untouched, so the dashboard subscribes to code scanning check runs to observe
+  the final result. Check runs from other apps are ignored, because their check
+  suite reports them and one refresh per job would multiply webhook volume.
+- Required checks reported as commit statuses, such as EasyCLA, never belong to
+  a check suite, so commit status events are subscribed as well.
 - A failing required status check routes a human-authored PR to the author
   before discussion and approval routing. The live PR status comment names the
   CI failure, including when review feedback also needs author action.

.github/scripts/pull-request-dashboard/WEBHOOK_SETUP.md

@@ -48,6 +48,7 @@ openssl rand -hex 32
 Repository permissions:
 
 - Checks: read-only
+- Commit statuses: read-only
 - Contents: read-only
 - Issues: read and write
 - Metadata: read-only
@@ -61,7 +62,8 @@ Permission rationale:
 
 | Permission | Access | Why it is needed |
 | ---------- | ------ | ---------------- |
-| Checks | Read | Required to subscribe to check-suite events and to read check data for dashboard rows. |
+| Checks | Read | Required to subscribe to check-suite and check-run events and to read check data for dashboard rows. |
+| Commit statuses | Read | Required to subscribe to commit status events, which are the only notification for checks reported as statuses instead of check runs, and to read those status contexts in the check rollup. |
 | Contents | Read | Reads PR commits and repository metadata needed by pull/commit APIs. |
 | Issues | Read and write | Finds, creates, and updates the dashboard issue. |
 | Metadata | Read | Required by GitHub for GitHub App repository access. |
@@ -76,23 +78,27 @@ a pull request, GitHub governs writing it with the `Pull requests` permission;
 
 Subscribe to events:
 
+- Check run
 - Check suite
 - Pull request
 - Issue comment
 - Pull request review
 - Pull request review comment
 - Pull request review thread
+- Status
 
 Event rationale:
 
 | Event | Why it is needed |
 | ----- | ---------------- |
+| Check run | Refreshes CI status when the code scanning app replaces its placeholder result in place, which does not update the enclosing check suite. Check runs reported by other apps are ignored because their check suite already reports them. |
 | Check suite | Refreshes CI status when checks complete. |
 | Pull request | Refreshes dashboard rows when PR state, draft status, labels, assignees, branches, or metadata change. |
 | Issue comment | Refreshes PR conversation state when PR issue comments are created, edited, or deleted. Events generated by the dashboard App changing its own comments are ignored. |
 | Pull request review | Refreshes approval/change-request state and the live PR status comment. |
 | Pull request review comment | Refreshes inline review-comment discussion state. |
 | Pull request review thread | Refreshes when inline review threads are resolved or unresolved. |
+| Status | Refreshes CI status for required checks reported as commit statuses, such as EasyCLA, which are never part of a check suite. Statuses on the default branch are ignored. |
 
 Create the app, update the logo, and generate a private key.
 
@@ -156,6 +162,7 @@ The webhook bridge should dispatch `pull-request-dashboard.yml` in
 {
   "repository": "opentelemetry-java-instrumentation",
   "pr_number": "12345",
+  "head_sha": "",
   "trigger_event": "pull_request_review_comment"
 }
 ```
@@ -165,6 +172,12 @@ Notes:
 - `repository` is the short repository name under `open-telemetry`, and must
   match a `repositories.json` entry exactly. An owner-prefixed name is rejected.
 - Omit `pr_number` or set it to an empty string for a backfill.
-- The central workflow validates `repository` and `pr_number` before using them.
-  `trigger_event` only selects a concurrency group, so it is validated on the
-  backfill path only; the bridge is what restricts it to known event names.
+- Send `head_sha` instead of `pr_number` when the event carries no pull request
+  number. Check and status events for a pull request whose head branch lives in
+  a fork report no pull request association, so the central workflow resolves
+  the head commit to an open pull request and skips the refresh when there is
+  none.
+- The central workflow validates `repository`, `pr_number` and `head_sha` before
+  using them. `trigger_event` only selects a concurrency group, so it is
+  validated on the backfill path only; the bridge is what restricts it to known
+  event names.

.github/scripts/pull-request-dashboard/github_cli.py

@@ -334,6 +334,7 @@ def fetch_pr_reviews(owner: str, repo_name: str, number: int) -> list[dict[str,
                                                 databaseId
                                             }
                                             workflowRun {
+                                                databaseId
                                                 workflow {
                                                     name
                                                 }
@@ -384,6 +385,7 @@ def normalize_check(node: dict[str, Any]) -> dict[str, Any]:
         "state": state,
         "bucket": check_bucket(state),
         "workflow": workflow.get("name") or "",
+        "workflow_run_id": workflow_run.get("databaseId"),
         "description": node.get("description") or "",
         "link": (node.get("targetUrl") if is_status else node.get("detailsUrl")) or "",
         "started_at": (node.get("createdAt") if is_status else node.get("startedAt")) or "",
@@ -404,16 +406,29 @@ def check_attempt_order(check: dict[str, Any]) -> tuple[int, int | str]:
     return 1, check["check_run_id"] or 0
 
 
+def check_identity(check: dict[str, Any]) -> tuple[str, str, int | None]:
+    # Separate workflows can name a job identically, so the workflow keeps their
+    # checks apart while rerun attempts of one check still collapse.
+    return check["workflow"], check["name"], check["integration_id"]
+
+
+def check_execution_identity(check: dict[str, Any]) -> tuple[int | None, str, int | None]:
+    # Two runs of one workflow can be in flight for the same commit, so a check
+    # that finished in one run must not hide the one still running in another.
+    return check["workflow_run_id"], check["name"], check["integration_id"]
+
+
 def gh_pr_check_rollup(
     repo: str,
     pr_id: str,
     non_blocking_check_patterns: list[str],
 ) -> dict[str, list[dict[str, Any]]] | None:
     del repo
     checks_by_identity: dict[
-        tuple[str, int | None, bool], tuple[dict[str, Any], bool]
+        tuple[str, str, int | None, bool], tuple[dict[str, Any], bool]
     ] = {}
-    code_scanning_by_identity: dict[tuple[str, int | None], dict[str, Any]] = {}
+    latest_by_execution: dict[tuple[int | None, str, int | None], dict[str, Any]] = {}
+    code_scanning_executions: set[tuple[int | None, str, int | None]] = set()
     after: str | None = None
     try:
         while True:
@@ -428,26 +443,25 @@ def gh_pr_check_rollup(
                 is_required = bool(node.get("isRequired"))
                 name = (node.get("context") or node.get("name") or "")
                 app = ((node.get("checkSuite") or {}).get("app") or {})
+                check = normalize_check(node)
+                identity = check_identity(check)
+                execution = check_execution_identity(check)
+                latest_attempt = latest_by_execution.get(execution)
+                if latest_attempt is None or check_attempt_order(
+                    check
+                ) >= check_attempt_order(latest_attempt):
+                    latest_by_execution[execution] = check
                 if app.get("databaseId") == CODE_SCANNING_APP_ID:
-                    check = normalize_check(node)
-                    code_scanning_identity = (check["name"], check["integration_id"])
-                    previous_attempt = code_scanning_by_identity.get(
-                        code_scanning_identity
-                    )
-                    if previous_attempt is None or check_attempt_order(
-                        check
-                    ) >= check_attempt_order(previous_attempt):
-                        code_scanning_by_identity[code_scanning_identity] = check
+                    code_scanning_executions.add(execution)
                 if not is_required and not any(
                     fnmatchcase(name, pattern)
                     for pattern in non_blocking_check_patterns
                 ):
                     continue
-                check = normalize_check(node)
-                identity = (check["name"], check["integration_id"], is_required)
-                previous = checks_by_identity.get(identity)
+                required_identity = (*identity, is_required)
+                previous = checks_by_identity.get(required_identity)
                 if previous is None or check_attempt_order(check) >= check_attempt_order(previous[0]):
-                    checks_by_identity[identity] = (check, is_required)
+                    checks_by_identity[required_identity] = (check, is_required)
             page_info = contexts.get("pageInfo") or {}
             if not page_info.get("hasNextPage"):
                 break
@@ -464,7 +478,19 @@ def gh_pr_check_rollup(
             for check, is_required in checks
             if not is_required and check.get("bucket") in ("fail", "cancel")
         ],
-        "code_scanning": list(code_scanning_by_identity.values()),
+        "code_scanning": [
+            check
+            for execution, check in latest_by_execution.items()
+            if execution in code_scanning_executions
+        ],
+        # Every check at the head, not just the required ones, because the
+        # optional job that uploads a code scanning analysis is what decides
+        # whether an undetermined result is final.
+        "pending": [
+            check
+            for check in latest_by_execution.values()
+            if check["bucket"] == "pending"
+        ],
     }
 
 
@@ -522,17 +548,24 @@ def code_scanning_tools(rules: list[dict[str, Any]] | None) -> list[str]:
 def required_code_scanning_checks(
     code_scanning_checks: list[dict[str, Any]],
     tools: list[str],
+    checks_still_running: bool,
 ) -> list[dict[str, Any]]:
     # A code_scanning ruleset rule holds the merge on a check named after the
     # tool, but GitHub never reports that check as required. NEUTRAL there means
     # the alerts introduced by the PR could not be determined, which also holds
-    # the merge, so it cannot be treated as a skip.
+    # the merge, so it cannot be treated as a skip. While other checks are still
+    # running that NEUTRAL is usually a placeholder for an analysis that has not
+    # been uploaded yet, and code scanning replaces it in place without any
+    # event that would refresh a stale failure.
     required: list[dict[str, Any]] = []
     for check in code_scanning_checks:
         if check["name"] not in tools:
             continue
         if check["bucket"] == "skipping":
-            check = {**check, "bucket": "fail"}
+            check = {
+                **check,
+                "bucket": "pending" if checks_still_running else "fail",
+            }
         required.append(check)
     return required
 
@@ -900,6 +933,7 @@ def fetch_pr_routing_raw(
                 required_code_scanning_checks(
                     check_rollup["code_scanning"],
                     code_scanning_tools(branch_rules),
+                    bool(check_rollup["pending"]),
                 ),
             )
         return {