From 2b4091a62bb39c357c634cdf7b81cb73ad40252b Mon Sep 17 00:00:00 2001 From: xnoto Date: Thu, 13 Aug 2026 19:47:51 -0600 Subject: [PATCH] docs: clarify ARC stale runner recovery --- AGENTS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 41b685d..9244030 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -180,7 +180,7 @@ pre-commit run --all-files 4. **Tor secret format** — Use `data` with raw binary base64; `stringData` double-encodes. 5. **KSOPS needs the age key in the repo-server pod** — Without `sops-age-keys` mounted, manifest generation fails before any sync. 6. **DNS-01 requires external resolvers** — cluster DNS cannot validate Let's Encrypt challenges; the cert-manager controller args above are required. -7. **ARC upgrades can leave a stale listener** — because pruning is disabled for controller-generated listener resources, a chart upgrade can leave a listener referencing a deleted `EphemeralRunnerSet`. The listener then restarts with `ephemeralrunnersets.actions.github.com "" not found`, and `arc-tf` jobs remain queued. Inspect the listener pod's owner and current ARC custom resources before deleting the stale `AutoscalingListener`; deleting only its pod recreates the same broken listener. +7. **ARC upgrades can leave stale runner objects** — because pruning is disabled for controller-generated listener resources, a chart upgrade can leave a listener referencing a deleted `EphemeralRunnerSet`. The listener then restarts with `ephemeralrunnersets.actions.github.com "" not found`, and `arc-tf` jobs remain queued. Inspect the listener pod's owner and current ARC custom resources before cleanup. During the 0.14.2 upgrade, the stale `EphemeralRunner` object had to be deleted manually so ARC could redeploy its objects; restarting only the listener pod was insufficient. ## Useful Commands