Skip to content

fix(budget): refuse to report a percentage the watchdog cannot substantiate - #133

Merged
ExtraToast merged 1 commit into
mainfrom
fix/budget-no-false-allclear
Aug 30, 2026
Merged

fix(budget): refuse to report a percentage the watchdog cannot substantiate#133
ExtraToast merged 1 commit into
mainfrom
fix/budget-no-false-allclear

Conversation

@ExtraToast

Copy link
Copy Markdown
Contributor

What went wrong

My previous change (#132) made the watchdog go green, reporting:

::notice::pricing 0 run(s) from the Actions API
::notice::about 0 of 3000 included minutes used (0%)
::notice::below the 70% warning threshold; nothing to raise

It could read neither billing nor workflow runs, warned about both, and then reported a comfortable zero as a pass.

That is worse than the red it replaced — and worse in exactly the way my own commit message had warned against two commits earlier: "a budget watchdog that quietly reports 0% when it cannot read the budget is worse than one that is absent, because it looks like good news." I wrote that, then shipped the thing it describes. A false all-clear on a budget alarm is the precise failure the alarm exists to prevent.

The org is at 109% of its allowance while this reported 0%.

Fix 1 — refuse to report what cannot be substantiated

  • Any rejected run-listing query → fail, naming the permission needed
  • Zero runs across every private repo in a month → fail. That is not a small number, it is evidence the measurement did not happen

A partial read is no longer averaged into a reassuring percentage.

Fix 2 — best-effort capable token instead of an opt-in flag

create-github-app-token fails the job when asked for a permission the installation lacks — that is why the app-has-billing-permission flag existed. But continue-on-error: true on that one step achieves the same protection with nothing to configure, and an App that does hold the grants is used automatically.

capable mint (continue-on-error)  ->  organization-plan: read + actions: read
minimal mint (always succeeds)    ->  metadata + issues, only to file the alert

Token precedence: capable App token → BUDGET_TOKEN → minimal token. The flag is removed.

Verified against the exact production condition

A shimmed gh that 403s both billing and runs — reproducing what actually happened:

before after
billing 403 + runs 403 0 of 3000 (0%), exit 0 ::error:: naming the grants, exit 1
billing reachable 3268 (109%), exit 1 3268 (109%), exit 1 — unchanged
::error::could not list workflow runs (50 query/queries refused), so the month cannot be priced.
::error::This needs actions: read across the org. Grant the App "Actions" (read) and
         "Organization plan" (read), accept the updated installation permissions,
         or supply BUDGET_TOKEN.

actionlint clean.

Honest status

The watchdog will stay red until the App gets actions: read (for the fallback) or organization-plan: read (for the exact figure), or a BUDGET_TOKEN is supplied. My attempt to remove that owner action did not succeed — it only moved which permission is needed.

Red-and-honest is the correct state for a deployed alarm that cannot see its subject. Tracked in #126.

Refs #126

…ntiate

The previous change made the watchdog go green reporting "0 of 3000 (0%)".
It could read neither billing nor workflow runs, warned about it, and then
reported a comfortable zero as a pass.

That is worse than the red it replaced, and worse in the specific way my own
commit message had warned against two commits earlier: a budget watchdog that
reports no usage because it cannot see usage looks like good news. A false
all-clear on a budget alarm is the failure mode the whole thing exists to
prevent.

Two changes.

A partial or empty read is now refused rather than averaged into a
percentage. If any run-listing query is rejected, or if zero runs are found
across every private repo in a month, the run fails and says what to grant.
Zero runs org-wide is not a small number, it is evidence the measurement did
not happen.

The capable token is now minted best-effort with continue-on-error instead of
behind an opt-in flag. create-github-app-token fails the job when asked for a
permission the installation lacks, which is why the flag existed -- but
allowing that one step to fail achieves the same protection without anyone
having to set anything, and an App that does hold the grants is used
automatically. It asks for organization-plan: read for the exact billing
figure and actions: read for the fallback, and falls back to a minimal token
that is only used to file the alert issue.

Verified by reproducing the production condition -- a token that lists repos
but is refused both billing and runs -- which now exits 1 naming the
permissions, where before it exited 0 claiming 0%.

Refs #126
@ExtraToast
ExtraToast merged commit 010c41c into main Aug 30, 2026
6 checks passed
@ExtraToast
ExtraToast deleted the fix/budget-no-false-allclear branch August 30, 2026 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant