Skip to content

Add an option to omit the branch label from the workflow metrics - #313

Open
danielheimburg wants to merge 1 commit into
cpanato:mainfrom
danielheimburg:omit-branch-label
Open

danielheimburg wants to merge 1 commit into
cpanato:mainfrom
danielheimburg:omit-branch-label

Conversation

@danielheimburg

Copy link
Copy Markdown

Every series of the webhook metrics carries a branch label, and the exporter keeps a series until it restarts. In an organization with many pull requests and a merge queue (gh-readonly-queue/main/pr-N-<sha> refs), each branch adds a new set of series, so the series count only grows. Dropping the label in Prometheus with labeldrop does not work, because series that differ only by branch then collide.

This adds --metrics.omit-branch-label (env GITHUB_METRICS_OMIT_BRANCH), off by default. When set, workflow_job_duration_seconds, workflow_job_duration_seconds_total, workflow_job_status_count, workflow_execution_time_seconds and workflow_status_count are registered without the branch label, and all branches of a workflow count towards the same series.

Changes:

  • PrometheusObserver now owns the five metrics and builds them with or without branch (NewPrometheusObserver). A second observer on the same registry reuses the registered metrics, so creating the server more than once (as the server tests do) keeps working.
  • README: the new flag in the flags table and a note in the metrics section.
  • Tests for both modes.

go test -race ./... and golangci-lint run (v2.14) pass locally. The Helm chart is unchanged, its container args are fixed in the template.

Every branch adds its own series to the workflow metrics and they are
never removed, so pull request and merge queue branches grow the series
count without bound. --metrics.omit-branch-label (GITHUB_METRICS_OMIT_BRANCH)
registers the five webhook metrics without the branch label. It is off
by default.

Signed-off-by: Daniel Heimburg <5831617+danielheimburg@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant