Skip to content

[-] fix pooler metrics missing from Prometheus sink - #1605

Merged
pashagolub merged 2 commits into
v7from
fix/pooler-prometheus-1571
Oct 22, 2026
Merged

pashagolub merged 2 commits into
v7from
fix/pooler-prometheus-1571

Conversation

@pashagolub

@pashagolub pashagolub commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1571

Summary

PgBouncer and Pgpool SHOW commands cannot alias columns, so the reaper does it for them.

 DbConnReaper.CollectAndDispatch
   pgx.CollectRows
+  normalizePoolerMeasurements      # pgbouncer / pgpool sources only
+    label column  -> tag_<col> text  (database, user, node_id, hostname, role, status, ...)
+    other column  -> float64          ('true'/'false' -> 1/0, Numeric, numeric text)
+    NULL / unconvertible -> dropped   (debug log)
   dispatchMetricData
 notSupportedMetrics  # pkg/sinks/prometheus.go
   change_events
-  pgbouncer_stats
   pgbouncer_clients   # per-connection cardinality
   pgpool_processes    # per-process cardinality
-  pgpool_stats
  • metrics.yaml: avg_* of pgbouncer_stats and lb_weight, replication_delay of pgpool_stats declared as gauges.
  • grafana/postgres/v13 pooler dashboards read labels as COALESCE(tag_data->>'x', data->>'x'), so rows written before and after the upgrade both render; last_status_change column removed (text, now dropped).

Evidence

  • Before: /metrics has no pgwatch_pgbouncer_stats_* / pgwatch_pgpool_stats_* samples; with the exclusion removed, values are skipped as unsupported datatype: {102 0 false finite true}.
    After:
    normalize(pgbouncer, {database: "pgbouncer", total_query_count: Numeric(1234)})
      == {tag_database: "pgbouncer", total_query_count: 1234.0}
    normalize(pgpool, {status: "up", load_balance_node: "true", last_status_change: "2026-..."})
      == {tag_status: "up", load_balance_node: 1.0}
    AddCacheEntry(pgbouncer_stats, pgpool_stats) cached; pgbouncer_clients, pgpool_processes not
    
    go test ./pkg/reaper/ ./pkg/sinks/ ./pkg/metrics/ passes. Not yet verified against live PgBouncer/Pgpool.

Merge Danger

Door: two-way

Pooler labels now arrive as tag_* source columns, so new Postgres-sink rows keep them in tag_data. v13 dashboards read both layouts, so neither upgrade nor revert breaks them.

Blast Radius: pooler

Only pgbouncer / pgpool sources. v12 dashboards are not updated and show no labels for new rows. Pgpool non-numeric text columns (last_status_change, replication_state, ...) are no longer stored; load_balance_node shows 1/0 instead of true/false.

@coveralls

coveralls commented Oct 16, 2026 •

Copy link
Copy Markdown

Coverage Report for CI Build 38902039285

Warning

No base build found for commit 7725207 on v7.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 89.629%

Details

  • Patch coverage: 5 uncovered changes across 1 file (49 of 54 lines covered, 90.74%).

Uncovered Changes

File Changed Covered %
pkg/reaper/pooler.go 53 48 90.57%
Total (2 files) 54 49 90.74%

Coverage Regressions

Requires a base build to compare against. How to fix this →


Coverage Stats

Coverage Status
Relevant Lines: 5660
Covered Lines: 5073
Line Coverage: 89.63%
Coverage Strength: 0.9 hits per line

💛 - Coveralls

PgBouncer and Pgpool admin consoles cannot alias columns, so label
columns never got the `tag_` prefix and numeric values arrived as
`pgtype.Numeric` or text. The reaper now normalizes rows from these
sources: known label columns become `tag_*` text, everything else
float64; NULLs and unconvertible values are dropped.

`pgbouncer_stats` and `pgpool_stats` are no longer excluded from the
Prometheus sink. `pgbouncer_clients` and `pgpool_processes` stay excluded
because of per-connection label cardinality. v13 Postgres dashboards
read the moved columns from `tag_data`.

Closes #1571
Rows written before the upgrade keep pooler labels in `data`, newer ones
in `tag_data`. `COALESCE` lets the dashboards render both.
@pashagolub
pashagolub force-pushed the fix/pooler-prometheus-1571 branch from f2bbd7e to 11621fa Compare October 16, 2026 12:39
@pashagolub pashagolub self-assigned this Oct 16, 2026
@pashagolub pashagolub added the reaper Reaper related label Oct 16, 2026
@pashagolub
pashagolub requested a review from 0xgouda October 16, 2026 15:35
@pashagolub
pashagolub merged commit 201f822 into v7 Oct 22, 2026
6 checks passed
@pashagolub
pashagolub deleted the fix/pooler-prometheus-1571 branch October 22, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reaper Reaper related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants