[AlexeyShalaev]

Closes #54.

Problem

The three pool gauges were sampled from the pool's checkout/checkin events and exported unchanged:

• pool_overflow was QueuePool.overflow(), which starts at -pool_size. A pool with size=10 and one open connection reported -9.
• checkin fires before the connection goes back to the pool, and nothing fires once the pool is idle. So an idle pool kept reporting its last burst (checked_out=2, overflow=2 with every connection returned).
• pool_size is the configured pool_size, but the HELP text and docs called it the current size.

Change

• New optional capability PoolStatsFunctionRecorder.set_pool_stats_functions(pool_size, pool_checked_out, pool_overflow). It takes zero-argument callables and lives next to CheckoutWaitRecorder, detected the same way with isinstance, so PostgresMetricsProtocol is unchanged. PostgresMetrics implements it with Gauge.set_function, which means the gauges are read when Prometheus scrapes. attach_metrics binds the callables to engine.pool, looked up on each read, so the pool that dispose() creates is the one reported. For such a recorder the pool events no longer sample the gauges. They still feed the held-duration histogram.
• pool_overflow is max(pool.overflow(), 0) on both paths.
• pool_size keeps its value, the configured pool_size, and the HELP text, protocol docstring and docs now say so. The utilization PromQL gets a note that it passes 100 % while overflow connections are in use.
• A custom recorder that implements only record_pool_stats keeps the event path (with the clamp). Its docstring and the docs say that it lags one return behind and point at the new capability.

Exported from sqlalchemy_foundation_kit and sqlalchemy_foundation_kit.protocols; docs/agents.md is updated.

Considered and rejected

• Making pool_size report open connections (checkedin() + checkedout()), as the old docs claimed. That would silently change the meaning of a series people already graph. Correcting the text costs nobody anything.
• Fixing staleness on the event path by computing the post-return state in the checkin listener. That depends on SQLAlchemy dispatching checkin before _return_conn and on QueuePool closing a returned connection when the queue is full. It's internal ordering, and reading at scrape time doesn't need it.

Who notices

Dashboards and alerts on postgres_db_pool_overflow (no more negative values) and postgres_db_pool_checked_out (drops to 0 when the pool is idle, instead of holding the last burst). pool_size values don't change, only its HELP text. A custom metrics class doesn't need to change. It can opt in to read-on-scrape by adding set_pool_stats_functions. Release-wise this is a fix: that adds one optional protocol.

Evidence

New tests against the old manager.py, metrics/postgres.py and protocols (swapped in from origin/master): 8 failed. That includes the integration test, which runs the reporter's sequence on PostgreSQL 17 and reads the registry by name.

The reporter's script after the fix:

 1 connections checked out:  size=10.0  checked_out=1.0  overflow=0.0   (open connections: 1)
all returned:                size=10.0  checked_out=0.0  overflow=0.0   (pool.checkedout()=0, open connections: 1)
12 connections checked out:  size=10.0  checked_out=12.0  overflow=2.0   (open connections: 12)
all returned:                size=10.0  checked_out=0.0  overflow=0.0   (pool.checkedout()=0, open connections: 10)

Gate: make check clean; make test 634 passed, coverage 93.87%.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!