From 51d9b81a37acbd260cf8f1ab4297f41ca644d84f Mon Sep 17 00:00:00 2001 From: Manuel de Brito Fontes Date: Thu, 12 Nov 2026 05:18:55 -0300 Subject: [PATCH] runbooks: the database pool's size is the control plane's document's spin#1147 (spin#1053) made the pool database.max_conns / min_conns in the control plane's bootstrap document, 10 and 2 by default and at most 20. The runbook said 10, fixed in the binary. Co-Authored-By: Claude Opus 5.5 --- runbooks/database-slow.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/runbooks/database-slow.md b/runbooks/database-slow.md index 4662bec..85dcced 100644 --- a/runbooks/database-slow.md +++ b/runbooks/database-slow.md @@ -33,7 +33,8 @@ calls time out and fail ([api-errors](api-errors.md)). - *Statement latency by operation* and *Statements a second by operation*: is one kind of statement slow, or are there simply many more? - *The pool over time* and *Waiting for a connection*: how full, and how long callers wait. The - pool holds at most 10 connections in this release (*Pool maximum*). + pool holds at most `database.max_conns` connections (*Pool maximum*): 10 unless the control + plane's document says otherwise, and never more than 20. 2. Find the slow callers in [Admin → Traces](/admin/traces): their spans carry each statement and how long it took. 3. On the control plane's machine, check the database itself: @@ -60,13 +61,16 @@ calls time out and fail ([api-errors](api-errors.md)). [database-restore](database-restore.md) before replacing it: the database moves by handover. - **Long-held connections** with few statements: restart the control plane, which closes its pool: `systemctl restart spin-controlplane.service`. Brief: the API is down while it starts. - -TODO: the pool's size (10) is fixed in the binary; there is no setting to change it ([spin#1053](https://github.com/spin-stack/spin/issues/1053)) +- **The pool is full with the machine idle** (callers waiting, CPU and I/O low): raise + `database.max_conns` (and `database.min_conns` if it starts cold) in the control plane's + document, which the installation's Terraform writes, then restart the control plane. Above 20 + it refuses to start: the machine's PostgreSQL has 30 connections, and the rest are its one-shot + commands', its backup's and an operator's. ## Verify On [Admin → Metrics](/admin/metrics?dashboard=spin-db-performance), connections in use drop below -8 of 10 and statement p99 below 0.5 s. Both alerts resolve. +four fifths of the pool maximum and statement p99 below 0.5 s. Both alerts resolve. ## Escalate