From 389a50e70c28014fe3c325873e685c9b4780b395 Mon Sep 17 00:00:00 2001 From: Daedalus Date: Tue, 20 Oct 2026 06:09:30 +0200 Subject: [PATCH] docs(adr-0025): record the watcher's measured tick delivery The first full day of the ingest watcher delivered 4 scheduled runs against 91 expected (4.4%), with gaps of 6.4, 4.2 and 3.0 hours. Counting only from the first tick gives the same rate, so this is steady-state drop and not registration lag. The decision stands: the acceptance bar is same-day detection and a 6.4 hour worst case clears it. What changes is the latency a reader infers from the cron expression, so the measured cadence is now recorded next to it. Also corrects the nominal "96 tickets a day" figure in the blind-half bullet, which is about 24 times the real rate. The dedup pair is still required; the requirement is one ticket per outage, not a tolerable count of them. Co-Authored-By: Daedalus Co-Authored-By: Claude Opus 5 --- ...5-best-effort-ingest-watcher-in-actions.md | 24 ++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/docs/decisions/0025-best-effort-ingest-watcher-in-actions.md b/docs/decisions/0025-best-effort-ingest-watcher-in-actions.md index 9c0d087..5883750 100644 --- a/docs/decisions/0025-best-effort-ingest-watcher-in-actions.md +++ b/docs/decisions/0025-best-effort-ingest-watcher-in-actions.md @@ -85,6 +85,23 @@ bound, and the first person to rely on that is the one holding a silent outage. - **The detection delay is unbounded**, and no amount of tuning the cron changes that. 15 minutes is a ceiling on how often it *could* look. +- **Measured on this watcher, over its first day: 4 ticks against 91 expected, + 4.4%.** The window was 2026-10-19 05:30Z to 2026-10-20 04:06Z, the ticks + landed at 11:07, 17:28, 21:42 and 00:42Z, and the gaps between them were + 6.4 h, 4.2 h and 3.0 h. Registration lag does not explain it: counting only + from the first tick gives 3 of 67, the same 4.5%. So the real cadence of this + watcher is **roughly one look every 5.7 hours**, not every 15 minutes, and + the worst gap observed so far is 6.4 hours. + + This is worse as a ratio than the `cron: "17 * * * *"` sample quoted above + (1 of 9, 11%) and slightly better in absolute terms (one look per 5.7 h + against one per 9 h), which is the only comparison that matters for + detection. **It does not reopen the decision.** The acceptance bar this + watcher was built against is catching a silent outage *the same day*, after + six days of blindness was the original failure; a worst case of 6.4 hours + clears that bar. What it does invalidate is the number a reader infers from + the cron expression, so quote the measured cadence and not `*/15` wherever + this watcher's latency is described. - **Whether it ran at all is a thing to measure, repeatedly.** The one-line check above belongs in any review of this watcher; empty output is the answer, not an error. @@ -121,7 +138,12 @@ bound, and the first person to rely on that is the one holding a silent outage. is three edits, not one — its pager steps hang on `inputs.page`, which `schedule` does not supply, and it does not set the `PC_ALERT_MAX_AGE_H=0` / `PC_STILL_RED_WAKE=0` pair above, so a pager wired up naively would mint a - ticket per tick, 96 a day for one outage. Scheduling it *without* a pager was + ticket per tick. That was first written here as 96 a day for one outage, + which is the nominal figure and about 24 times the real one: at the delivery + rate measured above it is nearer 4 a day. The pair is still required, because + the requirement is one ticket per outage and not a tolerable number of them, + but the case for it rests on the rule rather than on the scare figure. + Scheduling it *without* a pager was considered and rejected: a red run nobody is subscribed to is the appearance of detection, not detection, and this record already warns against exactly that trade one bullet up. So the choice is binary — page from inside the LAN,