From a25d32f70b60af6f84a2fa3fac58a093f6e73633 Mon Sep 17 00:00:00 2001 From: Erik Darling <2136037+erikdarlingdata@users.noreply.github.com> Date: Wed, 21 Oct 2026 08:25:38 -0400 Subject: [PATCH 01/17] Release 3.11.0: version, changelog entries, archive split --- CHANGELOG.md | 607 +++++++++++++++++- Directory.Build.props | 2 +- docs/changelog/3.11.md | 948 +++++++++++++++++++++++++++++ tools/changelog/archive-census.txt | 1 + 4 files changed, 1543 insertions(+), 15 deletions(-) create mode 100644 docs/changelog/3.11.md diff --git a/CHANGELOG.md b/CHANGELOG.md index e7eb833893..88c9f20538 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,25 +21,357 @@ Releases before 3.0.0 are not archived: those entries carry no prose to move. ## [Unreleased] +## [3.11.0] - 2026-10-22 + +Full entries: [docs/changelog/3.11.md](docs/changelog/3.11.md) + +### Important + +- **Upgrading from 3.10 runs the Darling store migrations V174 through V210 on the service's first start** +- **Upgrading Lite from 3.10 moves its DuckDB schema from version 67 to 77 on the first start** +- **Self-managed stores: re-run `provision-roles.sql` after upgrading** ([#6066]) + ### Added -- **Investigations panel in the Darling viewer and Lite** ([#5907]) - the Darling web viewer gets an Investigations page, and Lite gets an Investigations window (sidebar button), both read-only: agent investigation requests by state and family (opened, server, metric, family, agent, lease, reported, route) and, for a picked request, its reports, stored brief and journal. In Lite the reports, brief and journal text are masked at the level the MCP investigation tools serve. -- **Agent notes can be approved or rejected in the Darling web dashboard and in Lite** ([#5908]) - A note an agent proposes with its investigation report waited as pending with no way to approve it, so no brief ever carried one. The Darling web dashboard's Admin page has a Pending Notes tab (server, metric, family, the proposed text, the agent and the request) with Approve and Reject, and Lite has an Agent Notes screen under Settings (Review Agent Notes) over its local database. An approved note goes into later briefs; a rejected note is kept with who decided and when, and never reaches a brief. Each decision is stamped with the person and the time and journaled. On Darling, managed and compose stores connect the web dashboard as the viewer role, which can list notes but cannot decide them, so there the page says to decide in the desktop Viewer or to set postgres.webConnectionString to an admin login. Lite stays advise-only for agents: a person decides notes, and the note text can't be edited. -- **A new Darling MCP tool, `backtest_custom_alert_rule`, shows where a custom alert rule would have fired and cleared over stored history** ([#5919]) - there was no way to see how often a rule would fire before saving or changing it: `test_custom_alert_rule` reads only the current value and skips the breach and clear samples. The new tool replays a saved rule or a draft definition over up to 7 days (default 24 hours) on up to 10 servers. At each step it reads the rule's own panel as of that step and applies the same breach and clear samples the live engine applies, so the fire and clear points match what the live engine would do on the same samples. It writes and sends nothing. One call runs at most 2,880 queries and 60 seconds, and a call a cap stops answers `partial` with `stopped_at` and what it covered. A server with no stored samples answers `unavailable`, never 0 fires. The MCP catalog grows from 187 to 188 tools. -- **Store schema for Level 2 agent investigations** ([#5920]) - the store gains the tables and columns Level 2 needs, in one migration (store schema version 191). A notification route may now be set to agent level 2 (0 to 2) and an investigation request may carry autonomy level 2; any level outside the range is still refused. `config_service` gets an `agent_actions_enabled` switch that reads as on on a new and an upgraded store. A new `config_threshold_overrides` table holds per-server, per-database threshold overrides, is deleted with its server and bumps the `config_version` reload beacon on every change. A new `investigation_feedback` table holds a person's verdict (right, partly or wrong), corrected verdict, missed step and comment on a report, one per request per author, and goes with its request. Investigation requests gain a nullable `practice_of` column with no foreign key. The admin role can write both new tables, the viewer role can only read them, and the mcp role can write the overrides table and read feedback, in both the managed roles and `provision-roles.sql` (a bring-your-own store needs the script re-run after upgrading). Retention needs no new pass: feedback ages with its request and overrides are never aged. Nothing reads or writes the new tables yet; later Phase B slices do. -- **The alert engine can take per-server and per-database threshold overrides** ([#5921]) - the engine read every alert threshold as one global value, so a threshold could not be tuned for one server. Each tunable integer threshold now has a registered key and a scope (server, or server and database), and the engine reads every one of them through an optional overrides object. The version store (PVS) percent and floor and the file growth rise and level bars can also take a per-database value, because those alerts grade each database on its own. Nothing supplies overrides yet: Lite passes none and Darling passes none until a later release adds the store-backed overrides, so every alert behaves exactly as before. -- **Investigation reports can record Jira, pull request and bug report links** ([#5922]) - An agent's report may carry up to 10 links, which must be https with no user name or password (http, file and UNC links are refused, and are checked again whenever a link is shown). Later briefs for the same server and alert family list the last 30 days of links, and the viewer and web dashboard show a report's links. -- **The Level 2 agent action gate and its journal** ([#5925]) - Darling now has the one gate every Level 2 agent change will go through: a live claim on the request, the request's server only, a reason and evidence, with each refusal journaled at most once an hour and each change journaled before and after with its evidence masked at `mcp.dataDetail`. Requests now record the lowest agent level among their routes, and an alert from a custom rule an agent last wrote stays at Level 1. -- **People can give feedback on investigation reports** ([#5926]) - A report can be marked right, partly right or wrong, with a corrected verdict, the step the agent missed and a comment, from the Viewer's Investigations window (Report Feedback tab, admin login) or the web Admin page's new Report Feedback tab (recorded only under an admin store login; list-only otherwise). `get_investigation_reports` now returns each report's feedback and, per alert family, how many investigations ended as false positives before and after people's corrections. -- **Lite: Level 1 agent investigations** ([#5820]) - when an alert fires on a notification route whose agent level is 1, Lite opens an investigation request for it, and when `mcp_allow_investigations` is on with MCP writes off it serves four tools (`list_investigation_requests`, `claim_investigation_request`, `submit_investigation_report`, `get_investigation_reports`) so a local agent can claim the request, read its brief, and file a report that Lite keeps beside the alert. Requests, reports, notes and the journal live in Lite's own DuckDB file; nothing writes to a monitored server; with both MCP settings on the investigation tools stay off and Lite logs one Critical line saying so. -- **Deadlock graph window: a visible Copy button for the SQL text** ([#5663]) - the only ways to copy a process's SQL text were the right-click "Copy SQL Text" menu item or selecting the text box and pressing Ctrl+C, and the menu was easy to miss. The Properties panel now has a Copy button on the same row as the "SQL Text" heading, in Lite and the Darling viewer. It copies the same full SQL text as the menu item, through the same code, and is only shown when the process has SQL text. +- **Investigations panel in the Darling viewer and Lite** ([#5907]) +- **Agent notes can be approved or rejected in the Darling web dashboard and in Lite** ([#5908]) +- **A new Darling MCP tool, `backtest_custom_alert_rule`, shows where a custom alert rule would have fired and cleared over stored history** ([#5919]) +- **Store schema for Level 2 agent investigations** ([#5920]) +- **The alert engine can take per-server and per-database threshold overrides** ([#5921]) +- **Investigation reports can record Jira, pull request and bug report links** ([#5922]) +- **The Level 2 agent action gate and its journal** ([#5925]) +- **People can give feedback on investigation reports** ([#5926]) +- **Lite: Level 1 agent investigations** ([#5820]) +- **Deadlock graph window: a visible Copy button for the SQL text** ([#5663]) +- **A Runnable Queue alert warns when runnable tasks per scheduler stay high** ([#5641]) +- **Mute rules can match by client application** ([#5714]) +- **Other services can raise alerts through Darling** ([#5726]) +- **Enter opens a server tab** ([#5764]) +- **Darling records the collection slots it stepped over** ([#5741]) +- **Darling warns at start when the store was migrated by a newer build** ([#5773]) +- **A notification route can be limited to one server or to a tag, in Lite and Darling** ([#5769]) +- **Read latency capacity fields** ([#5792]) +- **Read caller** ([#5795]) +- **A store sizing guide for Darling, and a load-test harness for the Query Store tables** ([#5800]) +- **Darling limits how many long reads run at once** ([#5806]) +- **Darling can hand an alert to your own AI agent** ([#5632]) +- **Alert Details window: resizable, selectable values, Copy all details, and a link to the full chain** ([#5854]) +- **MCP long-poll: an agent waits for an investigation request to open for its routes** ([#5856]) +- **Alert resolutions go to the channels that got the fire, opt-in per channel** ([#5855]) +- **Generic webhook: {{investigation_json}} carries the alert's investigation request** ([#5853]) +- **Lite: Level 1 agent investigations** ([#5902]) +- **Practice requests with a score per agent** ([#5937]) +- **Agents at Level 2 can change mute rules and custom alert rules, journaled** ([#5934]) +- **Per-server alert threshold overrides** ([#5933]) +- **Agent changes list with one-click undo and an off switch** ([#5936]) +- **CI lints the workflow files with actionlint** ([#5947]) +- **Test code can no longer call `Thread.Sleep`, and CI failures are counted with the skip marker apart** ([#5943]) +- **CI can set aside a test that dev's own latest push broke, for up to 72 hours, on pull requests that do not reach it** ([#5942]) +- **Run all collectors now, or one collector now, for a server** ([#5941]) +- **Client Timeouts measurement scripts** ([#5736]) +- **Query tags: shared rule matcher** ([#5979]) +- **Query tags: store tables** ([#5977]) +- **Alert recurrence in investigation briefs** ([#6000]) +- **Query tag sightings from the collectors** ([#5980]) +- **Query tag rule MCP tools** ([#5982]) +- **Investigation evidence, follow-ups and signed-webhook keys in the store** ([#5997]) +- **Lite has a query-tag store** ([#5989]) +- **Tags on the Darling query reads** ([#5984]) +- **Query tag MCP tools** ([#5981]) +- **Investigation reports carry a family, fix links, evidence basis and the queries run** ([#6004]) +- **Query tag map built by the collector** ([#5985]) +- **Investigation evidence is frozen at the open** ([#6001]) +- **Lite now records which queries it saw and builds the query tag map** ([#5990]) +- **Resource use by query tag** ([#5983]) +- **Signed webhooks** ([#5998]) +- **Lite can roll resource use up by query tag** ([#5992]) +- **Investigation evidence now covers deadlocks, blocking and plan regressions** ([#6031]) +- **Lite MCP tools to manage query tags, assignments and rules** ([#5991]) +- **Lite can manage query tags and see resource use by tag** ([#5993]) +- **Investigation request events reach the agent and its people** ([#6032]) +- **Agents can read an investigation's frozen evidence** ([#6033]) +- **Notification routes can now set a fallback route** ([#6036]) +- **The Darling viewer shows query tags: a Tag column and filter on the Queries grids, and a by-tag resource view** ([#5988]) +- **Web dashboard: a Query Tag Usage page, and a tag column and filter on the query grids** ([#5987]) +- **The web dashboard can manage query tags, their hand assignments and their rules** ([#5986]) +- **Investigation briefs hint at a plan change behind high CPU, and the reverse** ([#6007]) +- **Investigation reports can schedule a follow-up request** ([#6008]) +- **Lite investigations freeze evidence sections per request** ([#6010]) +- **Deadlocks get a signature, a daily rollup, an agent gate and a 4-hour digest** ([#6005]) +- **Investigation follow-ups can wait for a module deploy** ([#6009]) +- **Lite investigation requests journal a quiet join and wait for every metric to resolve** ([#6010]) +- **An agent can merge two deadlock signatures, journaled and undoable** ([#6006]) +- **Lite investigations carry server names, time-of-day recurrence and the new report fields** ([#6011]) +- **Lite opens time follow-ups and signs generic webhooks** ([#6012]) +- **An MCP read checks a Query Store plan before anyone forces it** ([#5632]) +- **Darling collects each database's forced plans** ([#5632]) +- **Four small MCP read fixes for agents** ([#6074]) +- **Darling ranks queries by aborted and failed executions** ([#6073]) +- **Level 3 routes, and a per-server switch for each server action, all off** ([#5632]) +- **Darling says whether each forced plan works** ([#5632]) +- **The server write code for agent actions, in its own assembly and not called yet** ([#5632]) +- **Darling says who drove the load on a server** ([#6072]) +- **Agents can force and unforce a Query Store plan on SQL Server** ([#5632]) +- **Darling checks every plan an agent forces, and asks the agent to keep or remove it** ([#5632]) +- **A person can undo an agent's plan force or unforce from the Agent changes list** ([#5632]) +- **An agent can evict one cached plan, then see its next compile** ([#5632]) +- **The blocking capture records what a person needs to judge a head blocker** ([#5632]) +- **An agent can kill one SQL Server session, after checking it is still the session it named** ([#5632]) +- **Volume Free Space alert: a fleet-wide excluded volumes list** ([#6098]) +- **Sibling server tags can be put in order with Move Up and Move Down** ([#6111]) +- **Lite answers "which database, which caller, which statement piled up" the way Darling does** ([#6124]) +- **The Blocking grids show the head blocker's login time, last request end, open transactions and log used** ([#6146]) +- **An agent can end one PostgreSQL session with `kill_session`, after Darling checks it is still the session the agent named** ([#6126]) +- **kill_session works on Azure SQL Database** ([#6184]) +- **Plan changes an agent makes now say how to undo them** ([#6207]) +- **Custom views and notebooks can sit in shared folders and be personal** ([#6214]) +- **Custom alert rules can expire** ([#6235]) +- **Export the investigation cases that have feedback** ([#6236]) +- **Darling MCP can now stop, start and reschedule a collector** ([#6227]) +- **A recorded change opens a watch window, and an investigation opens only if something moved** ([#6234]) +- **Darling MCP can pause collection, disable a server and change its excluded databases** ([#6230]) +- **An agent with MCP write access can test a server connection, run an analysis pass, start a purge and undo a plan change** ([#6230]) +- **Quiet windows: no alerts for a server or the fleet for a set time** ([#6231]) +- **An MCP client can create and edit notification routes and dismiss alert history** ([#6231]) +- **MCP route tools set a route's own destinations** ([#6241]) +- **The web dashboard can close an investigation request, write an agent note and start a practice request** ([#6265]) +- **The web dashboard can create, edit and delete agent routes and edit environment profiles** ([#6264]) +- **A person with the admin role, or an MCP client under `mcp.allowServerAdmin`, can turn a server's agent action switches on and off** ([#6275]) + +### Changed + +- **Plan analysis and exports are bounded, spreadsheet-safe and warn on unencrypted mail** ([#5699]) +- **darling.json and the store's admin and viewer credential files are limited to a local operator group** ([#5700]) +- **A token in the web dashboard's address no longer signs anyone in; the dashboard and MCP host take the access token only from the sign-in form or bearer header, with a minimum length and a wait after failures** ([#5702]) +- **harden-files and the install scripts cover TLS key files** ([#5700]) +- **get_collection_health partial rows leave out null, false and 0 keys** ([#5748]) +- **MCP tool descriptions and server instructions use plain wording** ([#5779]) +- **CI selects every test class that names a symbol a change defines or edits, repo-walking tests run in the guard set, and a local mode prints the filter CI will use** ([#5801]) +- **The release signature guard checks the Darling PowerShell scripts** ([#5799]) +- **Lane orders and review traps ask for behavior tests, not pins** ([#5835]) +- **Guard-stage spelling tests deleted, slice 1** ([#5840]) +- **Guard-stage spelling tests deleted, slice 2** ([#5842]) +- **Guard-stage spelling tests deleted, slice 3** ([#5839]) +- **Guard-stage spelling tests deleted, slice 4** ([#5837]) +- **Guard-stage spelling tests deleted, slice 5** ([#5841]) +- **Guard-stage spelling tests deleted, slice 6** ([#5838]) +- **Guard-stage spelling tests deleted, follow-through** ([#5848]) +- **Settings window: tabs grouped by task, in both the Darling Viewer and Lite** ([#5852]) +- **The hourly store self-metrics sizing reads every hypertable in one catalog pass** ([#5885]) +- **The repository's top page is now a short README with a quick start and links** ([#5869]) +- **The pre-push gate no longer runs Guard classes twice or selects tests for a docs-only diff** ([#5895]) +- **The investigation contracts, tool rules, brief composer and dispatch core now live in the shared libraries** ([#5900]) +- **The hourly store self-metrics sweep sizes hypertables from the compression catalog** ([#5901]) +- **Two bookkeeping tests no longer fail a pull request that adds a store rung, and the MCP instructions budget states its rule** ([#5944]) +- **CI derives the Lite-reads and Darling-reads test sets from the source** ([#5961]) +- **CI now lints the shell scripts in the workflow files** ([#5970]) +- **Tag rules shared** ([#5978]) +- **Test selection for shared projects** ([#5951]) +- **Server names in investigations** ([#5999]) +- **Review prompt asks for behavior coverage, not pins** ([#5836]) +- **The force pre-check blocks only a force that cannot work** ([#5632]) +- **Alerts keep joining an investigation request while they keep firing** ([#5632]) +- **A login or host keeps one pseudonym across metadataOnly answers** ([#6066]) +- **A PostgreSQL server's tabs read plainly** ([#6151]) +- **Web page text reads plainly** ([#6172]) +- **The deadlock graph's detail pane shows each process's sql_handle** ([#6209]) +- **Starter agent file: use a small model for the first step** ([#6237]) +- **The Query Store regressions read shows Regular, Aborted (timeouts) and Exception (errors) runs side by side, and flags a rise in timeouts or errors as well as cost** ([#5506]) +- **The Query Store collector fetches text and plans only for each database-hour's top N plans, N a setting (default 100)** ([#6211]) ### Fixed -- **The Darling viewer's Manage Servers window now shows the service's version, once, instead of its own version on every server row** ([#5668]) - the grid's Installed Version column filled every row with the viewer's own version, so an old viewer talking to a newer service read as up to date. The service now writes its product version and process start time to the store when it starts (a new table, `config.service_version`, one row per service host, store schema V174, created by the first start after the upgrade), and Manage Servers shows it once above the grid beside the viewer's own version: one line per service host when more than one service writes to the store. A store with no row yet, or without the table, reads `unknown (older than 3.11)`. A write that fails at start logs a warning, and the service tries it again on every sweep until it succeeds. The Installed Version column is removed. The line is display only and shows no warning. -- **A collector that recovered no longer stays at WARNING** ([#5665]) - the 7-day failure rate gave WARNING above 20% errors and a later success never cleared it, so a new target with 2 errors in its first 8 runs stayed WARNING until its tenth run. The rate now counts only once the window holds 10 runs, or fewer when a collector runs so rarely that a full 7-day window holds fewer (7 for a daily collector, so a daily collector that failed 6 days of 7 and succeeded today stays WARNING), and a collector whose newest 3 runs in a row succeeded is HEALTHY with a note "recovered: the last 3 runs succeeded; last failure