From b5070a1b274d6ce8a6a108ccaaf980857be00ec4 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Tue, 3 Nov 2026 01:29:32 +0800 Subject: [PATCH] Hold the collector off around the presence reader's timed windows The quality run on the 1.2.0 release PR failed twice in a row on Ubuntu 3.14, at the first ready() of test_reader_loss_does_not_block_worker_owned_release_only_ack_cleanup. The reader must answer within 0.25 s. Under coverage and gdb, late in the suite, a whole-heap collection takes longer than that, and the fixed test order puts it in the same place on every run. The fixture now collects first and disables the collector for each test, and always re-enables it. The product deadline is unchanged. --- docs/updates/2026-11.md | 25 +++++++++++++++++++ docs/updates/README.md | 3 ++- .../headless/test_agent_presence_reader.py | 18 ++++++++++--- 3 files changed, 42 insertions(+), 4 deletions(-) diff --git a/docs/updates/2026-11.md b/docs/updates/2026-11.md index 7cc1c0fb..cc9ca94b 100644 --- a/docs/updates/2026-11.md +++ b/docs/updates/2026-11.md @@ -629,3 +629,28 @@ Index and query commands: [README.md](README.md). New entries go at the end. - `uv.lock`: the 3.10-only branches are gone; removed: `exceptiongroup`, `tomli`; one version left instead of a 3.10 one beside it: `dnspython`, `numpy`, `pillow`. - **Also**: The Pillow dependency is a single `pillow==13.0.0`, and the dnspython constraint applies on every interpreter. The Pillow and dnspython Dependabot alerts were open only because of the 3.10 entries. The CI cells that ran 3.10 to cover Windows, Ubuntu and macOS now run 3.11, and mypy targets 3.11. The version is declared as 1.2.0, and CHANGELOG.md says what changed. - **Open items**: none. + +## U-20261103-01 · 2026-11-03 · Hold the collector off around the presence reader's timed windows · #incident #agent #ci #test + +- **Run.** Quality run 41476383837 on the 1.2.0 release PR (#548). Ubuntu 22.04 3.14 failed twice, the + first attempt and its rerun, on the same test and in the same place: + `test_agent_presence_reader.py::test_reader_loss_does_not_block_worker_owned_release_only_ack_cleanup`. + Its first `harness.reader.ready()` raised `physical presence reader deadline exhausted`. In the same + run, the other 17,546 tests passed, and so did the other eight squares. +- **Why only there.** The same commit passed the Ubuntu 3.14 square of Dev CI (run 41476340230). The two + jobs differ only in `coverage run`, and on Linux both run under gdb (`ci_native_diagnostics.py + --native`). `ready()` must finish connect, handshake and reply within 0.25 s. On the 1.1.0 release PR + (#538) the same test passed at 0.455 s, against 0.254 s for its neighbours. +- **Cause (inferred, not observed).** Allocation counts trigger the collector, so a fixed test order + puts a collection in the same place on every run. After 17,000 tests under coverage and gdb, one + collection of the heap takes longer than the 0.25 s window. #546 and #547 shifted the allocation + pattern enough to move a collection into it. The CI log has no direct evidence of a collection. The + reasons are that the failure repeats at one point, that only the slowest configuration fails, and + that the reader's code did not change. +- **Fix.** The `harness` fixture collects once, disables the collector for the test and always + re-enables it, even when `Harness()` raises. The product deadline is unchanged. A collection pause + in production still fails closed and requires an explicit resume. The file's 34 tests run in 8.3 s, + as before. +- **Check.** If the cause is something else, the quality run on #548 will fail again in the same place. + It then needs a gc callback that logs long collections, not a wider deadline. +- **Open items:** none. diff --git a/docs/updates/README.md b/docs/updates/README.md index cbebe358..0f58b8f0 100644 --- a/docs/updates/README.md +++ b/docs/updates/README.md @@ -58,6 +58,7 @@ In the same commit: delete the item from `Progress.md`, add a `#done` entry here | ID | Date | Title | Tags | Batch | |---|---|---|---|---| +| U-20261103-01 | 2026-11-03 | Hold the collector off around the presence reader's timed windows | #incident #agent #ci #test | [2026-11](2026-11.md) | | U-20261102-03 | 2026-11-02 | Require Python 3.11; stop supporting 3.10 | #migration #deps #ci | [2026-11](2026-11.md) | | U-20261102-02 | 2026-11-02 | Clear the SonarCloud findings the 1.1.0 release PR reported | #ci #test #quality | [2026-11](2026-11.md) | | U-20261102-01 | 2026-11-02 | Merge the release line into dev for 1.1.0 | #release #migration | [2026-11](2026-11.md) | @@ -607,7 +608,7 @@ In the same commit: delete the item from `Progress.md`, add a `#done` entry here | File | Period | Entries | |---|---|---:| -| [2026-11.md](2026-11.md) | 2026-11 | 20 | +| [2026-11.md](2026-11.md) | 2026-11 | 21 | | [2026-10-f.md](2026-10-f.md) | 2026-10 | 5 | | [2026-10-e.md](2026-10-e.md) | 2026-10 | 91 | | [2026-10-d.md](2026-10-d.md) | 2026-10 | 15 | diff --git a/test/unit_test/headless/test_agent_presence_reader.py b/test/unit_test/headless/test_agent_presence_reader.py index fd71b292..7967b7a9 100644 --- a/test/unit_test/headless/test_agent_presence_reader.py +++ b/test/unit_test/headless/test_agent_presence_reader.py @@ -1,5 +1,6 @@ """Fresh capability checks and finite reader/watch retirement without native desktop observation.""" from concurrent.futures import ThreadPoolExecutor +import gc import json import threading import time @@ -102,11 +103,22 @@ def close(self): @pytest.fixture def harness(monkeypatch): - value = Harness(monkeypatch) + # The reader must answer each probe within 0.25 s. Late in the suite, under coverage and gdb, one + # collection of the whole heap takes longer than that, and a fixed test order makes it land in the + # same window on every run: ready() failed twice in a row on the 1.2.0 release PR. Collect first and + # hold the collector off, so these tests measure the reader, not the size of the suite's heap. + enabled = gc.isenabled() + gc.collect() + gc.disable() try: - yield value + value = Harness(monkeypatch) + try: + yield value + finally: + value.cleanup() finally: - value.cleanup() + if enabled: + gc.enable() def test_reader_construction_status_check_before_ready_and_unstarted_close_are_inert(harness):