Skip to content

OBSERVE subscribe burst bypasses the rate limit and can wedge a device at connect #53

Description

@QuiteYellow

Symptom

After a container recreate the dryer connects, seeds, and then stops answering entirely. Polls time out continuously until the unreachable watchdog forces a reconnect about 170 seconds later. From 2026-08-18:

19:30:01  dryer  DTLS connected — subscribing 11 paths
19:30:03  dryer  seeded → 25 links; sensors live
19:30:05  dryer  sweep complete (25 links, 928ms)
19:30:10  dryer  poll /operational/state/vs/0 timeout (cooldown 5s)
          ... continuous ...
19:31:18  dryer  liveness: no successful poll in the liveness window
19:32:51  dryer  DTLS connected — subscribing 11 paths   (forced reconnect)

The session is fine for roughly five seconds and then the device goes silent. The oven did the same thing on a recreate on 2026-08-07.

Cause

subscribe() never paces. It calls _send_dgram directly at dtls_session.py:1099, so the only throttle on the registration burst is whatever the caller does between calls. mqtt_demo hardcodes one:

# mqtt_demo/bridge.py:387-389
for path in self.descriptor.observe_paths:
    sess.subscribe(path)
    time.sleep(0.05)

50ms is 20 requests per second:

rate
bridge subscribe loop 20 rps
_DEFAULT_RATE_LIMIT_RPS (dtls_session.py:93) 5 rps
dryer firmware ceiling, measured 2026-06-03 ~14 rps

So eleven registrations go out at four times the library's own declared limit and above the measured ceiling for that firmware, immediately followed by the seed read. The hardcoded sleep also ignores rate_limit_rps entirely, so tuning the limit has no effect on this path.

The same gap downstream

LocalThings has the same loop with no delay at all, in observe.py:239-246:

for href in hrefs:
    segs = [s for s in href.strip("/").split("/") if s]
    try:
        session.subscribe(segs)

Unbounded rate. That burst runs at connect, which is when appliances in #37 fail.

Two independent callers both got this wrong, which is the real finding. Pacing is currently a caller contract enforced nowhere, and pace() being public is not enough to make callers use it correctly. mbillow reached the same conclusion from the write side in mbillow/localthings#384.

Fix

#51 already does most of this. It paces the block GET including block zero, post(), and delete() inside the session rather than leaving it to callers, and it correctly leaves the reader thread's auto-ACK (:667), ping() (:1050), and the teardown deregister (:464) alone, because delaying an ACK makes the device retransmit.

subscribe() is the request path it does not cover, and it is the one that produced the wedge above. It needs the same treatment.

#51 currently sits fourth in a stack behind #48, #49, and #50, so the pacing commit (75282c2) may be worth rebasing onto main on its own. It is a three-hunk change and does not depend on the rest of that stack.

Interim, mqtt_demo should call sess.pace() instead of time.sleep(0.05) so it at least honours the configured limit.

Not #10

#10 tunes the ceiling per descriptor and makes within-transfer Block2 pacing conditional on it. It assumes pacing is applied and asks what the value should be. This is a path that applies no pacing at any value, so the two are complementary; fixing this one makes #10's per-descriptor ceilings actually reach the subscribe burst.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions