Skip to content

fix(schema-1): per-message cost no longer grows with the device tree - #224

Merged
cayossarian merged 2 commits into
prerelease/3.8from
fix/dispatch-perf
Oct 30, 2026
Merged

cayossarian merged 2 commits into
prerelease/3.8from
fix/dispatch-perf

Conversation

@cayossarian

Copy link
Copy Markdown
Member

Refs #223.

Every MQTT message on a parent/child panel paid for two things that grow with the tree:

  • Routing. ControllerRoutes.dispatch tested each recorded subscription in turn with topic_matches_sub, which builds a fresh matcher per call. That's four routes per device, and the panel's own topics, recorded first, are matched last.
  • Readiness. SpanMqttClient._on_message asks is_ready() before and after each message, and every call walked the declared tree twice.

Changes

  • fix(transport): the recorded patterns live in one paho MQTTMatcher. Dispatch looks up the matching patterns in the typed route table, and the most recently recorded one still wins, decided by a recording number. _release indexes the new pattern once instead of once per held topic. paho leaves paho/mqtt/matcher.py unannotated, so mypy gets untyped_calls_exclude = ["paho.mqtt.matcher"], with a comment on the one shape that keeps its Any contained.
  • fix(adapter): is_ready() keeps its answer and forgets it on the Controller's own events: a device's first $state, a change of state, a removal, a $description, and the panel's info/model value. The next call works it out again. The tree walk is unchanged.
  • client.py is unchanged.

Tree changes stay correct:

  • Commissioning or decommissioning arrives as a $description or a removal, which updates the routes and clears readiness.
  • A rename in the SPAN app is only an info/name value. It doesn't touch routes or readiness, and it still reaches the snapshot.
  • A rebuilt adapter starts with no remembered answer.

Measured

Per message, both is_ready() calls plus handle_message, on an M-series Mac. The reference tree was grown by cloning circuits.

Devices Before After
14 79–115 µs ~2 µs
30 87–246 µs ~2 µs
46 97–378 µs ~2 µs

Not yet measured on a CM4; the issue reporter has offered to test.

Tests

  • Transport: the most recent route beats a more specific one, unsubscribing leaves a sibling or longer pattern routing, and a dropped route that is recorded again picks up what was held.
  • Readiness: after every message, the answer is compared with a fresh adapter fed the same messages. That covers 5 arrival orders and 5 tree-change scenarios. A reading or a rename causes no tree walk.

Falsified:

  • A first-match dispatch fails only the new overlap test; the existing ordering tests can't tell recency from specificity.
  • Leaving removed patterns in the matcher fails 4 transport tests.
  • Disabling each of the five readiness events in turn fails at least one test each time.

Full suite: 1872 passed, 7 skipped (live capture absent), 9 xfailed. All pre-commit hooks pass.

ControllerRoutes.dispatch tested every recorded subscription in turn
with paho's topic_matches_sub, which builds a fresh matcher for each
call. Tree-rooted discovery records four routes per device, so every
message cost grew with the panel, and the panel's own topics, recorded
first and matched last, cost the most.

The recorded patterns now live in one paho MQTTMatcher, and dispatch
resolves the patterns that match through the typed route table. The
most recently recorded match still wins, decided by a recording number
rather than by the matcher's order, and a pattern is removed from the
matcher only when it was recorded. _release indexes the new pattern
once instead of once per held topic.

paho leaves its matcher unannotated inside a py.typed package, so mypy
exempts calls into paho.mqtt.matcher. What the matcher yields is used
only as a key into the typed table.

The new overlap test is the only one a first-match matcher fails; the
existing ordering tests cannot tell recency from specificity.

Refs #223
The transport asks is_ready() before and after every message, and each
call walked the declared tree twice, through _awaiting_descriptions and
Controller.is_tree_complete. Readiness cannot change on an ordinary
value, so on a quiet tree that work bought nothing.

The adapter now keeps its last answer and forgets it on the Controller's
own events: a device's first $state, a change of state, a removal, a
$description, and the panel's info/model value. The next call works it
out again, never the event itself, because a description is announced
before the SDK has reconciled the children it declares. The walk is
unchanged, moved to _tree_ready().

Each readiness test asks after every message, as the transport does,
and compares with a fresh adapter fed the same messages. The replays
cover five arrival orders, and the scenarios cover the panel leaving
ready, a child removed and returning, a child declared and withdrawn,
the model's declaration changing, and a childless panel whose $state
arrives last. Disabling any one of the five events fails at least one
of them; the discovered event is caught only by the childless panel.

Refs #223
@cayossarian
cayossarian merged commit 52098f8 into prerelease/3.8 Oct 30, 2026
5 checks passed
@cayossarian
cayossarian deleted the fix/dispatch-perf branch October 30, 2026 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant