Skip to content

Homie 5 compliance, phase 1: producer wire correctness - #96

Merged
dcj merged 10 commits into
mainfrom
dcj/homie-compliance-phase1
Oct 8, 2026
Merged

dcj merged 10 commits into
mainfrom
dcj/homie-compliance-phase1

Conversation

@dcj

@dcj dcj commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Homie 5 compliance, phase 1: what a third-party Homie 5 controller can observe from an SDK device (#95, part of #91). Each item, with convention line citations, is in the CHANGELOG [Unreleased] entry. One commit per area:

Commit Area
4d2522c, d9f3303 Teardown: stop() announces disconnected for the whole tree; convention-order child removal; delete() then stop() no longer re-creates the device; the Last Will carries QoS and retain
cfb14c4 $description: content-hash version; changes only in init; no "type": null; json $format as a string; core keys protected from description_extras
0c68184 Values: refuse what the datatype cannot represent; datetime, duration and color encoders; 0x00 only for strings; event properties at QoS 0, never replayed; publish_alert() / clear_alert()
eff6f77 /set: validated per datatype (rounded to step, then min/max); a retained /set is ignored; supports_target inert; .ebus-spec.json no longer claims mdns-discovery
b07b65a Review fixes: no init flap when structure is unchanged; ASCII-only payload grammars; float edge cases; datetime payload checks; refresh_tree() ordering
b83a3b1 Floor ebus-mqtt-client>=0.7.0 (with_retain)

Behavior changes users will notice

  • stop(announce=False) no longer disconnects cleanly. On an SDK-owned client it drops the connection without an MQTT DISCONNECT, so the broker publishes the Last Will (lost). The convention requires disconnected before a clean disconnect (:281) and defines lost as a bad disconnect (:284). It uses only paho's public API (loop_stop(), then shutting down socket()).
  • Invalid outbound values publish nothing. set_value() returns False and logs reason=propertyValueRefused. Previously, nan, True on a float, and out-of-range integers reached the wire.
  • Invalid producer input (no datatype, an enum or color without format, and similar) warns once per object and is still published; it raises in the next minor release, per Homie 5 compliance: convention-compliant base layer, eBus policy above it #91.
  • $description version is now a content hash, not a timestamp, so an unchanged document keeps its version across reconnects.
  • Invalid /set payloads never reach the callback, and a /set carrying the retain flag is ignored.
  • The examples now call stop() on Ctrl-C. Before, exiting left the root lost and the children ready.

Verification

  • Tests: 789 pass with 1 skipped (it needs numpy), on Python 3.10 with ebus-mqtt-client 0.7.0 from PyPI in a fresh venv. 83 tests are new in tests/test_homie_compliance.py; each review fix's test fails on the pre-fix code. ruff check and format are clean.
  • Examples on mosquitto 2.1.2:
    • simple-tree-device: all six devices are retained ready while running, and all six are disconnected after Ctrl-C.
    • simple-device and utility-meter: ready, then disconnected.
    • No broker drops and no tracebacks in any of the three.
  • Downstream producers against this branch: each matches its result on main.
    • distribution-enclosure-simulator: 456 passed.
    • utility-meter-simulator: 4 passed.
    • cta2045-proxy: 13 passed.
    • ableedge-proxy: 89 passed; its 30 failures occur on main too.

🤖 Generated with Claude Code

https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm

dcj and others added 10 commits October 8, 2026 11:54
Lifecycle and teardown items of #95 (part of #91):

- stop() publishes $state=disconnected for every device in the tree,
  descendants first and the root last (the root's publish is the flushed
  one), before the clean disconnect. convention.md:280-281 requires each
  device to send disconnected before disconnecting cleanly, and the
  root-to-child cascade at :262-267 covers only lost.
- child.delete() follows :635-641: the parent re-announces first (init,
  $description without the child, ready), then the child's topics are
  cleared starting with $state. The removed subtree is still cleared
  leaves first, so no device is seen whose parent is already gone.
- delete() marks the subtree deleted. stop() on a deleted root announces
  nothing and disconnects cleanly in both modes, and refresh_tree() on a
  deleted device republishes nothing, so neither re-creates it.
- will() carries the tree QoS and retain=True.

The declare_lost() + stop(announce=False) sequence: the convention
forbids a clean disconnect that is not preceded by disconnected (:281),
and lost is defined as a bad disconnect (:284). Publishing disconnected
would contradict the declared lost, so the teardown must not be clean.
stop(announce=False) now ends an SDK-owned connection without an MQTT
DISCONNECT (stop paho's network loop, then shut the socket down under
it), so the broker publishes the will, which re-asserts the same
retained lost. This keeps declare_lost() meaningful (lost is published
while the link is up, flushed) and makes an unpaired silent stop end in
lost instead of the stale ready it used to leave. The alternative of
keeping the clean disconnect and documenting a deviation from :281 was
rejected because the convention is meant to hold in the base layer.
ebus-mqtt-client has no unclean-stop API, so the helper reaches the
owned client's paho instance and degrades to a warning if it cannot.
On an injected transport the SDK closes nothing; the docs tell the
caller to drop the connection the same way.

Tests: tests/test_homie_compliance.py (lifecycle section). No existing
test pinned the old behavior.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm
$description items of #95 (part of #91):

- version is the top 52 bits of the existing content hash
  (_description_content_hash, version excluded), so it changes exactly
  when the document does and stays put across reconnects (:215).
- publish_description() wraps a content change on a ready OR sleeping
  device in init and restores the prior state (:207); a reconnect
  republish of unchanged content is a plain republish with no init.
  _notify_structural_change() routes through it.
- New _reconfiguration() scope: add_property, add_node,
  delete_property, remove_node and delete_node outside a transition on
  a ready/sleeping device publish init, the changed values, the new
  $description, then the prior state (:279, :611). Inside a transition
  they join it, as before.
- type is omitted when unset for devices and nodes (:218, :305);
  retained/settable are booleans; a dict json format is published as a
  string (:395).
- Per the #91 invalid-input policy (warn once per object, still
  publish): missing datatype (omitted from the document), retained=None
  (published as false, matching the non-retained wire behavior it
  always had), enum/color without format, enum format with empty or
  duplicate values, color format with unknown entries.
- description_extras drops Homie core fields with a warning, so an
  extra can no longer add root/parent to a root (:220).
- Device.invalidate_description_cache(); clear_retained_topic() on the
  device's own $description and delete_all_from_mqtt() reset the hash so
  the next publish is not suppressed.

publish_description() now composes the document before Device.publish()
and catches a compose failure itself, preserving the best-effort refresh
that test_refresh_tree_forwards_force_to_children relies on.

Tests: tests/test_homie_compliance.py ($description section). No
existing test pinned the old behavior.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm
Property-value items of #95 (part of #91):

- Outbound values are encoded per datatype and refused (publish
  nothing, log reason=propertyValueRefused, set_value() returns False)
  when the datatype cannot represent them: non-finite floats (:99),
  bool on integer/float, integers not whole or outside 64 bits
  (:87-89), float strings outside the payload grammar (no '+' in the
  exponent, :97), enum values not in $format (:111), json that is not an
  array or object or that contains NaN (:150). The refusal leaves the
  last good retained value in place.
- Encoders: datetime/date as ISO 8601 (:132), timedelta as PTxHxMxS
  with days folded into hours (:137-144), a color component tuple in the
  first $format color type, and color strings checked for type, count
  and range (:118-123).
- 0x00 empty-string encoding only for string (:65-67); "" on any other
  datatype is refused by its encoder.
- Non-retained properties publish non-retained at QoS 0 (:50); the
  Node.publish republish walk and add_property's structural publish
  skip them (:695); clear_value() and delete_all_from_mqtt() publish
  nothing for them, since there is no retained value to retract.
- $alert: Device.publish_alert(alert_id, message) publishes retained to
  $alert/<alert-id> (:517), clear_alert() deletes it, alerts are
  tracked so refresh_tree() republishes them and delete() clears them
  (:289). publish("$alert", ...) without an id now warns and publishes
  nothing; an id breaking the topic structure is refused, one outside
  the ID format is warned about per the #91 input policy.

Tests: tests/test_homie_compliance.py (values section). No existing
test pinned the old behavior.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm
/set and metadata items of #95 (part of #91):

- _settable_callback validates the payload for the datatype before the
  callback: json must be an array or object (then the existing $format
  JSONschema check), integer/float follow the payload grammar, numbers
  are rounded to the format step with floor(x + 0.5) from min, else
  max, else the current value, and only then checked against min/max
  (:397-413), enum membership, boolean exactly true/false, color and
  duration grammar. Rejections log reason=propertySetRejected. A number
  that rounding changed is handed to the callback in rounded payload
  form; otherwise the payload text is unchanged. 0x00 decodes to ""
  only for string.
- A /set delivered with the retain flag is ignored with a warning
  (:48, :483). set_subscribe() passes with_retain=True when the
  transport's subscribe() has that parameter (ebus-mqtt-client 0.7.0,
  electrification-bus/ebus-mqtt-client#24) and otherwise subscribes as
  before with a debug line, so the SDK keeps working with older clients
  and injected transports. The dependency floor stays at 0.6.0.
- supports_target=True warns once at construction and the /set path no
  longer calls the publish_target_value() stub, which ran before the
  acceptance decision (:466, :470).
- .ebus-spec.json drops mdns-discovery from supports and lists it with
  the excluded features in notes.

Tests: tests/test_homie_compliance.py (/set section), including an
end-to-end delivery through the installed ebus-mqtt-client that adapts
to whether it supports with_retain. Suite run against ebus-mqtt-client
0.5.0 (installed) and 0.7.0 (source on PYTHONPATH). No existing test
pinned the old behavior.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm
Pre-existing format debt; ruff format --check does not reach the
extension-less example scripts unless they are named explicitly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm
Follow-up to the lifecycle commit, found running against mosquitto:
after delete() on the root, stop() announced nothing and disconnected
at once, so delete()'s QoS 2 clears still in flight were dropped and
the broker kept every device's retained $state=ready. The owned path
now publish_and_flush()es a re-clear of the root's $state (a no-op
deletion), which drains everything queued before it ahead of the
disconnect. Verified live: delete() then stop() and delete() then
stop(announce=False) both leave no retained topics.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm
simple-device, simple-tree-device and utility-meter exited on SIGINT
without calling stop(), so the process exit dropped the connection and
the broker published the Last Will: every run ended with the root
retained as lost (and, in the tree, the children as ready). Each now
calls stop() on KeyboardInterrupt, which publishes $state=disconnected
for every device in the tree before the clean disconnect. The two
adapters gain a stop() for that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm
…ualifies

The $description version and change-gating sentences added with the
content-hash version were inserted ahead of the note about empty
transitions, separating it from its context. Moved to the end of the
paragraph; no wording change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm
…ammars

- A structural change made outside a transition publishes init/ready only
  when the $description actually changes. A property added to a node not
  yet attached to its device, or an identical re-declaration, publishes
  only its value (convention.md:616). Callers now mutate the tree before
  entering _reconfiguration(), which compares content hashes.
- refresh_tree() after a stale $description memo publishes init first and
  the device's state once, after its values and children; the republish
  path no longer wraps the document itself (:207, :279).
- Integer, float and duration grammars match ASCII digits only (:89, :97).
- Float encoder accepts decimal.Decimal, keeps a non-float Real's own text
  when it is a valid payload (numpy float32 rounded to 0.1 stays "0.1"),
  and refuses values beyond the float64 range (:95-96).
- datetime strings are checked as ISO 8601 on publish and on /set (:132).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm
0.7.0 exposes the MQTT retain flag (subscribe(..., with_retain=True)),
which the device side now uses to ignore a retained /set the broker
replays at every resubscribe. An injected transport without the parameter
still skips that check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T32pvgqCzp1qCz7K5gS9Wm
@dcj
dcj merged commit 72b5629 into main Oct 8, 2026
5 checks passed
@dcj
dcj deleted the dcj/homie-compliance-phase1 branch October 8, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant