Feat/ivan/pure modules#2469
Draft
leshy wants to merge 10 commits into
Draft
Conversation
…e + health A PureModule declares when it runs (one tick() input), how every other input is sampled at that moment (latest/interpolate/window), and one pure step() bound to inputs by parameter name. The same class runs live on pubsub ports (NullStore-bridged, recording = store choice) or offline over stored memory2 streams via Module.over() — lazy, exact, chainable into other modules. Live ticks flow through a selectable BackpressureBuffer (KeepLast default); every queue in the path is bounded, incl. a max_pending cap so a dead interpolate() input can't accumulate ticks. Health follows "drops are metrics, not errors": per-reason drop counters, a 1 Hz _health stream in the module store, one warmup line comparing observed vs expected input rates, transition-logged DEGRADED/STALLED contract messages with throttled reminders, and strict mode for offline replay determinism.
Executable walkthrough (md-babel) in docs/usage/pure_modules.md: synthetic recording, first module with interpolation proof, sampler language, missing-data policy, explicit state, module chaining, live deployment notes, and a fake-clock health contract demo. Linked from the usage TOC and the puremodule.md reference.
❌ 2 Tests Failed:
View the top 2 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
…module MarkerDetectionPureModule sits next to MarkerDetectionStreamModule (untouched): the per-frame TF lookup becomes a declared interpolate(tolerance=0.5) camera_pose input, QualityWindow/SpeedLimit config knobs become upstream stream composition, and the emit_empty_frames sentinel plumbing reduces to step() returning the (possibly empty) Detection3DArray. smoothing_window is deliberately out of scope — it is recurrent state and maps to an explicit Mealy state parameter. Also: PureModule.offline() now returns Self.
Runnable Navigator example in the gentle intro (two Out ports, partial emission, slicing one output back out of the offline dict rows) and a proper Outputs section in the reference covering single/multi/None/ partial semantics, where outputs land per mode, and the dict-row vs per-port asymmetry as a known open point. Also fixes a map_data misuse hiding in a skip block — the callable receives the observation, not data.
docs/usage/pure_modules.md is now the single user-facing document: runnable tutorial followed by a Reference part with the exact rules (declaration/binding, alignment semantics, outputs, over(), deployment knobs, health). dimos/memory2/puremodule.md shrinks to design notes — the why behind ticks/backpressure/health, replay fidelity under drops, the state persistence journal plan, the run-handle plan, and the deferred list. No more duplicated tables; architecture.md links both.
The detached HealthMonitor in the contracts section looked like it was monitoring something; say explicitly that modules construct their own from config at start() (module.health_monitor), and the demo hand-feeds one to show the messages a deployed Follower would produce.
…iter Health contracts gain scale-free forms: max_drop_ratio (the step keeps up, independent of deployment rates), max_missing_ratio (the hardcoded >50% staleness rule made configurable), and max_tick_latency_s over a new end-to-end latency metric — trigger arrival to outputs published, covering queue wait + alignment + buffer + step. Latency is the felt consequence of queue growth under any backpressure policy, so queue depth stays a diagnostic gauge rather than a contract. Ratio contracts evaluate only above ratio_min_samples: tiny windows are noise and zero traffic passes vacuously, so the absolute contracts remain the liveness floor. Multi-output ergonomics: the reserved `out` writer parameter on step — assignment emits, skipping a port stays quiet, unknown ports raise at the assignment line, last write wins. With `out`, stateless steps return None and stateful steps return just new_state (no more (state, dict) tuple). The dict return stays as the low-level equivalent; single-output bare return is unchanged. Outputs is exported for annotation and is the seam for typed Out bundles later — design notes record the agreed bundle/structural-wiring direction, the flat-syntax-as-anonymous-bundle compatibility rule, and the contracts-on-inputs-vs-outputs rationale. Docs: health/contracts section rewritten from the deployment perspective around a captured log transcript; contracts table in the reference; Navigator example switched to the writer form.
Usage doc gains the second captured transcript — a heavy step that keeps the absolute 10 Hz contract while drop-ratio and tick-latency catch it — plus ratio/latency examples in the deployment block and recompute semantics for projected multi-output streams. Design notes: the agreed nested In/Out bundle design (annotation injection, nested-XOR-flat rule, binding by annotation, tick-row synergy, stated trade-offs).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.