Skip to content

Feat/ivan/pure modules#2469

Draft
leshy wants to merge 10 commits into
mainfrom
feat/ivan/pure-modules
Draft

Feat/ivan/pure modules#2469
leshy wants to merge 10 commits into
mainfrom
feat/ivan/pure-modules

Conversation

@leshy

@leshy leshy commented Jun 11, 2026

Copy link
Copy Markdown
Member

No description provided.

leshy added 2 commits June 11, 2026 18:48
…e + health

A PureModule declares when it runs (one tick() input), how every other
input is sampled at that moment (latest/interpolate/window), and one pure
step() bound to inputs by parameter name. The same class runs live on
pubsub ports (NullStore-bridged, recording = store choice) or offline
over stored memory2 streams via Module.over() — lazy, exact, chainable
into other modules.

Live ticks flow through a selectable BackpressureBuffer (KeepLast
default); every queue in the path is bounded, incl. a max_pending cap so
a dead interpolate() input can't accumulate ticks. Health follows
"drops are metrics, not errors": per-reason drop counters, a 1 Hz
_health stream in the module store, one warmup line comparing observed
vs expected input rates, transition-logged DEGRADED/STALLED contract
messages with throttled reminders, and strict mode for offline replay
determinism.
Executable walkthrough (md-babel) in docs/usage/pure_modules.md: synthetic
recording, first module with interpolation proof, sampler language,
missing-data policy, explicit state, module chaining, live deployment
notes, and a fake-clock health contract demo. Linked from the usage TOC
and the puremodule.md reference.
@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
1922 2 1920 155
View the top 2 failed test(s) by shortest run time
dimos.project.test_get_logger::test_no_get_logger
Stack Traces | 0.181s run time
def test_no_get_logger():
        """
        Fail if any file uses `= logging.getLogger` outside the whitelist.
        """
        violations = find_get_logger_usages()
        if violations:
            report_lines = [
                f"Found {len(violations)} forbidden use(s) of `logging.getLogger`. "
                "Use `setup_logger` instead:",
                "",
                "    from dimos.utils.logging_config import setup_logger",
                "",
                "    logger = setup_logger()",
                "",
                "If the usage is legitimate (e.g. standalone script, logging "
                "infrastructure, or third-party logger suppression), add it to the "
                "WHITELIST in dimos/project/test_get_logger.py.",
                "",
            ]
            for path, lineno, text in violations:
                report_lines.append(f"  {path}:{lineno}: {text.strip()}")
>           raise AssertionError("\n".join(report_lines))
E           AssertionError: Found 1 forbidden use(s) of `logging.getLogger`. Use `setup_logger` instead:
E           
E               from dimos.utils.logging_config import setup_logger
E           
E               logger = setup_logger()
E           
E           If the usage is legitimate (e.g. standalone script, logging infrastructure, or third-party logger suppression), add it to the WHITELIST in dimos/project/test_get_logger.py.
E           
E             dimos/memory2/tick.py:66: logger = logging.getLogger(__name__)

lineno     = 66
path       = 'dimos/memory2/tick.py'
report_lines = ['Found 1 forbidden use(s) of `logging.getLogger`. Use `setup_logger` instead:', '', '    from dimos.utils.logging_config import setup_logger', '', '    logger = setup_logger()', '', ...]
text       = 'logger = logging.getLogger(__name__)'
violations = [('dimos/memory2/tick.py', 66, 'logger = logging.getLogger(__name__)')]

dimos/project/test_get_logger.py:124: AssertionError
dimos.robot.test_all_blueprints_generation::test_all_blueprints_is_current
Stack Traces | 2.55s run time
def test_all_blueprints_is_current() -> None:
        root = DIMOS_PROJECT_ROOT / "dimos"
        all_blueprints, all_modules = _scan_for_blueprints(root)
    
        common = set(all_blueprints.keys()) & set(all_modules.keys())
        assert not common, (
            f"Names must be unique across blueprints and modules, "
            f"but these appear in both: {sorted(common)}"
        )
    
        generated_content = _generate_all_blueprints_content(all_blueprints, all_modules)
    
        file_path = root / "robot" / "all_blueprints.py"
    
        if "CI" in os.environ:
            if not file_path.exists():
                pytest.fail(f"all_blueprints.py does not exist at {file_path}")
    
            current_content = file_path.read_text()
            if current_content != generated_content:
                diff = difflib.unified_diff(
                    current_content.splitlines(keepends=True),
                    generated_content.splitlines(keepends=True),
                    fromfile="all_blueprints.py (current)",
                    tofile="all_blueprints.py (generated)",
                )
                diff_str = "".join(diff)
>               pytest.fail(
                    f"all_blueprints.py is out of date. Run "
                    f"`pytest dimos/robot/test_all_blueprints_generation.py` locally to update.\n\n"
                    f"Diff:\n{diff_str}"
                )
E               Failed: all_blueprints.py is out of date. Run `pytest dimos/robot/test_all_blueprints_generation.py` locally to update.
E               
E               Diff:
E               --- all_blueprints.py (current)
E               +++ all_blueprints.py (generated)
E               @@ -180,6 +180,7 @@
E                    "local-planner": "dimos.navigation.nav_stack.modules.local_planner.local_planner.LocalPlanner",
E                    "manipulation-module": "dimos.manipulation.manipulation_module.ManipulationModule",
E                    "map": "dimos.robot.unitree.type.map.Map",
E               +    "marker-detection-pure-module": "dimos.perception.fiducial.marker_detection_pure_module.MarkerDetectionPureModule",
E                    "marker-detection-stream-module": "dimos.perception.fiducial.marker_detection_stream_module.MarkerDetectionStreamModule",
E                    "marker-tf-module": "dimos.perception.fiducial.marker_tf_module.MarkerTfModule",
E                    "mcp-client": "dimos.agents.mcp.mcp_client.McpClient",
E               @@ -207,6 +208,7 @@
E                    "pgo": "dimos.navigation.nav_stack.modules.pgo.pgo.PGO",
E                    "phone-teleop-module": "dimos.teleop.phone.phone_teleop_module.PhoneTeleopModule",
E                    "pick-and-place-module": "dimos.manipulation.pick_and_place_module.PickAndPlaceModule",
E               +    "pure-module": "dimos.memory2.puremodule.PureModule",
E                    "quest-teleop-module": "dimos.teleop.quest.quest_teleop_module.QuestTeleopModule",
E                    "ray-tracing-voxel-map": "dimos.mapping.ray_tracing.module.RayTracingVoxelMap",
E                    "real-sense-camera": "dimos.hardware.sensors.camera.realsense.camera.RealSenseCamera",

all_blueprints = {'alfred-nav': 'dimos.robot.diy.alfred.blueprints.alfred_nav:alfred_nav', 'coordinator-basic': 'dimos.control.blueprin...sian_ik_mock', 'coordinator-cartesian-ik-piper': 'dimos.control.blueprints.teleop:coordinator_cartesian_ik_piper', ...}
all_modules = {'alfred-high-level': 'dimos.robot.diy.alfred.effector_high_level.AlfredHighLevel', 'arm-teleop-module': 'dimos.teleop..._navigation.BBoxNavigationModule', 'b1-connection-module': 'dimos.robot.unitree.b1.connection.B1ConnectionModule', ...}
common     = set()
current_content = '# Copyright 2025-2026 Dimensional Inc.\n#\n# Licensed under the Apache License, Version 2.0 (the "License");\n# you m...ebsocket_vis_module.WebsocketVisModule",\n    "zed-camera": "dimos.hardware.sensors.camera.zed.camera.ZEDCamera",\n}\n'
diff       = <generator object unified_diff at 0xff67e2879f20>
diff_str   = '--- all_blueprints.py (current)\n+++ all_blueprints.py (generated)\n@@ -180,6 +180,7 @@\n     "local-planner": "dimos...le.RayTracingVoxelMap",\n     "real-sense-camera": "dimos.hardware.sensors.camera.realsense.camera.RealSenseCamera",\n'
file_path  = PosixPath('.../dimos/robot/all_blueprints.py')
generated_content = '# Copyright 2025-2026 Dimensional Inc.\n#\n# Licensed under the Apache License, Version 2.0 (the "License");\n# you m...ebsocket_vis_module.WebsocketVisModule",\n    "zed-camera": "dimos.hardware.sensors.camera.zed.camera.ZEDCamera",\n}\n'
root       = PosixPath('.../dimos/dimos/dimos')

dimos/robot/test_all_blueprints_generation.py:66: Failed

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

leshy added 8 commits June 11, 2026 19:26
…module

MarkerDetectionPureModule sits next to MarkerDetectionStreamModule
(untouched): the per-frame TF lookup becomes a declared
interpolate(tolerance=0.5) camera_pose input, QualityWindow/SpeedLimit
config knobs become upstream stream composition, and the
emit_empty_frames sentinel plumbing reduces to step() returning the
(possibly empty) Detection3DArray. smoothing_window is deliberately out
of scope — it is recurrent state and maps to an explicit Mealy state
parameter. Also: PureModule.offline() now returns Self.
Runnable Navigator example in the gentle intro (two Out ports, partial
emission, slicing one output back out of the offline dict rows) and a
proper Outputs section in the reference covering single/multi/None/
partial semantics, where outputs land per mode, and the dict-row vs
per-port asymmetry as a known open point. Also fixes a map_data misuse
hiding in a skip block — the callable receives the observation, not data.
docs/usage/pure_modules.md is now the single user-facing document:
runnable tutorial followed by a Reference part with the exact rules
(declaration/binding, alignment semantics, outputs, over(), deployment
knobs, health). dimos/memory2/puremodule.md shrinks to design notes —
the why behind ticks/backpressure/health, replay fidelity under drops,
the state persistence journal plan, the run-handle plan, and the
deferred list. No more duplicated tables; architecture.md links both.
The detached HealthMonitor in the contracts section looked like it was
monitoring something; say explicitly that modules construct their own
from config at start() (module.health_monitor), and the demo hand-feeds
one to show the messages a deployed Follower would produce.
…iter

Health contracts gain scale-free forms: max_drop_ratio (the step keeps
up, independent of deployment rates), max_missing_ratio (the hardcoded
>50% staleness rule made configurable), and max_tick_latency_s over a
new end-to-end latency metric — trigger arrival to outputs published,
covering queue wait + alignment + buffer + step. Latency is the felt
consequence of queue growth under any backpressure policy, so queue
depth stays a diagnostic gauge rather than a contract. Ratio contracts
evaluate only above ratio_min_samples: tiny windows are noise and zero
traffic passes vacuously, so the absolute contracts remain the liveness
floor.

Multi-output ergonomics: the reserved `out` writer parameter on step —
assignment emits, skipping a port stays quiet, unknown ports raise at
the assignment line, last write wins. With `out`, stateless steps return
None and stateful steps return just new_state (no more (state, dict)
tuple). The dict return stays as the low-level equivalent; single-output
bare return is unchanged. Outputs is exported for annotation and is the
seam for typed Out bundles later — design notes record the agreed
bundle/structural-wiring direction, the flat-syntax-as-anonymous-bundle
compatibility rule, and the contracts-on-inputs-vs-outputs rationale.

Docs: health/contracts section rewritten from the deployment perspective
around a captured log transcript; contracts table in the reference;
Navigator example switched to the writer form.
Usage doc gains the second captured transcript — a heavy step that keeps
the absolute 10 Hz contract while drop-ratio and tick-latency catch it —
plus ratio/latency examples in the deployment block and recompute
semantics for projected multi-output streams. Design notes: the agreed
nested In/Out bundle design (annotation injection, nested-XOR-flat rule,
binding by annotation, tick-row synergy, stated trade-offs).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant